PARIS performance. The following figure shows how the benchmark

PARIS enables users to choose the
right VM type by providing runtime cost and performance cost on different VM
types. PARIS is a light weight service. A user provides a PARIS system a
workload, desired performance metric and a set of VM types. A user gives a
workload in the form of representative task. It is a sample code with input
data. A performance metric is a pre-defined metric code which implements
function of these metric in PARIS. Depending on the given VM types PARIS
predicts the runtime and performance cost of those VM types. A user will choose
the right VM type based on this information. The interaction with the user
looks like as following.PARIS model two things 1) workload
resource requirement and 2) impacts of VM types on workload. PARIS divides
modelling task into two phases: an offline extensive phase and an online,
inexpensive phase to avoid the performance cost overhead. Figure 4 shows the
architecture of PARIS. In figure you can see two phases.In this phase, PARIS uses a The
Profiler. Each benchmark task performance for a different metrics is recorded
by The Profiler. PARIS then uses the profiler to run a benchmark on different
Virtual Machine types. A benchmark is used to select a workload patterns. These
workload patterns are chosen to be diverse in terms of their types. If the new
system for Virtual Machine is introduced, then one only need to rerun the
benchmark on the new VM types. By doing this, offline phase removes extensive
profiling for predicting the workload performance. The following figure shows
how the benchmark workloads are chosen.  For benchmark workload we used
OLAP-style queries as shown in figure. We then added OLTP style latency
sensitive queries. Which YCSB core benchmark with MongoDB, Redis, Cassandra,
and Aerospike. Finally, we constructed common cloud hosted resource intensive
applications for compression. Profiler records a performance, cost and resource
usage during each run. To capture the performance statistics and resource
counter Ganglia instrument is used. To capture 20 resource counters the
following categories are used. ·        
CPU utilization·        
Network utilization ·        
Disk utilization·        
Memory utilization4.2 Online Performance Prediction: In online phase, user provides a desired
performance he wants. The user given performance metric is used to measure
cost. For example, we want to predict the 90th percentile
performance then we need to runt the task 10 times on each of the different VM
types. And then capture the 90th percentile performance on each of
these different Virtual Machine types. PARIS uses Fingerprint Generator that
runs a task on different VM types. In the process, it also uses the profiler as
described above to capture resource usage. Which then collects a runtime
measurement. These runtime measurements collect the resource usages of workload
from fingerprint generator. Now choose or take any two different VM types with
very different in their configuration to compare workload performance. This is
to compare workload performance in resource abundant and resource constrained
settings. In online performance prediction phase, fingerprinting requires
additional cost. This cost is small, and this cost is independent of Virtual
Machine types.  Fingerprint gives the information
on the resource usages and Virtual Machine configurations gives us the
information on which resources are available. These information is enough to
predict the workload performance. For example, if we run on large machine, the
profiler indicates that the task used 2 GB of memory. If we run on the machine
with 1 GB of memory, it runs poorly. Then if we run it on any machine with less
than 2 GB memory it runs poorly. If the task is doing lots of I/O operations
and it uses lot of time to be blocked by I/O operations, then I/O tasks might
be bottleneck. This kind of reasoning can be described as decision tree which
based on if-then-else conditions. The decision tree example is shown in the
following figure.  Making
decision tree for each of workload is quite difficult. That’s why we use the
data from offline benchmarking phase with random forest algorithms to create a
collection of decision tree for each workload.  1)     
Prediction Accuracy of PARIS:PARIS prediction accuracy is
evaluated first by comparing PARIS’ performance to the actual performance we
obtained by running the user provided task on all VM types. We evaluated PARIS
for video encoding task on AWS and AZURE. In this task we used runtime as the
target performance metric and OLTP workload as source performance metric using
latency and throughput.  Overall Prediction error:Figure shows a comparison of PARIS
prediction of baseline mean and 90th percentile for Runtime, Latency
and throughput. The final is the average across VM types. Standard deviations
are shown as error bars.