Dynamic Resource Provisioning in IaaS Cloud Environment

Transcription

1 Aalto University School of Science Degree Programme of Computer Science and Engineering Ramasivakarthik Mallavarapu Dynamic Resource Provisioning in IaaS Cloud Environment Master s Thesis Espoo, August 9, 2012 Supervisor: Instructor: Professor Antti Ylä-Jääski Yrjö Raivio Lic.Sc. (Tech.)

2 Aalto University School of Science Degree Programme of Computer Science and Engineering Author: Ramasivakarthik Mallavarapu Title: Dynamic Resource Provisioning in IaaS Cloud Environment ABSTRACT OF MASTER S THESIS Date: August 9, 2012 Pages: Professorship: Data Communication Software Code: T-110 Supervisor: Instructor: Professor Antti Ylä-Jääski Yrjö Raivio Lic.Sc. (Tech.) Elasticity is one of the key-enablers of cloud systems, minimizing the cost of resource provisioning while meeting critical Quality of Service (QoS) requirements of a service level agreement (SLA). Most internet based services have SLA s that demand stringent performance requirements. Automated resource provisioning (AutoScaling) is an effective way of dealing with workload fluctuations by allocating resources based on the current demand. Simple reactive approaches to AutoScaling can have a contrasting affect on performance, while over-provisioning substantially increases the costs. To tackle these challenges, there is a need for intelligent resource provisioning mechanisms that can model, analyze and predict the resource demand. This thesis outlines the key practical issues involved in AutoScaling in an Infrastructure as a Service (IaaS) cloud environment and provides tangible solutions. We study a few prediction models and make a comparative analysis on their strengths and weaknesses. We then present the predictive elastic resource controller that addresses the issues in AutoScaling by using modeling techniques from statistical analysis. The research also identifies issues relating to resource demand and capacity estimation in a multi-tenant cloud environment. A prototype model of the predictive resource controller was implemented on an OpenNebula based cluster. Real world and artificial workload traces were used to test the efficiency of the model. We have also made a comparative analysis of our proposed model with a simple, reactive resource controller. Simulation results show that our model outperforms a simple, reactive resource controller in terms of prediction error, QoS and number of SLA violations. Keywords: Language: cloud, autoscaling, IaaS cloud, workload modeling, SLA, time series analysis English ii

3 Acknowledgements I would like to thank Professor Antti Ylä-Jääski for supervising my thesis and giving me the opportunity to work in the Data Communications Software research group, School of Science, Aalto University. I owe my greatest gratitude to my Instructor, Yrjö Raivio. Yrjö s patience and guidance helped me a lot in the time of research and writing of this thesis. I would also like to thank Koushik Annapureddy for many insightful conversations that improved my thesis. Last but not the least, I would like to thank my family for supporting me throughout. Espoo, August 9, 2012 Ramasivakarthik Mallavarapu iii

4 Abbreviations & Acronyms API AR ARIMA ARMA AWS CAPEX CBMG CSP EC2 ES GAE GUI HA IaaS IT KPI KVM MA MPE MSE NIST OPEX PaaS QoS RPC RUBiS SaaS SLA VM Application Programming Interface Autoregressive Autoregressive Integrated Moving Average Autoregressive Moving Average Amazon Web Services Capital Expenditure Customer Behavior Model Graph Communication Services Provider Elastic Compute Cloud Exponential Smoothing Google App Engine Graphical User Interface High Availability Infrastructure-as-a-Service Information Technology Key Performance Indicator Kernel based Virtual Machine Moving Average Mean Percentage Error Mean Squared Error National Institute of Standards and Technology Operational Expenditure Platform-as-a-Service Quality of Service Remote Procedure Call Rice University Bidding System Software-as-a-Service Service Level Agreement Virtual Machine iv

5 Contents Abstract Acknowledgements Abbreviations & Acronyms List of Tables List of Figures ii iii iv vii viii 1 Introduction Motivation Problem Statement Targets and Methodology Contribution Document Structure Background Cloud Computing Cloud Service Models Infrastructure as a Service Platform as a Service Software as a Service IaaS Cloud Deployment Models Public Cloud Private Cloud Hybrid Cloud Multi-tier Architectures Rice University Bidding System v

6 3 Design Overview Dynamic Resource Provisioning Framework Predictive Models for AutoScaling Mean Max Exponential Smoothing ARIMA Traffic Monitoring and Fault Detection Algorithm Implementation Test Cloud Setup Workload Generation Resource Provisioning Framework Performance Evaluation Workload Traces Google Trace Artificial Trace Workload Prediction Mean Max Exponential Smoothing ARIMA Prediction Accuracy RUBiS Simulation SLA Violations Discussion Limitations Cloud Reliability Future Work Conclusions 42 Bibliography 42 vi

7 List of Tables 3.1 ARMA parameters List of variables used in Algorithms 1 and Google workload trace Artificial workload trace RUBiS Simulation vii

8 List of Figures 1.1 Average traffic distribution [7] Google cluster workload trace [37] Animoto AutoScaling [34] Resource provisioning approaches Cloud Service Models Multi-tier Web Architecture Predictive Elastic Resource Controller Test setup Google cluster workload trace [37] Artificial workload trace Google workload trace - Mean 16 prediction Artificial workload trace - Mean 8 prediction Google workload trace - Max 16 prediction Artificial workload trace - Max 8 prediction Google workload trace - Exponential smoothing prediction Artificial workload trace - Exponential smoothing prediction Google workload trace - ARIMA prediction Artificial workload trace - ARIMA prediction Predictive and Reactive error comparison viii

9 Chapter 1 Introduction This chapter begins with a look at the motivation, which forms the basis for choosing this area and exploring further into it. Motivation also highlights the inefficiency of traditional models of resource provisioning, while pointing out the major challenges in simple reactive resource provisioning approaches. Problem statement defines the exact problem that this thesis aims to solve. We outline the major goals of this thesis and explain why the goals are extremely important in the context of resource provisioning. Finally, we discuss the most important contributions of this thesis and the document structure. 1.1 Motivation Large internet based enterprises such as ebay, Amazon and Netflix must adhere to very strict Service Level Agreement (SLA) requirements in terms of high availability, high throughput and low latencies. Performance degradation in terms of any of the above mentioned metrics may result in loss of user base and hence their revenue. Traditionally, enterprises tend to overprovision resources to contain the demand fluctuations in the workload. The problem with this approach is that the peak workloads vary from the average workload by a factor of as high as Provisioning resources for the average case means that the resources are not sufficient to handle the excess workload, which has an adverse affect on the Quality of Service (QoS). Provisioning for peak workloads results in gross underutilization, as excess resources remain idle most of the time. More often than not, internet based services experience a seasonal demand, with demand fluctuations during the course of the day, week, month and year. Optimizing infrastructure to the varying workload demand is a complex process [29]. Figure 1.1 depicts the 1

10 CHAPTER 1. INTRODUCTION 2 daily distribution of a typical internet application in the USA which is similar to the average SMS traffic distribution [7, 27]. Figure 1.1: Average traffic distribution [7] Infrastructure as a Service (IaaS) cloud services have provided a paradigm shift in the way resources are provisioned and utilized. AutoScaling enables cloud subscribers to scale out their infrastructure only when the workload demand spikes up and deprovision excess resources when the demand lowers. This is especially vital for enterprises that experience workload variations on a hourly, daily or weekly basis. Unexpected workload variations should also be handled seemlessly without a loss in QoS. Figure 1.2 is a real world workload trace from Google, depicting the variations in workload demand [28, 37].

11 CHAPTER 1. INTRODUCTION Workload trace 800 Message request rate(msg/sec) Day 1 Day 6 Day 11 Day 16 Day 21 Day 26 Day 29 Time - 29 Days Figure 1.2: Google cluster workload trace [37] Figure 1.3 shows a real world scenario where AutoScaling has been successfully used [34]. Animoto, a video animation company had successfully employed AutoScaling, in order to deal with a sudden surge of user demand.

12 CHAPTER 1. INTRODUCTION 4 Figure 1.3: Animoto AutoScaling [34] Unfortunately the above stated approach has some inherent flaws due to several issues. The simple reactive approach to AutoScaling as stated above, manages to optimize the resource utilization and reduces the cost of resources utilized, but fails to maintain the QoS during a period of increased workload demand. The following are some of the major challenges involved in the process of AutoScaling. Non trivial launch times of virtual machines Accurate capacity estimation of virtual resources Unexpected workload variations In order to address the above mentioned issues, there is a need for proactive mechanisms that can forecast changes in the workload demand and preemptively scale the infrastructure, which forms the basis for pursuing this thesis.

13 CHAPTER 1. INTRODUCTION Problem Statement We seek to address the following research questions in this thesis. In order to successfully utilize the many services offered by the cloud computing paradigm, we need to understand the relevant issues and address them. In this thesis work, we aim to highlight some of the most important problems in the process of AutoScaling, suggest pertinent models and frameworks that solve the problems and demonstrate the efficiency of suggested solutions through simulations. As mentioned earlier, the major challenges in successfully applying AutoScaling to enterprise grade applications are the unexpected workload variations and non trivial launch times of virtual machines. Figure 1.4 shows the need for dynamic on-demand resource provisioning mechanisms as opposed to the traditional model of fixed infrastructure resources. Load Average resource provisioning (fixed) Peak resource provisioningi (fixed) On-demand resource provisioning Resource Usage Work Load Figure 1.4: Resource provisioning approaches A simple reactive approach fails in this aspect because, the unexpected

14 CHAPTER 1. INTRODUCTION 6 workload variations may cause a sudden spike in the demand forcing the infrastructure to scale out. While the resource controller attempts to provision new resources, the existing infrastructure must cater to the excess workload demand, effecting the QoS of the system. Typically, virtual machine with an enterprise grade application will require about 7-10 minutes of time before it becomes operational. The existing infrastructure that handles the excess load during the launch time of new resources, may cause performance degradation and the excess workload may also have a ripple effect on the next few minutes after the newly launched resources become operational. So in order to tackle this issue, there has to be a proactive resource provisioning approach that can successfully detect workload variations and spawn new resources well before the demand spike occurs. In this thesis, we chose a stochastic model to forecast workload variations and make scaling decisions well in advance, so that the overall system performance is not affected. 1.3 Targets and Methodology The major goal of this thesis is to design an automated predictive elastic resource controller that optimizes the infrastructure according to the workload variations, preventing a loss in QoS while eliminating the need for overprovisioning. We identify issues in AutoScaling and address them with the help of our model. The other important aspect of this thesis is to build a framework for AutoScaling and perform simulations in order to compare different approaches to AutoScaling. Rice University Bidding System (RU- BiS) is employed as the use case for carrying out the simulations. RUBiS represents a complex web service, which can be deployed as a distributed multi-tier web service. Apart from predictive AutoScaling, we attempt to extend the validity of our model to address the issues in scaling distributed multi-tier applications that may deal with heterogeneous requests. The research methodology adopted in this thesis is literature study, implementation, simulation and analysis. 1.4 Contribution This thesis outlines the most important practical issues involved in AutoScaling and explores solutions to address those issues. We have attempted to practically demonstrate the issues and the subsequent effect on the QoS by implementing a simple reactive resource controller that scales resources on certain predefined conditions. The major contribution of this thesis is the

15 CHAPTER 1. INTRODUCTION 7 design and implementation of an automated predictive resource controller, that utilizes prediction models from statistical analysis. RUBiS was used as the experimental use case in all the simulations. We describe in detail, a variety of workload prediction models and make a comparative analysis on the prediction accuracy of chosen approaches using artificial and real world workload traces. In our evaluation, we have found that Autoregressive Integrated Moving Averages (ARIMA) model outperformed the other models described in this thesis work. We implemented an ARIMA based automated predictive resource controller on an OpenNebula cloud deployment and conducted simulations to demonstrate the effectiveness of our approach. We also try to point out the performance variations that may occur in a multi-tenant cloud environment, and provide a roadmap to address such issues. We argue that, online capacity estimation methods and tools such as Customer Behavior Model Graphs (CBMG) could be vital in understanding the intricacies involved in scaling a complex service. 1.5 Document Structure The thesis is documented in the following order. Chapter 2 touches up on the relevant background information on cloud computing, service models and also briefly introduces the use case in this thesis. In Chapter 3, the proposed dynamic resource provisioning framework is explained in detail, including various prediction models for workload prediction. Algorithm for AutoScaling is also illustrated in this chapter. Chapter 4 discusses in detail, the experimental setup, simulation methodology and also provides the relevant implementation details. Chapter 5 compares different prediction models with the help of artificial and real world workload traces. Chapter 5 also includes the comparative analysis of predictive and reactive approaches to AutoScaling. Chapter 6 provides an indepth discussion about the issues involved in AutoScaling and extends the validity of the chosen approach to complex scenarios. Chapter 7 concludes the thesis.

16 Chapter 2 Background This chapter explains the background information about cloud computing, service models in cloud computing and relevant details about simulation use case. 2.1 Cloud Computing Cloud computing broadly refers to resources and resource enablers delivered as services over the internet. In the context of cloud computing, resources could be software applications, while resource enablers could be the hardware and software stack required for running/building the applications [3]. According to National Institute of Standards and Technology (NIST), U.S. Department of Commerce, there are five essential characteristics of a cloud deployment model [24]. On-demand self-service: The capability to provision computational resources automatically as and when the need arises. Broad network access: Network access through standard mechanisms to allow heterogeneous clients to make use of the resources. Resource pooling: The cloud provider s resources are pooled to serve cloud subscribers in a multi-tenant model where free resources are dynamically assigned to subscribers requesting resources. The subscriber is generally abstracted from details like the datacenter location, specific configuration, failover mechanisms etc. Rapid elasticity: Cloud subscribers must be able to perform on-demand provisioning/deprovisioning of cloud resources. The cloud provider may 8

17 CHAPTER 2. BACKGROUND 9 provide an illusion of unlimited resources, so that the cloud subscriber may request for resources at any point in time. Measured service: Cloud resource usage must be transparent to the cloud provider as well as the cloud subscriber. There must be provisions to monitor, control and report on cloud resource usage. The advancements in the virtualization technology coupled with the pricing model adopted by several cloud providers has significantly changed the dynamics of infrastructure investment. The pay-as-you-go model is a first step in reducing the upfront capital expenses and converting it into operating expenses. Although the pricing model of certain public cloud providers may seem expensive, they bring in additional benefits such as elasticity and scalability thus providing ways to efficiently utilize the infrastructure, reducing the burden of over provisioning and mitigating the risks of under provisioning [3, 10, 33]. 2.2 Cloud Service Models Cloud services may be classified into different service models depending on the services offered to the cloud subscriber. Based on the delivery model, cloud computing can broadly be divided into three service models. Figure 2.1 presents a cloud computing stack and the three service models [19] Infrastructure as a Service Infrastructure-as-a-Service (IaaS) clouds provide computing, storage or network resources as services, delivered usually over the internet. The cloud subscribers, without an upfront commitment or capital expenditure may get access to IaaS cloud resources within a matter of few minutes or hours. Thus the subscribers may get rid of huge Capex costs and operational expenses as the IaaS cloud providers offer an usage based pricing model. The most prominent player in the Iaas cloud market is Amazon Web Services (AWS) followed by Rackspace, Gogrid etc. IaaS cloud providers may offer services as dedicated or multi-tenant physical or virtual infrastructure resources. AWS s elastic compute cloud, for instance, offers virtualized resources abstracting the physical and virtualization layers. The cloud subscribers get complete access to the virtual machines, from the choice of operating system to the application software installation. Although the IaaS cloud provider pricing is slightly higher than the private infrastructure pricing calculated over a sufficiently long period of time,

18 IaaS PaaS SaaS CHAPTER 2. BACKGROUND 10 Applications Application Services Execution Environment Middleware O/S Virtual Resources Virtualization Computation Storage Network Figure 2.1: Cloud Service Models the distinct advantage of IaaS cloud model is the elimination of initial capital expense and a significant reduction in operating expenses. Also, because of the pay-as-you-go pricing model, and the efficient and fast infrastructure scaling provisions offered by the cloud providers, the subscribers may choose to dynamically adapt the infrastructure to their fluctuating needs. This is extremely crucial especially for small and medium enterprises that cannot afford huge initial investment costs and also enables efficient utilization of cloud services, using the resources only when it is absolutely needed [14, 15] Platform as a Service Platform-as-a-Service (PaaS) provides an application development environment, abstracting the underlying hardware, virtualization and operating system layers. PaaS is essentially an aggregation of set of development tools that the developers need for building applications and services in the cloud. PaaS further simplifies the job of cloud subscribers, by hiding the complexities of hardware and software resource management, application deployment and

19 CHAPTER 2. BACKGROUND 11 dynamic scaling of infrastructure to cater to the growing application needs. Developers may use PaaS services to build applications that are hosted by the PaaS provider and offered as a service to the end users, usually over the internet. PaaS model enables the enterprises to focus only on the software development cycle involved in building the application, as the other aspects such as infrastructure management, dynamic scaling mechanisms are made transparent by the PaaS provider. Google App Engine (GAE), Microsoft Azure and Force.com are some of the most important PaaS cloud providers [5] Software as a Service Software-as-a-Service (SaaS) clouds provide an on-demand software applications as a service offering, where the subscriber may be allowed a need based usage policy and thus an usage based pricing model. In this model, the subscriber has limited control over the physical hardware, software stack, application execution environment and other factors unlike in IaaS or PaaS clouds. One of the most widely used applications like is conceptually a software delivered as a service. Google Docs is another example of application software delivered as a service over the internet. SaaS represents a paradigm shift in the software service model, especially in the end user sector, as it reduces the need for client side software installation and mandates less system requirements than in the traditional software service model [3]. 2.3 IaaS Cloud Deployment Models IaaS clouds offer lower level resources such as computational, network and storage as a service to the subscribers. IaaS clouds may be categorized primarily in to 3 deployment models Public Cloud Public cloud providers are organizations that provide IaaS services to one or more subscribers on a pay-per-use principle. The emergence of public clouds considerably reduces the Capex involved in purchasing and setting up the infrastructure for enterprises. Also, most of the public clouds provide a REST based API for automatic provisioning and deprovisioning of resources. This gives subscribers, the freedom to dynamically adapt the resources, provisioning them only when the workload request demand and deprovisioning them when the demand subsides. With the arrival of public clouds, the subscribers

20 CHAPTER 2. BACKGROUND 12 can afford to make short term plans for decisions on resource allocation and optimization. The most prominent public cloud providers are Amazon Web Services, Rackspace, Gogrid, IBM etc. The key enablers of such a model is the advancement of virtualization where virtual resources can be spawned in a matter of few minutes, usage-based pricing model and cloud provider APIs for automatic control of infrastructure. The downside of public cloud deployment model is that the subscribers are generally not in control of the underlying hardware and software stacks involved in infrastructure management, location of datacenters, security implications of multi-tenancy and country specific laws that may impose restrictions on sensitive information crossing geographical borders [3] [20] Private Cloud Conceptually a private cloud is very similar to a public cloud except that the infrastructure services offered are internal to the organization owning the infrastructure. Private cloud deployment model is an efficient way of managing large infrastructures. A private cloud essentially is an infrastructure management software solution that enables organizations to deploy resources effectively, monitor the health and performance of the infrastructure and eases the administrative tasks involved in managing large scale infrastructures [31]. Eucalyptus 1, OpenNebula 2, OpenStack 3 are three most visible players in the private cloud space. The most distinct advantage of private cloud is that the organizations have total control on the infrastructure unlike in a public cloud. The private cloud infrastructure is fixed and dedicated and hence cannot cater to workloads beyond a certain limit, while the public clouds provide an illusion of unlimited resources. Although private clouds do not bring a lot of value to an organization apart from providing ways and means to efficient infrastructure management, private cloud adoption eases the future migration to public cloud or usage of a hybrid architecture utilizing the private infrastructure as well as the public infrastructures services offered by a public cloud provider Hybrid Cloud Hybrid cloud is a composition of private and public cloud deployment models enabling data and application portability, and seeking ways to overcome the shortcomings of private cloud, while addressing the risks attached with

21 CHAPTER 2. BACKGROUND 13 the public clouds [24]. An ideal deployment scenario would work in such a way that the private cloud would be designed to handle the average workload, while the public cloud resources are provisioned dynamically to deal with the excess demand. The main advantage of a hybrid cloud is the flexibility of using a public cloud only when the demand grows beyond a certain threshold and switching back to private-only mode to handle the average case. This approach tries to address the major hindrances in using a public cloud such as security implications, restrictions imposed by country specific laws, multitenancy etc; while also taking into account, the burden of overprovisioning a private cloud. The main challenge of using a hybrid cloud is to strike the right balance between the private and public cloud components. Enterprises dealing with sensitive information such as medical records, telecommunication operators, government data etc must also explore ways in which the enterprises can offload some of the non-critical computational tasks to the public clouds [11]. 2.4 Multi-tier Architectures Multi-tier architectures have become the industry standard in designing scalable client-server applications. Traditionally, enterprises with in-house infrastructure deployment, used to have large scale dedicated servers to handle each tier of the application. But offlate, large dedicated servers gave way to smaller distributed virtual servers. Figure 2.2 represents a two tier architecture with a web front end and database. 2.5 Rice University Bidding System Rice University Bidding System (RUBiS) is an auction site benchmark that implements the core functionality of an auction site. It has provisions for selling, browsing and bidding items. RUBiS also implements different sessions for different type of users in the form of visitor, buyer and seller. A registered user can sell, buy and browse through the website, while a visitor is only allowed to browse through the different sections of the website. RUBiS is modeled after ebay.com with a web front end and a database that maintains the records of users, items, bids etc. A client with a web browser can perform in total 26 interactions that include browsing items by region or category, bidding, buying and selling of items. RUBiS represents a dynamic web application with several web pages that require interactions with the database. It is often used to study the application design patterns and

22 CHAPTER 2. BACKGROUND 14 Client 1 Web 1 DB 1 Client 2 Web 2 DB 2 Client 3 Web Load Balancer Web 3 DB Load Balancer DB Client n Web n DB n Figure 2.2: Multi-tier Web Architecture evaluate performance bottlenecks. In this thesis, we use RUBiS as the use case for all simulations [2].

23 Chapter 3 Design Overview In this chapter, we discuss about the resource provisioning framework, predictive models used in the framework and also touch upon different scaling algorithms. 3.1 Dynamic Resource Provisioning Framework The process of AutoScaling in the cloud, involves dealing with challenges on many fronts. A dynamic resource provisioning framework (resource controller henceforth) consists of several individual components collaborating with each other. A resource controller may have the following basic functionalities, Monitor: Traffic monitoring for detecting workload changes. Monitor function may also include monitoring the infrastructure in order to detect faults Analyze: Workload modeling and subsequent decisions on infrastructure scaling Act: Implementing the scaling decision prompted by the analyze function Designing an automated elastic resource controller that ensures high availability and carrier grade performance is a complex task. Considering the relatively long startup time of a virtual machine (VM), it is essential that the resource controller provisions resources well before the need arises. In this thesis, we consider two kinds of resource controllers: reactive and proactive. A reactive resource controller, implements the basic functionalities of 15

24 CHAPTER 3. DESIGN OVERVIEW 16 monitoring, analysis and based on certain preconditions, decides on the scaling. On the other hand, a proactive resource controller, includes a prediction function, to predict the future workload changes and implements the scaling decisions based on the predicted changes. Figure 3.1 depicts the resource controller framework [32]. Figure 3.1: Predictive Elastic Resource Controller Predictive Models for AutoScaling Workload demand data obtained as a result of periodic monitoring can be considered as a time series data. Analyzing the workload data with the help of predictive models for time series analysis, helps us make a short term prediction of workload demand. A carefully constructed predictive model, with an ability to predict accurately, the future workload changes, is essential for mitigating the problems of reactive resource provisioning. In this thesis, we have studied and analysed the following prediction models.

25 CHAPTER 3. DESIGN OVERVIEW Mean According to this scheme, the predicted demand would be the mean value of all the samples in a given window. In this method, also called as the moving averages method, the prediction is the result of averaging the recent most t values of the series. The main drawback of this method is that the peaks are dampened resulting in an underestimation error. The relative accuracy of the prediction depends on the window size considered and also the pattern of the workload trace. The following equation provides a mathematical representation [13]. n X(i) X(n + 1) = (3.1) i=n+1 t t Max In this method, the prediction value would be the maximum value among the samples in a given window. Given a reasonable window size, this method tries to provide a safe estimation by choosing the maximum value observed in the recent past. In the context of workload prediction, the goal is to estimate accurately, the peak values, as the emphasis is on ensuring QoS. The method is not at all efficient and at best prepares the system for worst case scenario, occurred in the recent past [13] Exponential Smoothing Moving averages model can be classified as a naive prediction model mainly used for smoothing the curve. Essentially, a moving average model dampens the peaks and elevates the troughs. In the context of AutoScaling through workload prediction, over-estimation errors and slightly over allocated resources are tolerable, while underestimation of peaks may result in under provisioning taking toll on the performance and QoS of the system. Weighted moving averages model, a slight improvement on the moving averages model, assigns weights to observations that occur in the recent past. The most recent observations get more weightage than older observations. Based on the underlying pattern of the time series, the weights can be estimated to reflect the pattern in future observations as well. Exponential smoothing is similar to weighted moving averages approach as it assigns exponentially decreasing weights to observations giving preference to the most recent observation [16]. This approach is also a simplistic estimation as it tries to create a trend based on the most recent observation. The mathematical representation of

26 CHAPTER 3. DESIGN OVERVIEW 18 the model is provided in equation 3.2. X (t) = α X(t 1) + (1 α) X t 1 (3.2) where 0 < α 1 t 3 The process of smoothing begins with an assumption where X (2) = X(1) Iteratively substituting the values of smoothed observations of the past in terms past observations of time series, the smooth observation is expressed as past observations with exponentially decreasing weights. X (t) = α [X(t 1) + (1 α) X(t 2) + (1 α) 2 X(t 3) + + (1 α) t 3 X(2)] + (1 α) t 2 X(1) (3.3) ARIMA Autoregressive moving average (ARMA) model is well suited for modeling stationary time series data. In order to fit non-stationary time series data, ARMA model is further generalized to include differencing to help deal with both stationary and non-stationary time series data. This class of models is referred to as the autoregressive integrated moving average (ARIMA) model. ARIMA model is essentially made up of autoregressive (AR) component of lagged observations, moving average (MA) of past error terms. AR component expresses the current observation as a function of past observations of the same time series whereas the MA component is influenced by past and current errors [6]. ARIMA(p,d,q) model is usually represented in terms of the number of AR parameters, differences and MA parameters used in building the model. The order, in case of AR and MA components represents the number of lags in terms of past observations or past errors. According to the principle of parsimony, simplistic models with fewer parameters are always preferred to complex models with several parameters [23]. Equation 3.4 represents a time series expressed in terms of AR(n) model. X (t) = µ + α 1 X(t 1) + α 2 X(t 2) + + α n X(t n) (3.4)

27 CHAPTER 3. DESIGN OVERVIEW 19 Equation 3.5 represents a time series expressed in terms of moving averages of white noise or error terms. X (t) = µ + β 1 ɛ(t 1) + β 2 ɛ(t 2) + + ɛ(t n) (3.5) In this thesis, we have chosen a single order ARMA model, altering the number of differencing operations between 0 and 2. Equation 3.6 represents a single order ARMA(1,1) model. X (t) = µ + α X(t 1) + β ɛ(t 1) + ɛ(t) (3.6) where µ = Mean (1 α) 0 < α 1 0 < β 1 Table 3.1: ARMA parameters. Parameters Description α, β ARMA parameters ɛ White Noise µ Constant Traffic Monitoring and Fault Detection Monitoring is essential in estimating the performance of virtual servers and observing changes in the workload traffic pattern. Monitoring, as a function, can be applied at the level of frontend load balancers in order to observe workload changes and at the infrastructure level, to estimate the performance of resources and detecting faults. Most public clouds, today are multi-tenant, allowing different subscribers, possibly competing subscribers, to run their virtual machines on the same physical host. While multi-tenancy certainly has some security implications, recent studies have suggested the possibility of performance implications as well. An overloaded virtual machine, may have an adverse effect on other virtual machines running on the same host. According to Barker and Shenoy [4], the performance of a virtual machine varies quite significantly when observed over longer periods of time. Hence infrastructure monitoring and fault detection are essential in maintaining high availability and a stable QoS.

28 CHAPTER 3. DESIGN OVERVIEW Algorithm In order to effectively handle the workload variations, the first and foremost step is to accurately estimate the load handling capacity of a server.typically, a complex web service like RUBiS must deal with heterogeneous web requests that include browsing through regions, reading bids, posting new items for auction etc. In other words, the requests could be HTTP GET or POST which may or may not involve a call to the database. In such a scenario, one obvious way is to estimate capacity as a function of number of online users. However, for simulations carried out in this thesis work, only HTTP GET requests are considered, hence HTTP request/sec as a metric for capacity estimation is sufficient. One other important aspect of AutoScaling is that, in order to maintain the QoS in trying conditions, the system must scale up early while the scale down operation can be slower and less critical than the scale up operation [26]. The AutoScaling approach employed in this thesis work is outlined in Algorithm 1. Algorithm 1 Resource Controller [27] 1: start VM in C pr 2: V M cur V M pr 3: while true do 4: monitor W a (n); 5: W ft (n + 1) Forecast(W a (n)); 6: V M λ map(w ft (n + 1)); 7: if V M cur < V M λ then 8: scale up; 9: else if V M cur > V M λ then 10: scale down; 11: end if 12: sleep k; 13: end while Algorithm 2 explains the workload prediction while Table 3.2 lists the parameters and variables used in the algorithms. Non-stationary time series data must be stationarized, in order to work with prediction models such as ARMA. Hence, a stationarize block is included in the algorithm, which comes into play in case the time series data is non-stationary. The resource controller actively monitors the infrastructure for the changes in workload demand as well as the current running virtual machine instances. Given the workload demand prediction, resource controller calculates the appropriate

29 CHAPTER 3. DESIGN OVERVIEW 21 number of virtual machine instances required to handle the demand. If the required number is more than the current number of virtual machine instances, resource controller launches new instances, based on the predicted workload demand. Similarly, if the required number is less than the current number, resource controller scales down the infrastructure. Algorithm 2 Forecast [27] 1: W s (n) stationarize(w a (n)); 2: µ = M (1 α); 3: ɛ(n) = W at (n 1) β ɛ(n 1); 4: W st (n + 1) = µ + α W st (n) + β ɛ(n); 5: W ft (n + 1) destationarize(w st (n + 1));

30 CHAPTER 3. DESIGN OVERVIEW 22 Table 3.2: List of variables used in Algorithms 1 and 2. Variables Description W a (n) Actual workload till n th instant W at (n) Actual workload at n th instant W st (n) Stationarized workload at n th instant W ft (n + 1) Forecasted workload at (n + 1) th instant C pr Private cloud C pu Public cloud map Returns V M λ V M cur Number of running VMs V M pr Fixed number of running VMs in Private cloud V M λ Optimum number of VMs k Monitoring interval W s (n) Stationarized actual workload values α AR coefficient β MA coefficient µ ARMA constant ɛ White Noise

31 Chapter 4 Implementation This chapter deals with the practical setup and implementation details used in the simulations. Specifically this chapter talks about test setup architecture, implementation of dynamic resource provisioning framework, workload modeling and generation tools etc. 4.1 Test Cloud Setup In order to implement and test the hypothesis about efficient AutoScaling using prediction models, a private cloud was setup for conducting simulations. We used the private cluster at Aalto University and installed Open- Nebula to manage and administer it. OpenNebula, a stable open source cloud project was chosen as the private cloud management software because of its clean installation process, ease of usage and simplified cloud API that lets administrators and developers programmatically control the infrastructure. Although the concepts advocated in this thesis largely apply to public IaaS cloud platforms, they are platform agnostic. A public IaaS cloud platform also offers a simple API for programmatic access to control the resources and hence it is relatively simple to apply these concepts in practice on a public cloud platform like Amazon EC2. One major reason for conducting the experiments on a private cloud is that, a private cloud operating in a private network offers an environment to conduct experiments with different configurations. For instance, with a private cloud setup, it is possible to eliminate the side affects that may occur in a multi-tenant environment, while the same cannot be said so for a public cloud platform. One of the key factors for efficient AutoScaling is the API access to the infrastructure that enables automatic control of the resources. Figure 4.1 shows the practical test setup used for simulations. To simulate variable rate 23

32 CHAPTER 4. IMPLEMENTATION 24 workload requests, jmeter, a workload generation tool was used. JMeter 1 in this case represents the clients, sending out multiple concurrent requests. The requests are directed at a common interface, the load balancer that then distributes the requests to the virtual servers running in the cloud. HAProxy 2 is used as the load balancer and reverse proxy, that collects all the requests and distributes them to the private virtual servers in the cloud. The virtual servers running RUBiS, process the requests and send the responses back to the clients through the load balancer. Tcpstat 3 is used for periodic monitoring of the workload request rate at the load balancer. Tcpstat AutoScaler XML-RPC request Frontend JMeter Client JMeter Client... JMeter Client H A P R O X Y VM VM OpenNebula Node Figure 4.1: Test setup 4.2 Workload Generation Testing the performance of a web service distributed over several virtual servers, requires a comprehensive suite of load generation tools. To simu

33 CHAPTER 4. IMPLEMENTATION 25 late several hundreds of concurrent requests, a complex multi-threaded application is required. Thankfully, several such comprehensive applications already exist. Apache JMeter is one of the most prominent load testing applications with provisions to test functional behavior, performance of static and dynamic network based applications. JMeter is a complete java based load generation tool with a multi-threaded framework providing an ability to generate several concurrent requests. JMeter, besides providing a userfriendly GUI to build test plans, also has provisions for scheduling test runs, controlling request rates etc [12]. In this thesis, the goal was to generate random, variable rate workload requests and check the system performance. JMeter, in addition to a large number of attractive features, also allows the test clients to be distributed over multiple machines. This is especially helpful in scenarios where system must be tested for extremely high load rates generating several thousands of requests per second. It is also possible to pre-define a workload trace and let jmeter follow the trace by generating requests according to the rates defined by the trace. One extremely useful open source jmeter plugin to achieve such a task is Throughput Shaping Timer 4. This plugin allows for user defined traces, where a user can define the request rate for any given period of time and construct a workload trace accordingly. JMeter can then be used to generate the requests according to the trace defined by the user. One such workload trace was used for simulations with reactive and predictive resource controllers. Chapter 5 illustrates on the specific details. 4.3 Resource Provisioning Framework Resource controller is the focal point that controls all the components on the cloud side. Resource controller has access to the load balancer, monitoring data, opennebula frontend for requesting additional resources on demand. OpenNebula frontend runs a XML-RPC server, as an interface for querying and requesting infrastructure changes. A simple reactive resource controller analyses the data obtained from tcpstat and decides on the scaling decision. To implement the scaling decision, the resource controller makes a XML-RPC request to the frontend, requesting additional virtual servers. Predictive resource controller, in addition to the basic functionalities, predicts the future workloads based on a recent history of data obtained from the monitoring software. Based on the predicted value and pre-defined threshold limits that dictate the workload handling capacity of virtual servers, the resource con- 4

34 CHAPTER 4. IMPLEMENTATION 26 troller preemptively makes a request to launch more instances, if needed. Resource controller program aggregates the data collected from the load balancer, monitoring data from the OpenNebula frontend and also implements a XML-RPC client that can query and request resources from the frontend XML-RPC server. Besides this, the predictive resource controller also implements a prediction function that predict future workload values based on the history of workload data [21].

35 Chapter 5 Performance Evaluation In order to evaluate and compare the performance of different prediction models described in Chapter 3, we chose to test the models with real world and artificial workload traces. In this thesis, we used two such traces, two real world open source workload traces and one artificial workload trace. We evaluate the performance of each prediction model in terms of the prediction error, taking into account, both under-estimation and over-estimation errors. Under estimation error refers to an error scenario wherein the predicted value is lesser than that of the actual value. Similarly, over estimation error is an error condition where the predicted value is greater than the actual value. We compare the efficiency by considering the mean squared error (MSE) and mean percentage error (MPE). 5.1 Workload Traces In this section, we present the pictorial representation of real world and artificial workload traces and other relevant details. In the scope of this thesis, we have modeled the workload demand considering it in terms of the number of requests per second. However, there are other several other metrics such as the number of active users, processor utilization etc. The following subsections provide more details about individual traces Google Trace Google recently outsourced two real workload traces from a Google production cluster. The first trace was a relatively short one, about 7 hours long. The second trace was much longer, and covered a bigger cluster of roughly machines. The second trace is 29 days long and includes a variety 27

36 CHAPTER 5. PERFORMANCE EVALUATION 28 of information about resource requests, request failures, priorities, request constraints etc [37]. In order to apply this workload trace in the context of this thesis, we have extracted the provided data in to request per second collected over five minute intervals. The priorities, failures and other irrelevant aspects of the trace were ignored. In the gamut of data provided, the most interesting part of the trace is in the first one week, as there is a sudden spike occurring within a few minutes. So we considered the first one week trace data for experiments with the prediction models. Figure 5.1 shows the workload trace used in the thesis work Google Workload Trace Actual Trace 1000 Message request rate(msg/sec) Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 One Week Figure 5.1: Google cluster workload trace [37] Artificial Trace The artificial workload trace was created with the help of JMeter and other associated plugins of JMeter. In this thesis, this particular workload trace was also used for comparing the performance of reactive and predictive elastic resource controllers. As mentioned earlier, the test use case was RUBiS and JMeter was used for workload generation. The Throughput Shaping Timer 1 plugin is an open source JMeter plugin that allows, among other things, to create workloads according to the needs of the user. The user can decide on 1 ThroughputShapingTimer

37 CHAPTER 5. PERFORMANCE EVALUATION 29 the amount of load that JMeter should generate at any given instant of time, thus simplifying the process of variable rate workload generation. Figure 5.2 presents the workload trace. The x-axis represents the time in minutes, while the y-axis denotes the number of HTTP requests generated on a per second basis Artificial Workload Actual Trace 800 Message request rate(msg/sec) Time in minutes Figure 5.2: Artificial workload trace 5.2 Workload Prediction The four prediction models considered in this thesis are tested for prediction accuracy and prediction error. The real world and artificial traces are used for simulating one-step ahead prediction and a comparative analysis of the prediction models is performed. In this section, we present the figures depicting the workload prediction for the two traces and a table comparing the prediction error Mean Mean is an estimation procedure that dampens the crests and elevates the troughs. A window size of 16 was used for the Google workload trace, while on the other hand, window size of 8 was used for the artificial trace. The window

38 CHAPTER 5. PERFORMANCE EVALUATION 30 size usually depends on the length of the trace as well as the fluctuations in the trace. Figures 5.3 and 5.4 represent the prediction curves Mean16 Actual Prediction 1000 Message request rate(msg/sec) Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 One Week Figure 5.3: Google workload trace - Mean 16 prediction 1000 Mean 8 Actual Prediction 800 Message request rate(msg/sec) Time in minutes Figure 5.4: Artificial workload trace - Mean 8 prediction

39 CHAPTER 5. PERFORMANCE EVALUATION Max Max is also a simplistic estimation procedure that prepares for a worst case scenario occurred in the recent past. Given a window size, max always chooses the maximum occurring observation. A window size of 16 was used for the Google workload trace, while on the other hand, window size of 8 was used for the artificial trace. The window size usually depends on the length of the trace as well as the fluctuations in the trace. Figures 5.5 and 5.6 represent the prediction curves Max16 Actual Prediction 1000 Message request rate(msg/sec) Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 One Week Figure 5.5: Google workload trace - Max 16 prediction

40 CHAPTER 5. PERFORMANCE EVALUATION Max 8 Actual Prediction 800 Message request rate(msg/sec) Time in minutes Figure 5.6: Artificial workload trace - Max 8 prediction Exponential Smoothing In this thesis, single exponential smoothing technique was employed. Exponential smoothing assigns weights to the observations from the recent past in an exponential decreasing manner. Hence the most recent observations get more weightage than the less recent ones. The smoothing coefficient used for the Google and the artificial traces are and respectively. Figures 5.7 and 5.8 depict the prediction curves with exponential smoothing as the prediction model.

41 CHAPTER 5. PERFORMANCE EVALUATION Exponential Smoothing Actual Prediction 1000 Message request rate(msg/sec) Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 One Week Figure 5.7: Google workload trace - Exponential smoothing prediction 1000 Exponential Smoothing Actual Prediction 800 Message request rate(msg/sec) Time in minutes Figure 5.8: Artificial workload trace - Exponential smoothing prediction

42 CHAPTER 5. PERFORMANCE EVALUATION ARIMA Google workload trace provides significant data which can be used for model building and parameterization of ARIMA constants. ARMA(1,1,1) model was used as first order differencing was sufficient in this case. The parameter values are and for α and β respectively ARIMA(1,1,1) Actual Prediction 1000 Message request rate(msg/sec) Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 One week Figure 5.9: Google workload trace - ARIMA prediction The artificial workload trace is shorter and hence the first few minutes of the trace are utilized for model building and ARIMA parameterization. In this case ARIMA(1,2,1) model was used where the order of differencing was 2. The parameter values are and for α and β respectively. Figures 5.9 and 5.10 represents the prediction curves with ARIMA.

43 CHAPTER 5. PERFORMANCE EVALUATION Artificial Workload - ARIMA(1,2,1) Actual Prediction 800 Message request rate(msg/sec) Time in minutes Figure 5.10: Artificial workload trace - ARIMA prediction Prediction Accuracy Tables 5.1 and 5.2 list the MSE and MPE errors for the chosen four prediction models tested with real world and artificial workload traces. MPE(OE) and MPE(UE) stand for MPE over estimation and MPE under estimation. Over estimation error occurs when the predicted value is more than the actual or measured value. Similarly, under estimation error is an error condition that occurs when the predicted value is less than the actual or measured value. Table 5.1: Google workload trace Model MSE MPE(OE) in % MPE(UE) in % Mean Max ES ARIMA MSE is usually a better indicator of performance for workloads with a lot of variations. However, MPE does highlight the overall deviation in terms of

44 CHAPTER 5. PERFORMANCE EVALUATION 36 Table 5.2: Artificial workload trace Model MSE MPE(OE) in % MPE(UE) in % Mean Max ES ARIMA relative percentages, but need not necessarily provide an accurate measure of a model s performance. When the observed value is small, even a tolerable deviation is highlighted in an alarming fashion, because the percentage is always calculated relative to the actual observed value. Consider for example, an error of 5 when the observed value is 10, results in a 50% error, while an error of 50 when the observed value is 500, results in a 10% error. Since MPE averages all the error percentages, tolerable errors may elevate the overall error percentage. Hence error in absolute numbers, could complement the MPE and provide a comprehensive picture on the performance of a model. From tables 5.1 and 5.2 it is quite clear that ARIMA and exponential smoothing are by far the most consistent and efficient prediction models. Expectedly, Max-8 and Max-16 produced huge over estimation errors and comparatively lower under estimation errors. This is mainly due to the fact that Max-8 and Max-16 provide worst case estimates for the future workloads by assigning the highest occurring observation from the recent past. Similarly Mean-8 and Mean-16 perform only slightly better than Max, resulting in higher MSE and MPE. ARIMA on the other hand outperforms exponential smoothing in terms of lower MSE values and lower under estimation errors. It is a common notion in workload forecasting that, slightly over estimated forecasts are usually tolerable but more critical are the under estimation errors. ARIMA produces lower under estimation errors while maintaining lower MSE values. 5.3 RUBiS Simulation Workload trace depicted in 5.2 is simulated with JMeter clients, to compare the performance implications of predictive AutoScaling in the cloud. RUBiS is configured as a distributed web service running across multiple virtual servers in the cloud. The configuration is tested with two different

45 CHAPTER 5. PERFORMANCE EVALUATION 37 resource controllers; reactive and predictive. JMeter was configured with the Throughput Shaping Timer to follow the request rate pattern in 5.2 and reports several metrics like the average response time, average throughput etc. One crucial factor for internet based services is the response time during service demand peaks. In order to have better points of comparison, we have defined a threshold for the acceptable response time for a request. For this simulation, any web request with a response time of more than 1000 millisecond is considered an service level violation. Based on the comparative analysis of prediction models, ARIMA(1,2,1) model was chosen as the perfect fit for simulations. Table 5.3 illustrates the performance impact of using prediction models in comparison with a simple reactive approach. Table 5.3: RUBiS Simulation Metrics Reactive Predictive Throughput (msg/s) Avg. response time (msec) Violations (%) SLA Violations Predictive approach performed better than the reactive one in all of the metrics mentioned in the table. The throughput was clearly higher, average response time was lower and most importantly, the number of violations were significantly lesser than in the reactive approach. Considering that there were in total, about 2 million requests during the course of the simulation, 1.45 % improvement in the SLA violations speaks volumes about the efficiency of the predictive approach.

46 CHAPTER 5. PERFORMANCE EVALUATION 38 6 Reactive Predictive 5 Error percentage during scaleup period Time in minutes Figure 5.11: Predictive and Reactive error comparison In order to reiterate the significance of SLA violations, Figure 5.11 depicts the per minute error percentage during the steepest scale up in the simulation. The graph covers the violations for about 20 minutes from 36 minutes to 56 minutes in the simulation, where the curve hits the peak demand. Clearly, the predictive resource controller out performed the reactive controller and brought a significant improvement in the overall system performance.

47 Chapter 6 Discussion In this thesis, we presented a dynamic resource provisioning framework capable of predicting future workloads and optimizing the overall system performance. We chose RUBiS as the use case for simulations, as it represents a web service identical to Amazon, ebay and other SLA critical services. RUBiS can also be configured as a multi-tier web service, separating the web frontend from the database backend. In order to evaluate the relative efficiency of models in comparison, we chose average response time, average throughput and service level violations as critical performance indicators. Service level violation defines the acceptable response time for a message request and in the context of this thesis, it is 1000 msec. Based on the experimental evaluation, the predictive resource controller excelled in all of the critical performance indicators. We observed that the predictive approach produced very few service violations overall and the performance of the system during the scale up period was tolerable and superior to the performance obtained with the reactive resource controller. 6.1 Limitations The research work presented in this thesis has certain limitations. The practical simulations were conducted on the basis of an assumption that the workloads are homogeneous by nature. Ideally, a reasonable metric for measuring the load on a system such as the one used in this work would be the number of active users. Apart from the active users, the heterogeneous user requests could be further segregated to evaluate the true impact on the system under test. Often times, for websites such as ebay and Amazon, CBMG is constructed based on historical data, to effectively estimate the future load on individual tiers of the system [22]. Workload prediction 39

48 CHAPTER 6. DISCUSSION 40 models in the context of multi-tier application architectures, coupled with the estimates provided by CBMG graphs, could be a potent framework in dealing with variable workloads. But for the ease of simulations, we confined to HTTP GET requests and measured the load on the system as a function of request per second. The other major limitation is the capacity estimation process followed in this thesis. Capacity estimation is an extremely important step in designing and implementing a dynamic resource provisioning framework. Under ideal conditions, static capacity estimation procedure which defines certain conditions based on which auto-scaling is performed. For instance, given a VM configuration, we tested the RUBiS VM performance at different request rates and arrived at an optimal threshold value at which the VM performed reasonably well, with fewer service violations. Unfortunately, applying this approach in a realistic scenario, may not be efficient due to the performance variations that can occur in a multi-tenant public IaaS cloud environment. There is a need for dynamic capacity estimation techniques, that take into account the under-performance of a VM. One effective approach in addressing this issue could be to mathematically model the correlation between throughput, service violations, average response time and VM performance metrics such as processor and memory utilization. This way, a badly performing VM could be identified and appropriately reduce the workload handled by that particular VM or provision additional resources in order to maintain the QoS. 6.2 Cloud Reliability Several research results [8, 9, 17, 35, 36] show the effect of multi-tenant hosts running VMs of different cloud subscribers. Static capacity estimation techniques, unfortunately in a realistic scenario, may not be efficient due to the performance variations that can occur in a multi-tenant public IaaS cloud environment. There is a need for dynamic capacity estimation techniques, that take into account the under-performance of a VM [18, 30]. One effective approach in addressing this issue could be to mathematically model the correlation between throughput, service violations, average response time and VM performance metrics such as processor and memory utilization. This way, a badly performing VM could be identified and appropriately reduce the workload handled by that particular VM or provision additional resources in order to maintain the QoS.

49 CHAPTER 6. DISCUSSION Future Work Auto-scaling in a real world scenario would require, taking into account several complex aspects such as the multi-tier application architectures, longer VM startup times etc. Auto-scaling multi-tier applications is the next logical problem to solve, while accommodating longer VM startup time requires a detailed study on multi-step prediction models. We would also like to evaluate the impact of multi-tenancy on the performance of an application and explore ways to mitigate the effects. The possible performance degradation of sensitive applications, due to multi-tenant cloud environments could be addressed in two ways. Firstly, from a cloud provider perspective, there is a need for novel VM scheduling and placement algorithms that consider the issue of multi-tenancy [25]. The second approach would be to find alternative ways to mitigate the effect from a cloud subscriber s point of view. Unfortunately, traditional monitoring methods may not reveal the under-performing VM and hence we need to explore ways to estimate the health of a VM based on several performance metrics such as throughput, response time etc. Another interesting topic of research is to investigate the advantages of using a hybrid approach to research provisioning. The hybrid resource provisioning approach essentially includes both the proactive and reactive components, in order to effectively deal with the possible failure of a prediction model, in predicting the future workloads [1].

50 Chapter 7 Conclusions In this thesis, we propose a dynamic resource provisioning framework for tackling the most important issues of auto-scaling in IaaS cloud environments. The research work begins by highlighting the problems involved in the simplistic reactive approach to auto-scaling. We provide the motivation for intelligent resource provisioning mechanisms and study a few prediction models. The relative performance of the chosen models is evaluated with real world and artificial workload traces based on the prediction accuracy of each of the four models. We then present an ARIMA based resource provisioning model which predicts the future workload demand and allocates resources preemptively. RUBiS was used as a test use case for conducting simulations and comparing the efficiency of ARIMA based resource controller and a simple reactive resource controller. Even though the proactive approach to auto-scaling works better than the reactive one, there are some inherent limitations in the approach followed during the course of the simulations. The most important ones being, ignoring the heterogeneity of the workloads and static capacity estimation of virtual resources. We argue that there is a need for dynamic capacity estimation techniques and seek ways to address the issues that may occur due to the multi-tenant nature of a public IaaS cloud environment. 42

51 Bibliography [1] Ali-Eldin, A., Kihl, M., Tordsson, J., and Elmroth, E. Efficient provisioning of bursty scientific workloads on the cloud using adaptive elasticity control. In Proceedings of the 3rd workshop on Scientific Cloud Computing Date (New York, NY, USA, 2012), ScienceCloud 12, ACM, pp [2] Amza, C., Cecchet, E., Ch, A., Cox, A. L., Elnikety, S., Gil, R., Marguerite, J., Rajamani, K., and Zwaenepoel, W. Bottleneck characterization of dynamic web site benchmarks. Tech. rep., [3] Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., Lee, G., Patterson, D. A., Rabkin, A., Stoica, I., and Zaharia, M. Above the clouds: A berkeley view of cloud computing. Tech. Rep. UCB/EECS , EECS Department, University of California, Berkeley, Feb [4] Barker, S. K., and Shenoy, P. Empirical evaluation of latencysensitive application performance in the cloud. In Proceedings of the first annual ACM SIGMM conference on Multimedia systems (New York, NY, USA, 2010), MMSys 10, ACM, pp [5] Beimborn, D., Miletzki, T., and Wenzel, S. Platform as a service (paas). Business & Information Systems Engineering 3, 6 (2011), [6] Box, G. E. P., and Jenkins, G. Time Series Analysis, Forecasting and Control. Holden-Day, Incorporated, [7] Charlie Oppenheimer, M. P. Which is less expensive: Amazon or self-hosted?, Feb

52 BIBLIOGRAPHY 44 [8] Garraghan, P., Townend, P., and Xu, J. Realtime fault-tolerance in federated cloud environments. In Object/Component/Service-Oriented Real-Time Distributed Computing Workshops (ISORCW), th IEEE International Symposium on (april 2012), pp [9] Ghoshal, D., Canon, R. S., and Ramakrishnan, L. I/o performance of virtualized cloud environments. In Proceedings of the second international workshop on Data intensive computing in the clouds (New York, NY, USA, 2011), DataCloud-SC 11, ACM, pp [10] Greenberg, A., Hamilton, J., Maltz, D. A., and Patel, P. The cost of a cloud: research problems in data center networks. SIGCOMM Comput. Commun. Rev. 39, 1 (Dec. 2008), [11] Hajjat, M., Sun, X., Sung, Y.-W. E., Maltz, D., Rao, S., Sripanidkulchai, K., and Tawarmalani, M. Cloudward bound: planning for beneficial migration of enterprise applications to the cloud. In Proceedings of the ACM SIGCOMM 2010 conference (New York, NY, USA, 2010), SIGCOMM 10, ACM, pp [12] Halili, E. Apache JMeter. Packt Publishing, [13] Hamilton, J. D. Time Series Analysis. Princeton Univerity Press, [14] Harms, R., and Yamartino, M. The economics of the cloud. Microsoft whitepaper, Microsoft Corporation, Redmond, WA, USA, November [15] Hill, Z., and Humphrey, M. A quantitative analysis of high performance computing with Amazon s EC2 infrastructure: The death of the local cluster? In Grid Computing, th IEEE/ACM International Conference on (Oct 2009), pp [16] Holt, C. C. Forecasting seasonals and trends by exponentially weighted moving averages. International Journal of Forecasting 20, 1 (2004), [17] Hovestadt, M., Kao, O., Kliem, A., and Warneke, D. Evaluating adaptive compression to mitigate the effects of shared i/o in clouds. In Proceedings of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PhD Forum (Washington, DC, USA, 2011), IPDPSW 11, IEEE Computer Society, pp

53 BIBLIOGRAPHY 45 [18] Isci, C., Hanson, J., Whalley, I., Steinder, M., and Kephart, J. Runtime demand estimation for effective dynamic resource management. In Network Operations and Management Symposium (NOMS), 2010 IEEE (april 2010), pp [19] Lenk, A., Klems, M., Nimis, J., Tai, S., and Sandholm, T. What s inside the cloud? an architectural map of the cloud landscape. In Proceedings of the 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing (Washington, DC, USA, 2009), CLOUD 09, IEEE Computer Society, pp [20] Li, A., Yang, X., Kandula, S., and Zhang, M. Cloudcmp: comparing public cloud providers. In Proceedings of the 10th annual conference on Internet measurement (New York, NY, USA, 2010), IMC 10, ACM, pp [21] Mao, M., and Humphrey, M. Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In Proc International Conference for High Performance Computing, Networking, Storage and Analysis (2011), SC 11, ACM, pp. 49:1 49:12. [22] Mark, K., and Csaba, L. Analyzing customer behavior model graph (cbmg) using markov chains. In Intelligent Engineering Systems, INES th International Conference on ( july ), pp [23] Mcleod, A. I. Parsimony, model adequacy and periodic correlation in time series forecasting, [24] Mell, P., and Grance, T. The nist definition of cloud computing. National Institute of Standards and Technology 53, 6 (2009), 50. [25] Mishra, M., and Sahoo, A. On theory of vm placement: Anomalies in existing methodologies and their mitigation using a novel vector based approach. In Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing (Washington, DC, USA, 2011), CLOUD 11, IEEE Computer Society, pp [26] Orzell, G., and Becker, J. Auto scaling in the amazon cloud, January

54 BIBLIOGRAPHY 46 [27] Raivio, Y., Mazhelis, O., Annapureddy, K., Mallavarapu, R., and Tyrväinen, P. Hybrid cloud architecture for short message services. In CLOSER (2012), pp [28] Reiss, C., Wilkes, J., and Hellerstein, J. L. Google clusterusage traces: format + schema. Technical report, Google Inc., Mountain View, CA, USA, Nov Revised Posted at URL http: //code.google.com/p/googleclusterdata/wiki/traceversion2. [29] Roy, N., Dubey, A., and Gokhale, A. Efficient autoscaling in the cloud using predictive models for workload forecasting. In Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing (Washington, DC, USA, 2011), CLOUD 11, IEEE Computer Society, pp [30] Roy, N., Dubey, A., Gokhale, A., and Dowdy, L. A capacity planning process for performance assurance of component-based distributed systems. SIGSOFT Softw. Eng. Notes 36, 5 (Sept. 2011), [31] Sempolinski, P., and Thain, D. A comparison and critique of eucalyptus, opennebula and nimbus. In Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on ( dec ), pp [32] Vaquero, L. M., Rodero-Merino, L., and Buyya, R. Dynamically scaling applications in the cloud. SIGCOMM Comput. Commun. Rev. 41, 1, [33] Vaquero, L. M., Rodero-Merino, L., Caceres, J., and Lindner, M. A break in the clouds: towards a cloud definition. SIGCOMM Comput. Commun. Rev. 39, 1 (Dec. 2008), [34] von Eicken, T. Animoto s facebook scale-up, April [35] Wang, G., and Ng, T. S. E. The impact of virtualization on network performance of amazon ec2 data center. In Proceedings of the 29th conference on Information communications (Piscataway, NJ, USA, 2010), INFOCOM 10, IEEE Press, pp [36] Wang, J., Varman, P. J., and Xie, C. Avoiding performance fluctuation in cloud storage. In HiPC (2010), pp. 1 9.

55 BIBLIOGRAPHY 47 [37] Wilkes, J. More google cluster data. Available at http: //googleresearch.blogspot.com/2011/11/more-google-cluster-data. html, 2011.