A Flexible Elastic Control Plane for Private Clouds

Transcription

1 A Flexible Elastic otrol Plae for Private louds Upedra Sharma IBM Watso Prashat Sheoy Dept. of omputer Sciece Amherst MA Sambit Sahu IBM Watso ABSTRAT While public cloud computig platforms have become popular i recet years, private clouds operated by eterprises for their iteral use have also begu gaiig tractio. The cofiguratio ad cotiuous tuig of a private cloud to meet user demads is a complex task. While private cloud maagemet frameworks provide a umber of flexible cofiguratio optios for this purpose, they leave it to the admiistrator to determie how to best cofigure ad tue the cloud platform for local eeds. I this paper, we argue for a adaptive cotrol plae for private clouds that simplifies the tasks of cofigurig ad operatig a private cloud such that each cotrol plae service is adaptive to the workload see due to ed-user requests. We preset a logistic regressio model to automate the provisioig ad dyamic recofiguratio of cotrol plae services i a private cloud. We implemet our approach for two cotrol plae services moitorig ad messagig for OpeStack-based private clouds. Our experimetal results o a laboratory private cloud testbed ad usig public cloud workloads demostrates the ability of our approach to provisio ad adapt such services from very small to very large private cloud cofiguratios. ategories ad Subject Descriptors D.4.8 [Operatig Systems]: Performace Keywords loud computig, dyamic provisioig, logistic regressio 1. INTRODUTION loud computig has become popular i recet years for ruig Iteret ad eterprise applicatios due to its pay-as-you-go pricig model ad ability to elastically allocate resources. While public cloud platforms have attracted much attetio, the desig of private clouds cloud platforms that are operated by eterprises for their ow iteral use have begi gaiig tractio. Today a umber of private cloud maagemet frameworks are available, ragig from commercial offerigs from IBM ad VMWare to ope-source Permissio to make digital or hard copies of all or part of this work for persoal or classroom use is grated without fee provided that copies are ot made or distributed for profit or commercial advatage ad that copies bear this otice ad the full citatio o the first page. To copy otherwise, to republish, to post o servers or to redistribute to lists, requires prior specific permissio ad/or a fee. A 13, August 5 9, 2013, Miami, FL, USA. opyright 2013 AM /13/08 $ frameworks such as OpeStack, loudstack ad OpeNebula. Despite the availability of these platforms, the task of cofigurig, maagig ad operatig a private cloud remais challegig. Most private cloud maagemet frameworks expose a rage of flexible cofiguratio optios ad settigs to support various deploymet architectures. However, they leave it to the system admiistrator to determie a deploymet architecture ad cofiguratio settigs that are best suited for local eeds. I particular, most private cloud maagemet frameworks implemet a cotrol plae for maagig various cloud services such as moitorig, messagig, allocatio of compute ad storage resources ad VM image maagemet (see Fig. 1). The task of cofigurig each service, allocatig sufficiet resources to service ed-user requests, ad cotiuously tuig the service to adjust to chagig eeds is left to the admiistrator. Messagig Server Server Server Server Server Hypervisor Network Moitorig Mgmt Mgmt otrol Plae Services Storage Storage Storage Storage IT Ifrastructure Storage Mgmt Network IP Addresses Figure 1: Architecture of private cloud ad its cotrol plae. I this paper, we argue for a adaptive cotrol plae for private clouds that simplifies the tasks of cofigurig ad operatig a private cloud. Such a adaptive cotrol plae must simplify the iitial setup ad cofiguratio of each cotrol plae service ad esure that each service is resposive to the workload see due to eduser requests. Further, as the demads imposed by the private cloud vary over time, the cotrol plae must adapt the service to chagig workloads. loud platforms have log supported the otio of elasticity for ed-user applicatios. Elasticity implies that the resources (such as the umber of VMs or servers) allocated to the applicatio is automatically adjusted to match the variatios i the icomig workload. I this work, we propose that the cotrol plae of the cloud must itself be elastic ad adjust the resources allocated to various cotrol plae services automatically to chagig eeds just as it does for ed-user applicatios. Thus our paper focuses o the desig of a flexible, adaptive cotrol plae that automates the iitial cofiguratio of each cotrol plae service to match the eeds of a private cloud of a desired size ad elastically provisios resources for these services as their

2 workload demads chage over time. I desigig our elastic cotrol plae, we make the followig cotributios. First, we model the iteractios of each cotrol plae service with ed-user VMs ad betwee themselves ad develop a logistic regressio based model to estimate capacity eeded to sustai a certai workload with a certai SLO. A key beefit of usig logistic regressio over other techiques is that it does ot require a large traiig set to model the behavior of the service. Our adaptive cotrol plae the uses this model to determie how may odes (or VMs) are eeded to service the expected workload. I the evet the cotrol plae service eeds to be replicated, it also determies whether these replicas should be federated or clustered to meet the desired SLO i the most efficiet maer. Such approach greatly simplifies the iitial setup of each cotrol plae service by the admiistrator. Secod, sice the workload see by a cotrol plae service may vary or grow over time, our cotrol plae implemets elasticity of each service. We preset reactive ad proactive elasticity mechaisms that ca dyamically provisio additioal capacity for each service o-the-fly. Our proactive approach combies our logistic regressio model with workload forecastig techiques to proactively allocate resources to each elastic cotrol plae service. Prototype implemetatio ad experimetal validatio: Third, we implemet a prototype of our flexible elastic cotrol plae for a OpeStack-based private cloud ad demostrate its efficacy for two essetial cotrol plae services: moitorig ad messagig. Our experimetal results o a laboratory private cloud testbed ad usig public cloud workloads demostrates the ability of our approach to provisio ad adapt these services for private clouds ragig from very small to very large cofiguratios. We also demostrate the ability of our dyamic recofiguratio approach to elastically provisio capacity to these services o-the-fly. 2. BAKGROUND Private louds: A private cloud cosists of ifrastructure resources like compute, storage ad etwork ad allows its users to create virtual resources o-demad. Private clouds implemet similar fuctioality as public clouds like Amazo E2, except that use they ifrastructure owed by a eterprise to implemet cloud fuctioality for iteral use. A umber of ope source cloud maagemet platforms are available to establish ad operate a private cloud, amely OpeStack [11], loudstack, Eucalyptus, OpeNebula etc. These assume a cluster of liux machies ad provide a cotrol plae to maage the cloud ifrastructure ad perform maagemet tasks, like hypervisor maagemet, user maagemet, messagig, moitorig, image maagemet, etc. as depicted i Figure 1. Each such maagemet task is performed by a cotrol plae service that rus i oe or more virtual machies. I this work we have chose OpeStack as our cloud maagemet system of study; this is primarily because it offers a rich set of cotrol plae services ad has become a popular choice amogst the ope source commuity [8]. Problem Formulatio: osider a orgaizatio that wishes to deploy a private cloud o a cluster of size N. Most private cloud maagemet frameworks are desiged to work with as few as tes of hosts/machies to very large clusters cosistig of thousad machies, but for successful ad efficiet operatio, the cloud maagemet system has to be cofigured accordig to the size of the cluster. To do so, the admiistrator must cofigure each cotrol plae service ad provisio sufficiet capacity so that it ca service the cotrol plae workload geerated by the maagemet tasks i a cluster of that size. I the simplest case, each cotrol plae service will ru o a sigle virtualized ode. A sigle ode per service setup is adequate for a small to medium size clusters. However, as the cluster size grows, a sigle ode setup will become a bottleeck. For istace, cosider a moitorig service, which performs two major tasks, recordig the moitored metrics for all resource as well servig queries regardig the same. A sigle ode deploymet of the moitorig service may easily hadle the moitorig data from a cluster cotaiig a few tes to a few hudred odes. However, if the cluster grows to a thousad machies or more, the amout of moitorig data that is geerated by the cliets will overwhelm the sigle ode moitorig service. To scale the cotrol plae i such scearios, the service will eed to be replicated o multiple odes ad the icomig workload to the service will eed to be distributed across the replicas of the service. Typically replicatio ca be doe i oe of two ways: (i) by employig clusterig, where a group of replicas of the cotrol plae service collectively serve the requests made to it, or (ii) by employig federatio, which partitios the workload across multiple istaces of the cotrol plae service. I the clusterig approach, all replicas collectively serve all the requests as a sigle logical etity as show i Figure 2a. I federatio, each service istace services a subset of cliets ad forwards oly the ecessary requests to the other as show i Figure 2b. Both clusterig ad federatio approaches partitio the workload but clusterig based approach also allows high availability while federatio does ot. cliets otrol plae service odes (a) lusterig cliets otrol plae service odes (b) Federatio Figure 2: lusterig ad Federated approaches Give such a private cloud maagemet system ad the cotrol plae services, a IT admiistrator is faced with a two fold task of appropriately cofigurig each cotrol plae service ad esurig that there is sufficiet capacity to service the requests. Maual cofiguratio ad capacity allocatio is a challegig task as a large umber of iterdepedet services are ivolved. We, thus, have the problem of cofigurig ad provisioig each service so that the task of deployig the cotrol plae service ca be automated. While there are rules of thumb o how to cofigure these cotrol plae services, it is ot apparet which approach to use to scale up ad i what situatios. I additio, it is challegig to determie how may istaces to provisio for a private cloud of a certai size. Our approach is to desig a flexible cotrol plae service that automates this task by solvig two sub problems: (i) Give the size of cluster, say N, choose which approach is suitable, i.e. sigle ode, clusterig, or federatio for each service. (ii) Determie ad provisio sufficiet umber of odes if the service is replicated. Dyamic Provisioig: The iitial setup of the cotrol plae ad its various services is based o a estimate of the workload likely to be see by each cotrol plae service. However, the workload observed by cotrol plae services may chage over time either due to imperfect iitial estimates of cliet workloads or due to icremetal growth of the maaged ifrastructure or eve a sudde chage i maaged workload. For istace the admiistrator may icrease the moitorig resolutio from 15 mi to 1 mi, causig

3 SLO-Violatios a order of magitude icrease i the moitorig data. I such situatios, some services required to be recofigured by dyamically icreasig (or decreasig) the capacity of the cotrol plae service. Thus the cotrol plae must itself be adaptive ad elastic it eeds the ability to dyamically recofigure a cotrol plae service by provisioig ew capacity for the service whe the specified SLOs ca o loger be met. While the problem of dyamic provisioig of applicatio VMs has bee well studied [15, 13, 19, 21, 14, 20, 17], elasticity ad provisioig of cotrol plae services of a private cloud has ot received much attetio. As we argue i this work, prior methods such as queuig models for provisioig of applicatio VMs are ot suitable i this cotext, primarily because models ofte ca ot accout for software artifacts that limit the applicatio capacity from scalig. Secodly, models are ofte specific to a software with a specific topology type ad are very expesive to develop. Istead we exploit the particular ature of cotrol plae service iteractios to model a cotrol service ad desig elasticity mechaisms that are tailored for such scearios. System Model: Each cotrol plae service is assumed to be composed of multiple software compoets; these compoets ca be deployed i dedicated virtual machies we refer to them as compoet odes of a cotrol plae service. I this work we assume that all the compoet odes of a cotrol plae service are idetical (thus we also address a compoet ode as a replica). This is ot a limitatio of our approach but a simplificatio, which we have adopted for ease of expositio of our approach. A fully fuctioal cotrol plae service is assumed to be created by arragig these compoet odes i a sigle ode, clustered or federated cofiguratio as show i Figure 2. We assume that each compoet ode has a associated SLO it ad the admiistrator must pick a cofiguratio ad umber of odes such that there is eough capacity to serve the request see by the service. Further, it is assumed that the SLO violatios of each service ca be moitored by loggig the performace see by cotrol plae requests. 3. MODELING AND ONFIGURING ON- TROL PLANE SERVIES Sice each cotrol plae service ca be clustered, federated or ru o a sigle ode, we model service as a set of oe or more idetical compoets (referred as compoet odes). A compoet ode is assumed to service two types of requests, amely exteral requests from ifrastructure odes or other services ad iteral requests from the other compoet odes of the same service. Let λ c ad λ deote the average workload due to exteral cliet requests ad iteral odes requests, respectively. We also assume that each cotrol plae service eeds to meet a performace threshold to meet a Service Level Objective (SLO). SLO of a cotrol plae ca be specified usig a threshold o applicatio performace metric (e.g. latecy) or o a resource utilizatio metric, for istace 80% of PU utilizatio. Admiistrators must estimate ad provisio sufficiet resource capacity to a cotrol plae service to avoid violatig the SLO. We automate this task of cofigurig ad provisioig the cotrol plae service by determiig whether a sigle ode or clustered or federated cofiguratio is best suited for the cotrol plae service ad how may odes are ecessary to provide the desired capacity. Our approach comprises of derivig a aalytical model to determie the capacity eeded ad a algorithm to dyamically reprovisio whe the workload icreases beyod the capacity. We gather empirical data of system performace by offlie empirical profilig; it aids i accoutig for i) software artifacts which limit the applicatios capacity ad ii) performace variatio due to various hidde factors, like shared resource allocatio, hardware etc. 3.1 Aalytical model otrol plae service uses differet resources, amely memory, PU, etwork etc. The performace of a cotrol plae service ca be affected by may factors, icludig its cofiguratio, workload variatios, resource utilizatio, ad also artifacts of the ivolved software compoets as well as those of the system. We preset a probabilistic model, based o logistic regressio, to estimate the capacity eeded by a cotrol plae service to service a particular workload. SLO Threshold apacity (a) Workload (x) (b) (workload) Figure 3: Ituitio of SLO violatio curve Let λ T be the be the total estimated workload ad let k be the umber of replicas (k 1) eeded to service this workload. Thus, we must estimate the umber of replicas k required by a cotrol plae to service a workload of requests arrivig at rate λ T for a give SLO. Our approach cosists of gatherig empirical data of SLO violatios of each ode/replica of the cotrol plae service ad use these observatios to build a probabilistic model/fuctio of SLO violatios give the observed workload at the ode. We the use this model/fuctio to determie the max load λ c that ca be serviced by a sigle ode; give this capacity of a sigle ode, we ca estimate the umber of odes, i.e. k, for a specific cofiguratio (i.e. clustered, federated). We ow determie a fuctio that relates λ to the SLO. More formally, let Y be a biary radom variable, which represets presece/absece of a SLO violatio ad λ be the total workload observed by a ode (i.e. λ = λ c +λ ). We, the, wish to estimate the coditioal expectatio of SLO violatio, i.e. E(Y λ). There are a umber of sophisticated o parametric techiques which ca estimate coditioal probabilities but these techiques ofte require a large amout of traiig data to create reliable models. Logistic regressio [7] is a alterative that does ot require a large umber of traiig samples to determie the coditioal expectatio. Let π(λ) deote the coditioal expectatio E(Y λ), whe assumig a logistic distributio. The specific form of logistic distributio we use is: π(λ) = e(β 0+β 1 λ) 1 + e (β 0+β 1 λ), (1) where, β 0 is the itercept parameter ad β 1s is the slope parameter. We re-write (1) to obtai a liear equatio i λ: ( ) π(λ) g(λ) = l = β 0 + β 1λ. (2) 1 π(λ) The parameters β 0 ad β 1 ca be estimated usig logistic regressio; they are maximum likelihood estimates of π(λ) expectatio of SLO violatio for a give λ. Usig (2), we ca compute the value of λ for a give probability of SLO violatio, say =λ. For istace, let us suppose we wat to

4 compute the capacity λ for a coservative threshold o probability of SLO violatio, say 0.5; this essetially meas that wheever λ λ there is more tha 50% chace of SLO violatio (as show i Figure 3b). Thus equatig π(λ) = 0.5 i equatio 2 yields β 0 + β 1λ = 0 (3) Estimatig β 0 ad β 1 requires some real observatios of workloads ad SLO of a cotrol plae ode i a real settig. For that we perform offlie empirical profilig of cotrol plae services i differet topologies as outlied i the ext sectio. 3.2 Workload Estimatio The workload λ see by a ode of each cotrol plae service has two compoets, amely requests from the cliets (λ c) ad requests from the other replicas/odes of the same service (λ ), i.e. λ = λ c + λ. Itra service workload (λ ) is a fuctio of cliet workload (i.e. λ = f(λ c)) ad the exact form of the fuctio depeds o the cofiguratio. We ca use kowledge of the cotrol plae to provide the fuctio f. For istace, i a federated setup the cliets are partitioed ito smaller groups ad each partitio is serviced by oe ode/replica. Thus λ is a fractio of λ c, i.e. λ = δλ c. Similarly, i the case of clusterig, the itra service workload depeds o the size of the cluster ad also o the way it has bee implemeted. This meas that if a clustered cofiguratio implemets iformatio exchage via broadcast the the messages received by each ode will equal the size of the cluster; ow, if the service uses multicast trasmissio to implemet the same the oly oe message eed to be set, however,if the implemetatio adopts uicast trasmissio the the umber of outgoig messages will be equal to the size of the cluster. Thus for a cluster of size we will have λ = 2( 1)λ c if the uicast is adopted, while i the case of multicast based implemetatio it will be λ = λ c. O the other had if othig is kow about the cotrol plae service the we ca treat the cotrol plae service as a black box ad estimate λ as fuctio of λ c by regressig over the empirical profilig data, i.e. λ = α 0 + α 1λ c. (4) For the purpose of computig iitial estimate of cotrol plae s capacity, ad also for performig empirical profilig, we require a estimate the cliet workload, i.e. λ c, ad the workload geerated by a sigle cliet, say λ c. We make use of rules of thumb or prior experiece for the same; for istace, if it is kow that for each moitored machie a moitorig service records a average of 25 metrics at a graularity of 1-sec, the λ c = 25. Now, if the moitorig ode services cliets the the average cliet workload observed by a sigle moitorig ode will be λ c = 25 ad the total cliet workload observed by the whole moitorig service for a cluster of size N will be λ T = N λ c. 3.3 Provisioig Algorithm Havig modeled the cotrol plae service ad estimated the workload parameters, we compute the umber of odes required for service as follows: Step 1: First we use the traiig data to compute the βs i (2) usig logistic regressio. Usig the values of βs we compute a coservative capacity of a cotrol plae ode i terms of workload which it ca hadle, i.e. λ, usig (3). Step 2: Next we estimate the maximum cliet workload a cotrol plae ode ca service, say λ c. Sice observed iteral workload (λ ) is a kow fuctio, f(), of cliet workload (λ c) we estimate the capacity of the service i terms of umber of cliets that ca be serviced, say λ c, by solvig the followig equatio for λ c: λ c + f(λ c) = λ Step 3: We estimate the capacity of a cotrol plae service, i.e. total umber of cotrol plae odes (say k), required to service a cluster of size N, i.e. k = λ T /λ c. Step 4: If the above steps idicate that a sigle ode is ot sufficiet to hadle the workload, i.e., the k is foud to be greater tha 1, the we must determie whether to employ clusterig or federatio for the replicated service. To judiciously choose betwee them, the above steps are repeated for clusterig ad federatio by usig the appropriate fuctio f() for each cofiguratio as derived i Sectio 3.2. We the choose the cofiguratio that is more efficiet, i.e., yields a smaller k. The fial step the provisios the estimated capacity k for that cofiguratio, i.e. clustered or federated. 4. ELASTI REONFIGURATION Our provisioig algorithm provides a techique to determie the appropriate cofiguratio (e.g., sigle ode, clustered or federated) ad the capacity k eeded to service the estimated workload. Sice the iitial provisioig is based o a estimate of the workload likely to be see, the actual workload may be differet or may grow over time. Hece our cotrol plae implemets elasticity for each service by eablig them to be re-provisioed as ad whe eeded. For example, if the admiistrator chages the frequecy of moitorig each ode from 5 miutes to 1 miute, there will be a five-fold icrease i moitorig data, which may require the moitorig service to be reprovisioed if ay ode gets saturated due to this chage. Such elastic reprovisioig ad recofiguratio ivolves two steps: i) Whe to trigger dyamic reprovisioig? ii) How to migrate from curret cofiguratio to ew oe? Whe to trigger? Elastic reprovisioig ca be triggered reactively or proactively. Reactive reprovisioig is triggered whe the cotrol plae detects SLO violatios for a particular service, while proactive reprovisioig is triggered whe future workload forecasts idicate SLO violatios are likely i the ear future. Reactive: The cotrol plae moitors each service ad reacts to observed SLO violatios by ivokig re-provisioig. I this simplest case, the cotrol plae ca gradually icrease the umber of replicas allocated to a service i steps util the SLO violatios stop (e.g., icrease the umber of replicas by oe ode at a time step util the violatios stop). A better approach is to use the recet history of the workload see by the service re-ru the provisioig algorithm from the previous sectio. Doig so will yield a ew k for the umber of replicas eeded by the service ad the cotrol plae ca simply start k k ew replicas, where k is the curret umber replicas for the service. Proactive: Proactive provisioig ivolves combiig workload forecastig with the provisioig algorithm from the previous sectio to aticipate SLO violatios before they occur ad take corrective actio. To do so, we ca employ a workload forecastig techique to predict the expected workload t time uits ito the future. If the predicted workload is higher tha the peak estimate used for the curretly provisioed capacity, the SLO violatios are likely ad the cotrol plae will ivoke the provisioig algorithm from the previous sectio with the ew workload forecast. While ay workload forecastig method ca be used by the cotrol plae, we curretly employ time-series forecastig. Similar to the approach used i [15], we obtai a time series of workload observatios, model the workload as a ARIMA time series [4], ad use stadard ARIMA-based forecastig to predict the workload for a fixed time iterval t ito the future. This predictio is used by

5 the provisioig algorithm to compute a ew capacity k ad additioal replicas are spawed by the cotrol plae for the service. How to migrate to ew cofiguratio? There are two mai steps i migratig the cotrol plae service from old cofiguratio to ew cofiguratio, amely i) redeploymet ad ii) redistributio of the cliets across the ew cofiguratio. Redeploymet ivolves deployig the ecessary additioal VM replicas for the service. Applicatio topologies ca be ecoded i a Ope Virtualizatio Format (OVF) [6], which ca be used by exteral deploymet scripts to provisio the replicas. Most commo cloud maagemet platforms support OVF makig it a good implemetatio choice. I this work our re-deploymet task provisios ewly computed capacity ad ad iter-coects the deployed compoets formig the same cofiguratio patter; however, selectig ad switchig to a differet cofiguratio patter is a relatively easy extesio of this work. Redistributio: Oce ew replicas have bee provisioed, the workload has to be redistributed across ew ad old replicas equally. This ivolves idetifyig the cliets of the service ad chagig their cofiguratios to apped ew replicas to the list of available replicas for the service, ad perhaps specifyig the preferred replica to use for the service. 5. PROTOTYPE IMPLEMENTATION This sectio describes the prototype of our elastic cotrol plae. Empirical Profilig Model Geerator Data Data P Service Metadata Maager Provisioig Algorithm Actuator P Service loud VM Provisioig Dyamic Recofigurator P Service loud Maagemet Layer Provisioig Egie Adaptatio otroller Workload Moitorig Forecastig Figure 4: Architecture of our elastic cotrol plae 5.1 System Architecture Our prototype depeds o moitorig of the system ad performace metric of each of the cotrol plae service odes. Moitorig of systems ad resources is a stadard practice followed i all large system deploymets ad besides this our approach does ot put ay additioal load o the cotrol plae service. Other compoets of our prototype, amely adaptatio cotroller ad moitorig ad foreastig compoets, are hosted o a dedicated VM ad implemeted i pytho; details of each these compoets are (see Figure 4): Model Geerator takes the empirical profilig data for each cotrol plae service ad geerates a model (set of βs) ad stores it i the Metadata maager. We have used the STATA 10 s implemetatio of logistic regressio [2] to obtai our models. Metadata maager stores models for each cotrol plae service, their curret cofiguratio ad capacities. We have implemeted it as a pytho class, which stores all the iformatio i i-memory data structures with a optio to persist the models o disk. Workload Moitorig ad Forecastig Egie collects time-series moitorig data of all the virtual machies as well as of those of the cotrol plae services. It stores all the results i a database, which ca be queried. We have implemeted this as a part of moitorig service of OpeStack usig Gaglia. We have used STATA 10 for implemetig the ARIMA forecaster [18]. ofiguratio ad Provisioig Egie implemets the provisioig algorithm. It takes the geerated model from Metadata maager ad computes the umber of replicas eeded for a cofiguratio. I case of chage i cofiguratio, it provisios ew replicas usig the Actuator module ad updates the details of ew cofiguratio to Metadata maager. It also performs dyamic recofiguratio by costatly evaluatig the SLO metric ad by computig the chage i average cliet workload. As a solutio to the less frequet situatio where the model requires re-learig, the provisioig egie queries ad collects the cases of SLO violatios ad updates the learig data. It the re-estimates the model parameters ad updates the records i metadata maager. Dyamic recofigurator: This compoet exposes two iterfaces, redeploy ad redistribute. We provide a implemetatio for each cotrol plae service. urretly we have implemeted a plugis for moitorig ad messagig services. Actuator: This is module is a part of cofiguratio ad provisioig egie. It performs the task of deployig ew virtual machies of each cotrol plae service. After deploymet it executes the ecessary scripts i each replica of the cotrol plae service for creatig the correct cofiguratio. The actuator also looks up the depedet cliets ad alter s their cofiguratio so that the cliet workload is evely distributed across all the replicas. It essetially keeps a fixed umber of cliets for each replica. 5.2 Private cloud maagemet system We use OpeStack as our private cloud maagemet system. OpeStack comprises four mai compoets: compute (Nova), image repository (Glace), autheticatio (Keystoe), ad storage (Swift). Nova, Glace, ad Keystoe provide hypervisor maagemet, image maagemet ad autheticatio, respectively, while Swift provides a object-store service [10]. We use Nova as a example to expose some of the desig details of scalable cloud service compoets. Nova has multiple cotrol plae services which together provide the the fuctioality of compute ad storage maagemet. Nova s various services commuicate with each other via message queuig [16]. Here we use a ope-source message queuig system, amely RabbitMQ [12]. Although moitorig is a key compoet of a cloud platform, OpeStack curretly lack a full fledged moitorig service. We implemet our ow prototype moitorig service for OpeStack to mimic Amazo s loudwatch moitorig service i the E2 public cloud. We build moitorig for OpeStack by itegratig two ope-source moitorig systems ito OpeStack: Gaglia ad Nagios [9]. Gaglia is used to moitor odes ad VMs ad archive moitored data i its database while Nagios is used to set simple triggers o moitored data; for istace, we ca use Nagios to report if the average resource utilizatio of a group of odes exceeds a threshold. 5.3 Empirical profilig As the first step towards learig the parameters we perform empirical profilig of each cotrol plae service i both clustered ad federated cofiguratios. I order to empirically profile a compoet ode of a service i a particular cofiguratio, we start with

6 a sigle ode cofiguratio ad systematically icrease the cliet workload o the service (i.e. λ c) util we observe SLO violatios. We, the, repeat the same procedure with the ext bigger graph of the same cofiguratio ad so o; at each step we record the average itra service workload as well (i.e. λ ) liets (a) Star/Sigle Node liets liets (b) luster (c) Federated -(d) Federated - Rig Heirarchical liets (root) (itermediate) (leaf) Figure 5: Profilig cofiguratios; grey odes represet cotrol plae service odes ad the white odes are its cliets Sigle ode: this is the smallest cofiguratio Figure 5a. We assume the average workload geerated by a sigle cliet, i.e. λ c ad for the purpose of icreasig λ c we simply icrease the umber of cliets util we observe SLO violatios. luster cofiguratio: We start with a cluster of size 2 ad distribute the cliets betwee the two odes. We, the gradually icrease the workload o both the odes, distributed evely, util we observe SLO violatios o ay of the odes. We measure both the average umber of cliet requests per sec, λ c, ad the average umber of itra service requests per sec, λ. We repeat the same experimet for larger cluster sizes. This helps us gather data ecessary for capturig the impact of cluster size as well. Federated cofiguratio: A federated cofiguratio partitios its cliets betwee compoet odes ad ca be hierarchical or o (e.g., a rig). For ohierarchical kid of federated cofiguratio, the method of profilig is like that of clustered cofiguratio, i.e. we start with smallest possible cofiguratio ad profile till it is saturatio ad the icrease the umber of compoet odes by oe ad repeat the same procedure. I the case of hierarchical cofiguratio, we have to differetiate betwee odes, i.e. leaf odes, itermediate odes ad root ode. This makes the profilig a little more ivolved. We start with a tree of depth=1, i.e. with a root ode ad sigle leaf ode; The, similar to cluster topology we icrease the cliet workload o the leaf ode while keepig the cliet workload of root ode to zero util leaf ode s saturatio. We keep the root ode s cliet workload to zero ad gradually icrease the λ by addig more leaf odes. We repeat the same experimet with a tree of depth two, where there is a leaf ode ad itermediate ode ad the a leaf ode. We empirically profile the itermediate ode. Determiig SLO metric: SLO metric of each cotrol plae service should selected such that a faithful operatio of the maagemet service ca be esured by moitorig a easily observable value threshold of the metric. I situatios where the SLO metric is ot directly observable, admiistrators ca empirical profile the service i a closely cotrolled eviromet ad choose that metric (or a set of metrics) which are strogly correlated with SLO. Empirical profilig also assists admiistrators to heuristically determie a coservative threshold value for the moitored metrics, which ca be used to trigger dyamic provisioig much before the actual SLO violatio happes. We maually perform empirical profilig by gradually icreasig the workload ad record SLO violatios. This data is used to obtai the model of the maagemet service. Offlie empirical profilig is ot a limitatio of the approach as admiistrators ofte perform empirical profilig before deployig a large-scale system; i additio to this, the empirical profilig data ca be used as a startig poit of the dyamic provisioig algorithm. 6. EXPERIMENTAL EVALUATION This sectio describes our experimetal setup ad our experimets to test the efficacy of our elastic cotrol plae. 6.1 Experimetal Setup We have used OpeStack as our private cloud maagemet system ad have experimeted with two of its cotrol plae services, amely moitorig ad messagig. Moitorig: As explaied earlier, we have implemeted the moitorig cotrol plae service of OpeStack usig Gaglia. Gaglia cosists a moitorig aget, gmod, which gathers ad broadcasts moitored data usig UDP multicast/uicast. The moitored data is pushed to a metadata server, gmetad, for archival; gmetad stores data i a Roud Robi Database (RRD) ad leverages rrdtool to extract ad graph the moitored data usig Apache web-server ad php techology. For our experimets, we defied SLO of our moitorig subsystem as a threshold o the percetage of data loss due to ureliable message delivery or system saturatio. We have used Gaglia o Ubutu to create each ode of our moitorig cotrol plae service by deployig both a gmetad ad a gmod daemo o it. This server is resposible for gatherig all data from the moitored odes. liet workload geerated by a sigle cliet (i.e. λ c) is depedet o umber of metrics beig moitored ad the frequecy at which they are moitored. We have cosidered three types of moitorig workloads for our experimets. Each of these workloads ivolve moitorig 25 metrics but at differet moitorig frequecies, i.e. 1-sec, 5-sec ad, 15-sec. To simulate large clusters tha available i our testbed, we created cliet workload geerators that emulate the data set by gaglia o real odes. Our cliets geerate ad sed sythetic gmod 2.x data packets to gmod. For each moitored ode we set 25 separate metrics at the pre-cofigured moitorig frequecy. O the moitorig ode the metrics are saved i separate files ad folders, where files are amed usig the metric s ame ad the folder is amed usig the host ame. Messagig: OpeStack s message queuig subsystem is the backboe of this scalable private cloud maagemet system. All cotrol plae services of the compute cloud of OpeStack (i.e. Nova) commuicate with each other via blockig ad o-blockig RP calls usig the message queuig system [16]. We use RabbitMQ 2.8 as OpeStack s message queueig system for our experimets. We experimeted o a private cloud created over 12 Itel Xeo (X3430) machies each with 8 GB RAM ad 500 GB SATA Disk. The machies were istalled with Ubutu ad we created the private cloud usig OpeStack Essex release. Each RabbitMQ ode possesses both queue-maagemet capabilities as well as router capabilities. We istalled each such ode o sigle core VMs with 6GB of RAM. The cliets were deployed o the hosts described above. O each VM as well o each host we set the limit to um-

7 ber of ope files to (80 K). I order to sychroize the clock we used NTP Sice RabbitMQ is memory boud ad stops receivig messages whe memory gets saturated, for our experimets, we defie a SLO as a high threshold of the memory utilizatio of the RabbitMQ ode (E.g., 75% utilizatio threshold). To geerate the workload, we assume that each cliet, i.e. Nova-compute ad Nova-volume geerate oe VM ad volume creatio ad deletio request every hour. We assume that each cliet ode is of 64 cores ad thus each ode create oe VM as well as volume creatio request every secod. Sice a VM creatio requires 5 messages ad VM-deletio requires 6 messages ad equal for volume creatio ad deletio, thus λ c = Empirical Profilig ad apacity Estimatio I this sectio we empirically profile two cotrol plae services, amely moitorig ad messagig, i differet cofiguratios, amely sigle ode, cluster ad federated. We the use the profilig data ad the results developed i Sectio 3 to compute the maximum capacity of a cofiguratio. Fially, we test our dyamic recofiguratio approach o moitorig subsystem deployed i a federated topology Sigle Node ofiguratio Moitorig: We created a sigle ode cofiguratio of moitorig cotrol plae service by usig the m2.xlarge istace of E2 as our moitorig ode. We coducted profilig i three differet moitorig graularities, i.e. 1-sec, 5-sec ad 15-sec. We have simulated c cliet odes by sedig the moitored data of c 25 metrics 1 to the moitorig ode from 5 cliet machies ruig the cliet workload simulator. The λ c observed by the moitorig ode i the three respective moitorig graularities is c 25, 5 c, ad 5 c/3. The observed data-loss at the moitorig ode for each of the three differet moitorig graularities is recorded for traiig the model. The SLO plots for each of the experimet are show i Figure 6, where each poit o the graph is a average of more tha 50 samples. (a) Estimated ad observed capacities as a fuctio of λ c 1-sec 5-sec 15-sec (λ c = 25) (λ c = 5) (λ c = 5/3) apacity (b) Estimated apacities Figure 7: Empirical ad estimated capacities of sigle ode moitorig cofiguratio with moitorig ode o a m2.xlarge istace type. Messagig: We coducted the experimet with a sigle ode cofiguratio by usig a sigle core VM but with varyig amout RAM to RabbitMQ, i.e. startig from from 400MB to 2.4GB. For each RAM cofiguratio, we gradually icrease the umber of compute ad volume odes, which icreases the message traffic via the message queue. We, the, measure memory utilizatio of the RabbitMQ ode. The results are show i Figure 8. ompute ompute ompute ompute RabbitMQ Volume Volume Volume Volume Volume Volume Scheduler (a) Sigle ode setup Network (b) Memory utilizatio (c) apacity Figure 6: Data loss i a sigle ode cofiguratio We used the complete profilig data to compute the capacity model of a sigle ode cofiguratio i three differet settigs. We estimate the parameters of the model usig logistic regressio ad compute capacity usig (3). Figure 7b summarizes the results of profilig of a sigle ode cofiguratio with the three differet moitorig workloads as a table. It ca be see from Figure 7a that capacity does ot vary liearly i λ c ad that the model provides a coservative estimate of capacity with the data geerated by empirical profilig. 1 Amazo cloudwatch moitors 25 metrics for a istace ad its volume Figure 8: Memory utilizatio ad average message latecy observed i a sigle ode cofiguratio of RabbitMQ I each experimet we gradually icreased the umber of compute odes, keepig a sigle scheduler ad a sigle etwork ode. Icreasig scheduler ad etwork is ot a recommeded cofiguratio i opestack. I each experimet we scaled up the umber of cliets i batches of 250 cliets. We stop addig cliets whe the memory utilizatio reaches 75% of the total allowed memory to RabbitMQ server. We foud that the memory utilizatio liearly icreases with umber of cliets, as show i figure 8b. To geerate a model of sigle ode cofiguratio, we coducted experimets

8 where we varied the RAM from 400MB to 2400MB ad the results are show i figure 8c. It ca be observed that the capacity scales liearly with RAM for workload geerated by OpeStack cliets ad the model captures a coservative estimate of the same. oclusio: Empirical profilig effectively captures the software artifacts. I additio the model allows us to capture that kowledge ad geerate coservative estimates Federated cofiguratio For a federated cofiguratio we coducted a experimet with a tree of depth two, as show i figure 9a (which meas a tree of depth oe for cotrol plae odes). I this cofiguratio there are multiple moitorig odes each of which gathers the data from their idividual group of moitored odes, called clusters. It is ofte useful for admiistrators to have a summary of moitored metrics at cluster level. I our case the root ode pulls the summary statistics data every t r-secods. This places additioal load o the leaf moitorig odes ad thus impacts data loss. I a hierarchical cofiguratio there are three types of odes, amely a leaf metadata ode (subjected to both cliet ad itraservice workload) ad a root ode (with oly itra service workload); we profiled each of these odes. For leaf metadata odes, we geerated the cliet workload i the same maer as for sigleode cofiguratio profilig. However, for geeratig itra service workload (λ ), we setup the root ode to pull data from the leaf metadata odes at three differet graularities, i.e. 15-sec, 30- sec ad, 1-mi. This is because the higher level odes i the tree collect oly summary statistics of the lower level odes ad thus the resolutio is ofte quite low. For each resolutio, we measure SLO while gradually icreasig λ. The variatio i the SLO metric with icrease i workload, for both leaf as well as root metadata ode, is show i Figure 9b ad 9c respectively. We have used a average workload of 25 metrics per moitored cliet, thus λ c = 25/t l, where t l is the moitorig graularity of the leaf metadata ode. Similarly average workload geerated by a leaf metadata ode for its paret ode is λ = 150/t r, where t r is the moitorig graularity of the root metadata ode. We coducted ie profilig experimets ad developed capacity models for each of them. As expected, we fid that the data loss characteristics of the leaf moitorig odes are very similar to those of a sigle ode cofiguratio except oly slightly less (show i Table 1a). However, as the root metadata ode s moitorig graularity icreases to 30-sec ad 60-sec the impact becomes early egligible. Table 1c summarizes the maximum capacities of a tree topology of depth oe with ie differet settigs of moitorig graularities. The total capacity of each of the ie cofiguratios is computed by multiplyig capacities of leaf ad root odes. This is because of the fact that we assume λ = 150/t r (a costat). Note that a approximate fuctioal form of λ = f(λ c) is estimated usig the kowledge of the moitorig service. Sice the root ode collects oly the averaged values from each child metadata odes, it computes to λ = λ c 6/t r. oclusio: Empirical profilig assists i capturig the applicatio artifacts. This coupled with our modelig approach helps i estimatig maximum capacity of ay cofiguratio luster ofiguratio RabbitMQ supports a clusterig cofiguratio, where each broker ode i the cluster has all a replica of all the data ecessary for operatio. This meas that ay queue ca be accessed from ay broker ode, however, the queues ad its messages are ot replicated ad thus it saves uecessary excess commuicatio. We Moitorig Node Moitored Nodes Root Node Moitorig Node (a) A federated cofiguratio Moitorig Node (b) Data loss o leaf moitorig ode; root ode moitorig at 15-sec graularity (c) Data loss o root moitorig ode Figure 9: Data loss i a federated cofiguratio experimeted with three types of cluster ode cofiguratios ad the results are show i Figure 10. We coducted two set of experimets: First with by hostig RabbitMQ o a sigle core VM but with 2.4GB RAM, secod with a RabbitMQ server with sigle core VM ad with 0.4GB RAM. For both the experimets we varied the size of cluster, say k, ad for each such cluster we gradually icreased the umber of cliets till SLO violatios were observed. The secod experimet was coducted with a limited RAM to study the asymptotic behavior of the cofiguratio ad also to test if the model ca capture this kowledge faithfully. I the case of clusterig cofiguratio, the itra service workload (i.e. λ ) scales liearly with λ c. The liear fuctio is such that it also depeds o the size of cluster, say k. So we estimated the followig fuctio from our empirical profilig data λ = α 0 + kα 1λ c. We estimate the capacity of the clustered setup usig our logistic regressio. Results of empirical observatios for the cluster with 2.4GB RAM are show i figure 10d. We observed that this data creates a model which depicts a liear growth. To study the impact of icreasig cluster size o the capacity we coducted the same experimet but with much less amout of RAM to the RabbitMQ odes, i.e. 0.4GB. Sice λ is ot liearly depedet o λ c (because cluster size k is also a variable), we ra a multiple logistic regressio with λ c ad λ as our idepedet variables ad SLO as the depedet variable. apacity i terms of umber

9 RootNode/LeafNode 15-sec 30-sec 1-mi λ = 10 λ = 5 λ = sec (λ c = 25) sec (λ c = 5) sec (λ c = 1.67) (a) Leaf metadata ode capacity 15 -sec 30-sec 1-mi apacity (b) Root metadata ode capacity Uit Uit (a) Two Node Uit Uit Uit (b) Three Node Uit Uit Uit Uit liets (c) Four Node RootNode/LeafNode 15-sec 30-sec 1-mi 1-sec sec sec (c) apacity of a hierarchical cofiguratio patter Table 1: Empirical capacity of federated moitorig cofiguratio deployed as a tree of depth of two; moitorig ode o a m2.xlarge istace type. of cliets which a ode ca hadle reduces to the followig form: λ c = 22.25/( k 0.003). We plot the capacity for each k usig this result ad the capacity curve is show i figure 10e. The figure depicts that the capacity of a clustered cofiguratio starts to saturate as cluster size icreases. Thus after some poit i time it will ot be useful to scale usig clustered cofiguratio. oclusio: apacity of a clustered cofiguratio starts to saturate as the size of the cluster icreases. Also the model provides better estimates of SLO violatio with more umber of idepedet parameters, amely λ c ad λ. 6.3 Dyamic Provisioig To showcase the efficacy of our dyamic provisioig approach we coducted two experimets: First where we perform reactive provisioig, i.e. we trigger provisioig at SLO violatios. Secod, where we use a model ad a forecaster that causes proactive provisioig based o forecasted workload. We have experimeted with moitorig service deployed i a federated topology, more precisely, we started with a tree with oe root metadata ode ad oe leaf metadata ode. We setup the leaf ode with moitorig graularity of 1-sec, while the root ode at 15-sec graularity. The capacity model of this cofiguratio is already evaluated i Table 1a ad 1b. We used this model, which reports a capacity of 56 cliets for leaf ode ad 28 cliets for root ode. I both the cases, amely reactive ad proactive, we coducted the experimet i followig maer: i) We started with the two ode tree with 10 cliets attached to the leaf metadata ode. The leaf metadata ode was cofigured to moitor at 1-sec moitorig graularity, while the root metadata ode at a 15-sec graularity. ii) We icreased the workload i uits of 10 cliets (i.e. c+ = 10) after every 5-miutes. I the case of reactive-provisioig experimet, first recofiguratio process triggered whe the -c coectios reached 50 odes because the SLO got violated (show i Figure 11). The ew cofiguratio at this poit cotais a secod leaf ode ( 2). We distribute the cliet workload equally betwee 1 ad 2. As we gradually icrease the workload each of the recofiguratios are triggered by SLO violatio ad leads to icrease i capacity by oe additioal ode. We used this SLO violatio data from the reactive experimet as additioal traiig data to re-lear the capacity model of leaf metadata odes. Our ew model reported a much coservative estimate of capacity, i.e. 33 cliet odes for each leaf (d) ofiguratio apacity (RAM=2.4GB) (e) ofiguratio apacity (RAM=0.4GB) Figure 10: Various cluster cofiguratios ad their empirically estimated capacity. metadata ode. We used this capacity model ad a sigle step perfect forecaster (i.e. provides perfect forecast of workload at the ext time istat) to evaluate the proactive provisioig approach. As metioed, this approach triggers provisioig whe the forecasted workload is more tha the capacity of curret cofiguratio; thus i our experimet the provisioig happeed at time istats whe the umber of cliets reach 30, 60, 90 ad 130; this is because whe the workload reached 30-cliet ode, the forecaster predicted a workload of 40-cliets but the moitorig system s capacity was oly 33-cliets thus it provisioed a extra ode i advace. We observe that proactive provisioig approach observes a slightly higher average data loss (i.e. 5.4%) at 90 cliets tha that observed by the reactive approach at 100 cliet workload. We believe that such radomess is because Gaglia uses UDP ad the routers i E2 would be observig higher workload at that istat, which could cotribute to icrease i data loss. oclusio: i) Our approach of dyamic adaptatio is effective i both the situatios, amely reactive ad model-drive. ii) Models for specific workloads ca be simple but effective. 7. RELATED WORK Dyamic provisioig: There is a large body of related work i the area of dyamic capacity provisioig i data ceters or compute clusters [15, 13, 19, 21, 14]. Much of this work is dyamic provisioig of the deployed web applicatios usig aalytic models, while our work cosiders dyamic provisioig of cotrol plae services i multiple cofiguratios, amely clustered ad federated.

10 Figure 11: Dyamic provisioig of moitorig service Geeric system performace model based o esemble of tree augmeted bayesia etworks has bee developed by Zhag et al i [20] to capture the performace behavior of a system applicatio uder chagig workload coditios. Watso et al. i [17] develop a probabilistic performace model for virtual machies with the objective of capturig the effect of statistical multiplexig i clouds ad impact of other measurable factors to provide performace guaratees expressed i percetiles. I our work, we have used a logistic regressio based approach to model a cotrol plae service for performig dyamic provisioig. loud Bechmarkig May researchers have coducted empirical evaluatio of cloud platforms; Researchers i [5] bechmark Amazo E2 to quatify PU, disk ad etwork performace of the provisioed virtual machies. Sharada et al. i [1] evaluate differet virtualizatio techologies by ruig database workloads i a virtualized eviromet. ooper et al. i [3] propose bechmark for the data storage subsystems popular i clouds, amely Hadoop, assadra, HBase ad compare their bech markig results with shared MySQL implemetatio. Ulike much of the prior work, i this work we bechmark idividual cotrol plae services with differet cofiguratios. 8. ONLUSION AND FUTURE WORK I this paper we cosidered the problem of cofigurig ad maagig a private cloud ad argued that the cotrol plae of such cloud platforms must themselves be elastic to support dyamic cotrol workloads. We preseted a logistic regressio model to automate the provisioig ad dyamic recofiguratio of cotrol plae services i a private cloud. We preseted reactive ad proactive methods for implemetig the provisioig of elastic cotrol plae services. We implemeted our approach for two cotrol plae services moitorig ad messagig for OpeStack-based private clouds. Our experimetal results o a laboratory private cloud testbed ad usig public cloud workloads demostrated the ability of our approach to provisio ad adapt such services from very small to very large private cloud cofiguratios. Ackowledgemets: We ackowledge the aoymous reviewers for their valuable suggestios. Upedra Sharma was supported i part by a IBM Graduate fellowship. This research was supported i part by a IBM OR award ad NSF grats NS , OI , NS , NS REFERENES [1] S. Bose, P. Mishra, P. Sethurama, ad R. Taheri. I Performace Evaluatio ad Bechmarkig, pages Spriger-Verlag, [2] M. L. Buis. predict ad adjust with logistic regressio. Stata Joural, 7(2): (6), [3] B. F. ooper, A. Silberstei, E. Tam, R. Ramakrisha, ad R. Sears. Bechmarkig cloud servig systems with ycsb. So 10, pages , New York, NY, USA, [4] M. David, M. Richard, E. M. Errol, ad J. Richard A. Hay. Iterrupted Time Series Aalysis. SAGE Publicatios, Ic., 0 editio, [5] J. Deju, G. Pierre, ad.-h. hi. Ec2 performace aalysis for resource provisioig of service-orieted applicatios. ISO/ServiceWave 09, pages , Berli, Heidelberg, Spriger-Verlag. [6] Dmtf - ope virtualizatio format specificatio. stadards/documets/dsp0243_1.0.0.pdf, [7] D. W. Hosmer ad S. Lemeshow. Applied logistic regressio (Wiley Series i probability ad statistics). Wiley-Itersciece Publicatio, 2 editio, [8] Q. Jiag. Ope Source IaaS ommuity Aalysis. [9] D. Josephse. Buildig a Moitorig Ifrastructure with Nagios. Pretice Hall PTR, Upper Saddle River, NJ, USA, [10] Amazo Simple Storage Service. [11] OpeStack loud Software. [12] J. Russell ad R. oh. Rabbitmq. Book o Demad, [13]. Stewart ad K. She. Performace modelig ad system maagemet for multi-compoet olie services. NSDI 05, pages 71 84, Berkeley, A, USA, USENIX Associatio. [14] B. Urgaokar, G. Pacifici, P. Sheoy, M. Spreitzer, ad A. Tatawi. A Aalytical Model for Multi-tier Iteret Services ad Its Applicatios. I Proc. of the AM SIGMETRIS of., Baff, aada, Jue [15] B. Urgaokar, P. Sheoy, A. hadra, P. Goyal, ad T. Wood. Agile dyamic provisioig of multi-tier iteret applicatios. AM TAAS., 3:1:1 1:39, March [16] S. Vioski. Advaced message queuig protocol. IEEE Iteret omputig, 10(6):87 89, Nov [17] B. J. Watso, M. Marwah, D. Gmach, Y. he, M. Arlitt, ad Z. Wag. Probabilistic performace modelig of virtualized resource allocatio. IA 10, pages , New York, NY, USA, AM. [18] R. A. Yaffee. Forecast evaluatio with stata. Uited kigdom stata users group meetigs 2010, Stata Users Group, [19] Q. Zhag, L. herkasova, ad E. Smiri. A regressio-based aalytic model for dyamic resource provisioig of multi-tier applicatios. IA 07, Washigto, D, USA, [20] S. Zhag, I. ohe, J. Symos, ad A. Fox. Esembles of models for automated diagosis of system performace problems. DSN 05, pages , Washigto, D, USA, IEEE omputer Society. [21] X. Zhu, D. Youg, B. J. Watso, Z. Wag, J. Rolia, S. Sighal, B. McKee,. Hyser, D. Gmach, R. Garder, T. hristia, ad L. herkasova islads: Itegrated capacity ad workload maagemet for the ext geeratio data ceter. IA 08, pages , 2008.