Flexible Distributed Capacity Allocation and Load Redirect Algorithms for Cloud Systems

Transcription

1 2011 IEEE 4th Internatonal Conference on Cloud Computng Flexble Dstrbuted Capacty Allocaton and Load Redrect Algorthms for Cloud Systems Danlo Ardagna, Sara Casolar, Barbara Pancucc Poltecnco d Mlano, Dpartmento d Elettronca Informazone Unverstà d Modena e Reggo Emla, Dpartmento d Ingegnera dell Informazone Emal: {ardagna,pancucc}@elet.polm.t, {sara.casolar}@unmore.t Abstract In Cloud computng systems, resource management s one of the man ssues. Indeed, n any tme nstant resources have to be allocated to handle effectvely worload fluctuatons, whle provdng Qualty of Servce QoS) guarantees to the end users. In such systems, worload predcton-based autonomc computng technques have been developed. In ths paper we propose capacty allocaton technques able to coordnate multple dstrbuted resource controllers worng n geographcally dstrbuted cloud stes. Furthermore, capacty allocaton solutons are ntegrated wth a load redrecton mechansm whch forwards ncomng requests between dfferent domans. The overall goal s to mnmze the costs of the allocated vrtual machne nstances, whle guaranteeng QoS constrants expressed as a threshold on the average response tme. We compare multple heurstcs whch ntegrate worload predcton and dstrbuted non-lnear optmzaton technques. Expermental results show how our solutons sgnfcantly mprove other heurstcs proposed n the lterature 5-35% on average), wthout ntroducng sgnfcant QoS volatons. Keywords: Infrastructure-as-a-Servce Clouds, Performance Modelng and Management, Capacty Allocaton, Load Balancng, QoS. I. INTRODUCTION Cloud computng s an emergng paradgm that ams at streamlnng the on-demand provsonng of software, hardware, and data as servces, provdng end-users wth flexble and scalable servces accessble through the Internet [15]. Modern cloud nfrastructures lve n an open world, characterzed by contnuous changes n the envronment and n the requrements they have to meet. Contnuous changes occur autonomously and unpredctably, and they are out of control of the cloud provder. Therefore, n order to provde nfrastructure or software as a servce, advanced solutons have to be developed able to dynamcally adapt the cloud nfrastructure, whle provdng contnuous servce and performance guarantees. In ths paper we propose worload predcton-based capacty allocaton technques able to coordnate multple dstrbuted resource controllers worng n geographcally dstrbuted cloud stes. We propose also a dynamc load redrecton mechansm whch allows to mae near-nstantaneous and ntellgent decsons on the requests that have to be redrected durng pea loads from heavly loaded stes to other stes. Requests dstrbuton s optmzed accordng to the average response tme of ncomng requests and the QoS requrements of end users. In cloud systems centralzed approaches to capacty allocaton and load balancng have several crtcal desgn lmtatons ncludng lac scalablty and hgh networ communcaton cost such as networ bottlenec congeston) [5], [25]. Centralzed solutons are not sutable for geographcally dstrbuted systems, such as the cloud or more n general massvely dstrbuted systems [3], [19], [16], snce no entty has global nformaton about all of the system resources. Therefore, effcent decentralzed solutons are mandatory. Dstrbuted resource management polces have been proposed to govern effcently geographcally dstrbuted systems that cannot mplement centralzed decsons and support strong nteractons among the remote nodes [3]. Sometmes, local decsons could lead the system even to unstable oscllatons [18]. It s, thus, dffcult to determne the best control mechansm at each node n solaton, so that the overall system performance s optmzed. Dynamcally choosng when, where and how allocate resources and coordnatng the resource allocaton accordngly s an open problem and s becomng more and more relevant wth the advances of clouds [16]. One of the frst contrbutons for resource management n geographcally dstrbuted systems has been proposed n [3], where novel autonomc dstrbuted load balancng algorthms have been proposed. In dstrbuted streamng networs, authors n [19] have proposed a ont admsson control and resource allocaton scheme.in our wor, the capacty allocaton and load redrect of multple class of requests are modeled as non-lnear programmng problems and solved wth decomposton technques explotng predctve models of the ncomng worload at each physcal ste. We compare our approach wth other heurstcs proposed n the lterature [13], [27], [26] obtanng 5-35% cost savngs, wthout ncurrng n sgnfcant Servce Level Agreement SLA) volatons. To the best of our nowledge ths paper s the very frst contrbuton that proposes an analytcal soluton to the capacty allocaton and load redrecton for cloud systems. The remander of the paper s organzed as follows. The next Secton ntroduces the problem under study, whle Secton III descrbes our man desgn assumptons. The predcton technques used n our wor are ntroduced n Secton IV. The optmzaton problem formulaton s presented n Secton V. The expermental results demonstratng the qualty and effcency of our solutons are reported n Secton VI. Conclusons are fnally drawn n Secton VII. II. PROBLEM STATEMENT In ths paper we tae the perspectve of a Web servce provder whch offers multple transactonal Web Servces /11 $ IEEE DOI /CLOUD

2 IaaS Provder =1 =2 Local worload manager =4! "# $%! &# $%'$%! # %% Local worload manager!!" #$! "" #$%#$! &" $$ =3! "# $%! &# $%'$%! # %% Fg. 1. Vrtualzed Servers! "# $% "# & & & !)!)!)!"#$%&')&*+",-).,"$.# /&#01&#- 23 6!!" #$! %" #$&#$! '" $$ Local WS arrval rates! "# $%! &# $%'$%! # %% Cloud System Reference Framewor.!) Executon rate of local arrvals! "# $%! &# $%'$%! # %% Redrect rate of local arrvals Local CA and LR manager Vrtualzed Servers WSs) hosted at multple stes of an Infrastructure as a Servce IaaS) provder. The hosted WSs represents dfferent applcatons whch can be heterogeneous wth respect to resource demands, worload ntenstes, and QoS requrements. Servces wth dfferent QoS and worload profles are categorzed nto ndependent WS classes. An SLA contract, assocated wth each WS class s establshed between the WS provder and ts end users. It specfes the QoS levels, expressed n terms of average response tme R, the WS provder must meet whle respondng to end users requests for a gven servce class. Overall, the system serves a set K of WS classes and average response tme thresholds are denoted wth R. Applcatons are hosted n vrtual machnes VMs) whch are provded on demand by the IaaS provder. For the sae of smplcty, we assume that each VM hosts a sngle Web servce applcaton. Multple VMs mplementng the same WS class can run n parallel at each physcal locaton. In that case, we assume that the runnng VMs are homogeneous n terms of RAM and CPU capacty and evenly share the ncomng worload ths corresponds to the soluton currently mplemented by IaaS provders [2]). Furthermore, servces can be located on multple stes see Fgure 1). For example, f we consder Amazon Inc. wth ts Elastc Compute Cloud EC2) [2] as IaaS provder, EC2 allows software provders to dynamcally deploy VMs on fve regons located around the world whch are further spread on multple avalablty zones. IaaS provders usually charge software provders on a hourly bass [2]. Hence, the WS provder has to face the Capacty Allocaton CA) problem whch conssts on determnng every hour the optmal number of VMs for each WS class n each IaaS ste accordng to the average load predcted on a hourly bass, whle guaranteeng SLA constrants. In the followng we wll denote by T 1 the md-long tme scale adopted for VMs provsonng. On the other hand, f a ste resources are nsuffcent e.g., because of an unpredctable worload fluctuaton) and the computng condtons become crtcal, ncomng requests can be redrected to other stes. As n other approaches, dynamc Load Redrecton LR) [5], [27] s performed perodcally every T 2 << T 1 tme nstants see Fgure 2) at a more fne-graned tme scale e.g., 5 to 10 mnutes) on the bass of a short-term predcton of future WS worloads [1], [8] or can be trggered by a montorng system n order to react to unexpected events e.g., system falures). By consderng two dfferent tme scales, we are able to capture two types of nformaton related to the IaaS stes [27]. In partcular, the fne-graned worload traces exhbt a hgh varablty due to the short-term varatons of the typcal Webbased worload and for ths reason, the fne-graned tme scale provdes useful nformaton for the dynamc load redrectons. Instead, ncreasng the tme scale, the worload traces are more smoothed and are not characterzed by the nstantaneous peas of the fne-graned tme scale. Ths aspect allows us to use the md-long tme scale to predct the worload trend that represents useful nformaton for the capacty allocaton algorthm. In the followng, we wll denote by I the set of IaaS stes. Each ste s assocated wth ts resource attrbute avalable VMs capacty). For smplcty we assume that VMs are homogeneous n terms of ther computng capacty. In fact, even n case of heterogeneous VMs, cloud provders have a lmted set of avalable confguratons, say S, and therefore a ste wth heterogeneous resources can be modelled as S stes wth homogeneous resources. The capacty of VMs at ste s denoted wth C, whle for each ste and WS class we denote wth Λ the arrval rate predcted at the tme scale T 1 comng from the tme zone where the ste s located, whle we denote wth Λ the correspondng predcton at tme scale T 2 see Fgure 2). The obectve of the CA problem s to determne the number of VMs able to serve Λ requests/s, whle mnmzng VMs costs and guaranteeng that R R. We assume that a WS provder can establsh two dfferent contracts wth the IaaS provder. Namely, t may be possble to access VMs on a pure on-demand bass and the WS provder wll be charged on a hourly bass see e.g., Amazon EC2 on-demand prcng scheme, [2]). Otherwse, t may be possble to pay a fxed annual flat rate for each VM and then access the VMs on a pay-per-use bass wth a fee lower than the pure on-demand case see e.g., Amazon EC2 reserved nstances prcng scheme, [2]). The tme unt cost e.g., $ per hour of VM usage) for the use of flat VMs at ste s denoted by c, whle the cost for VMs on demand wll be denoted by c, wth c < c. The CA problem soluton determnes every T 1 tme unt the number of flat VMs to be allocated to WS class at ste, N, and the number of on demand VMs to be allocated to class at ste, M. We wll denote wth N, the number of flat VMs avalable at ste obtaned wth the annual flat contract). On the other hand, the LR problem ams at determnng every T 2 tme nstants) the executon rate of local arrvals for class at ste, x, and the redrect rate of class at ste toward the other stes, z, n order to satsfy the predcton Λ 164

3 ! $#!" ##! "#! "# Fg. 2. $#! $#!" ## Predcton model tme scales. for the local arrvals, whle guaranteeng that R R. For the sae of clarty, the notaton adopted n ths paper s summarzed n Table I. III. DESIGN ASSUMPTIONS Our dynamc CA and LR technques combne a worload predctor and an optmzaton model. In the followng we wll model each WS class hosted n a VM as an M/G/1 queue [10] as authors n [8] and we assume that requests are served accordng to the processor sharng schedulng dscplne whch s common among Web servce contaners. Future performance for each WS class are obtaned on the bass of the predcton of future worloads. The optmzaton model uses these estmates to determne the number of VM ntances N and M, the executon rate of local arrvals for each class x, and, possbly, the worload redrect to other stes z. For the sae of smplcty, f the worload s redrected to other stes, the fracton of worload to ndvdual stes s nversely proportonal to the networ delay or equvalently s drectly proportonal to the conductance g, of the networ ln between ste and defned as g, = 1/d,. In other words, f we defne the equvalent conductance G at ste as G = g,, the overall load at ste due to the I, redrect of other stes s gven by: g, z G. I, At the tme scale T 2, the total rate of class requests executed at ste s the sum of the requests executed from local arrval,.e. x Λ, and the requests executed from the redrect whch, accordng to the prevous equaton s gven by: x + g, z G. 1) In other words, n our LR scheme requests can be redrected only once. Otherwse multple hops could penalze too much some ndvdual requests, ncreasng the overall response tme varance of requests wthn the same WS class. IV. WORKLOAD PREDICTION MODELS For each ste and WS class, our CA and LR algorthms use two dfferent predctons of the real) ncomng worload Λ comng from the tme zone where the ste s located. I K C c c N T 1 T 2 Λ Λ System Parameters Set of stes Set of WS classes VM nstances capacty at ste Tme unt cost for flat VMs at ste Tme unt cost for on demand VMs at ste Number of flat VMs avalable at ste Long term CA tme horzon Short term LR tme horzon Real local arrval rate for WS class at ste Local arrval rate predcton for WS class at ste at tme scale T 1 Λ Local arrval rate predcton for WS class at ste at tme scale T 2 µ Maxmum servce rate of a capacty 1 VM for executng WS class requests d,, Delay s) for requests redrectng from ste to ste g, = 1 d,, Conductance of the communcaton ln,) G = g,, Equvalent conductance seen from ste to the other stes R Response tme for executng WS class request at ste R WS class request threshold Decson Varables N Number of flat VMs allocated for class request at ste M Number of on demand VMs allocated for class request at ste x Executon rate of local arrvals for WS class request at ste z Redrect of WS class request at ste toward other stes TABLE I CA AND LR PROBLEMS PARAMETERS AND DECISION VARIABLES. In order to predct the local arrval rate Λ, we tae nto account a smple well nown model, the Exponental Smoothng ES). Our choce for a smple model s motvated by the applcaton context characterzed by short-tme predctons sutable for autonomc decsons subect to real-tme constrants n cloud systems. ES s an ntutve forecastng method that unequally weghts the samples of the nput tme seres Λ [17]. Non-unform weghtng s acheved through smoothng parameters whch determne how much mportance s assgned to each sample. ES models have been adopted n many felds for decades [12] and are sutable to runtme and non-statonary applcatons. In our wor, we consder a versons of ES, where parameters are dynamcally chosen n order to adapt the predcton model to the worload fluctuatons that characterze the modern cloud systems. In the followng descrpton of the ES-based predcton model, we consder the tme scale T 1. At sample t, the ES model predcts the local arrval rate at T 1 steps ahead, Λ t + T 1), as a weghted average of the last sample Λ t) and of correspondng predcted sample Λ t), that s equal to: Λ T 1 ) = 1 T 1 Λ T t) 1 t=1 Λ t + T 1 ) = γ t) Λ t) + 1 γ t))λ t), t > T 1 165

4 where Λ T 1) s the ntal predcted value and 0 < γ t) < 1 s the smoothng factor at current sample t related to the ste and the class that determnes how much weght s gven to each sample. We obtan a dynamc ES model by re-evaluatng the smoothng factor γ t) at each predcton sample t. There are dfferent proposals for the dynamc estmaton of γ t) e.g., [24], [17]). Although there s no consensus, a wdely used procedure s proposed by Trgg and Leach [24]. They defne the smoothng parameter as the absolute value of the rato of the smoothed error, A t), to the absolute error, E t), γ t) = A t). The smoothed and absolute errors are equal to: E t) A t) = φɛ t) + 1 φ)a t T 1 ) E t) = φ ɛ t) + 1 φ)e t T 1 ) where ɛ t) s the forecast error at sample t, ɛ t) = Λ t) Λ t), and φ s set arbtrary, wth 0.2 beng a common choce [24]. Ths dynamcal choce of γ t) should mprove the predcton qualty and should lmt the delay problem related to the tradtonal ES model based on a statc choce of the γ parameter. The consdered model s expected to be useful n contexts characterzed by tme seres wth non statonary behavour and a varable nose component. We use an analogous mplementaton of the ES predcton model to predct the local arrval rate at tme granularty T 2, Λ. For the sae of smplcty, n the remander of the paper the t sample ndex wll be omtted. V. OPTIMIZATION PROBLEM FORMULATION As dscussed n Secton II, Capacty Allocaton and Load Redrect are performed wth dfferent tme scales. The Capacty Allocaton problem s formulated n the next Secton, whle our Load Redrect mechansm s presented n Secton V-B. A. Capacty Allocaton problem The CA problem s solved wth T 1 tme perod and ams at mnmzng the overall costs for flat and on demand VM nstances of multple dstrbuted IaaS stes, whle guaranteeng that the average response tme of each class s lower than the SLA threshold. The CA determnes the number of VMs N and M requred to serve the arrval rate Λ. In ths phase the LR mechansm s neglected. Prelmnary results, ndeed, have shown that the LR mechansm, even f sgnfcant at the lower tme scale T 2, ntroduces a lmted ncrement to each class local ncomng worload Λ whch s comparable wth the worload predcton accuracy obtaned n practce. If we denote wth µ the maxmum servce rate of a capacty 1 VM for executng WS class requests, the response tme for executng locally WS class at ste s gven by R = 1. In partcular t must be M/G/1 equlbrum C µ Λ N +M condton) Λ tme for class request over all stes s: < C µ N + M ), and the total response R = Λ R Λ. 2) Hence, after some basc algebra, the CA problem can be formulated as: CA) mn c N N,M + c M subect to Λ < C µ N + M) K, I Λ N + M ) C µ N + M ) Λ R Λ K N N, I, K where the last constrants famly guarantees that the number of VMs allocated to the whole set of classes at ste s at most equal to the number of flat VMs avalable at each ste. Note that, n the problem formulaton we have not mposed varables N and M to be nteger, as n realty they are. In fact, requrng varables to be nteger maes the soluton much more dffcult snce non lnear constrants are ntroduced. We therefore decde to deal wth contnuous varables, actually consderng a relaxaton of the real problem. However, prelmnary expermental results have shown that f the optmal values of the varables are fractonal and they are rounded to the closest nteger soluton, the gap between the soluton of the real nteger problem and the relaxed one s very small, ustfyng the use of a relaxed model. Furthermore, we can always choose a roundng to an nteger soluton whch preserves the feasblty and the correspondng gap n the obectve functon s a few percentage ponts. The CA problem has a lnear obectve functon over a convex set. Hence, the global optmum soluton can be obtaned solvng CA n parallel at each ste by adoptng standard non lnear solvers. Ths requres that each ste broadcasts ts Λ predctons whch can be obtaned however consderng only local nformaton. Snce ths broadcast s performed every T 1 tme nstants, the networ overhead for the CA soluton s very lmted. B. Load Redrect problem Once the number of on demand nstances has been determned, local requests can be dynamcally redrected to other stes wth tme granularty T 2 n order to, e.g., avod epsodc local congestons due to the varablty of the ncomng worload at tme granularty T 2 around ts hourly average predcton see Fgure 2). Accordng to equaton 1), the response tme for executng locally WS class at ste.e., wthout consderng the networ delay due to redrects) s gven by: R = C µ 1 x + ) g, z G N +M Durng tme nterval T 2 the number of executons of class request at ste s T 2 x + T g, z 2 G, and the response tme for remote requests s gven by both R and the delay d, g, z G. Therefore the total response tme for executng g, z G. 166

5 class request at ste s: z G R = R + x + g, z ), G and the total response tme for class request over all stes s: x R = + g, z ) G R. The goal of our load redrect scheme s to cooperatvely mnmze request average response tmes. Formally the LR can be formulated as a constrant programmng problem snce R R must hold and the cost for request executon s determned by the CA soluton and s not nfluenced by the LR decson varables. However, n order to provde an effcent dstrbuted soluton, n our LR problem formulaton we consder the total requests response tme as the metrc to be mnmzed. Prelmnary expermental results have shown, ndeed, that ntroducng an obectve functon allows to speed up the dstrbuted algorthm convergence relyng on standard non lnear solvers. The LR problem can be formulated as follows: LR) Λ N + M ) x + mn x,z C µ N + M ) x + subect to x + g, z ) G + g, z G ) z G x + z = Λ K, I, 3) g, z G < C µ N + M ) K, I, 4) x, z 0 K, I. Constrants 3) ensure that the overall class requests at node are locally executed or are redrected toward the other stes, whle constrants 4) guarantee that VMs saturaton condtons are avoded. LR) defnes a centralzed load balancng problem: All the system nformaton.e., the local ncomng worload predctons Λ ) has to be gathered together and used to get the optmal worload balancng. However, for large scale cloud systems, ths centralzed load balancng scheme s not sutable. Even assumng that the broadcast of Λ values does not add a sgnfcant networ overhead n the system ndeed T 2 s around 5-10 mnutes), the soluton of the LR) problem for large system cannot be obtaned wthn the T 2 tme lmt wth the non lnear solvers currently avalable. For ths reason, we have devsed a dstrbuted decomposable soluton for problem LR) relyng on Lagrangan technques and obtanng closed formulas for elementary problems to be solved by applyng the Karush Kuhn Tucer KKT) condtons. Our mplementaton supports a dstrbuted protocol for LR) soluton n whch each ste solves ts problem usng both local nformaton and nformaton receved from other stes. In partcular, as dscussed n the followng, we develop an teratve method and at each teraton z s the only nformaton that wll be shared among stes. Our decomposton technque s founded on dualty theory n optmzaton, [9]. Frst of all, we observe that the optmzaton problem s convex see [4]), the dualty gap s zero and then the global optmum soluton can be dentfed [23] solvng the prmal va the dual. Secondly, LR) can be decomposed nto K ndependent sub-problems one for every class) whch can be obtaned from LR) smply omttng the ndex. Furthermore, LR s characterzed by two types of couplng: Couplng constrants, and coupled utltes [23]. Indeed, each term n the obectve functon not only depends on the local varable x x n the followng) but also on the varables of the other stes z z ). The ey dea to address coupled utltes s to ntroduce auxlary varables and addtonal equalty constrants. The LR) problem soluton then can be obtaned by solvng K problems as: LR ) [ N + M ) x + y ) ] mn x,y,z,w C µ N + M ) x + y ) + w subect to x + z = Λ I, 5) x + y < C µ N + M ) I, 6) y = g, z G I, 7) w = z G I, 8) x, y, z, w 0 I, 9) where x, y, and w are local varables at ste. Next, we consder the Lagrangan: RP ) mn x,y,z,w [ N + M ) x + y ) C µ N + M ) x + y ) + w + +Θ y g, z ) +η w G z ) G subect to constrants 5), 6), and 9), where the Θ s and η s are the consstency prces [23]. By explotng the decomposable structure of the Lagrangan, the relaxed problem RP) further separates nto I subproblems: ) +M ) x +y N SUB ) mn x,y,z,w C µ N +M ) x +y ) + w + +Θ y ) g, z G +η w ) z G subect to constrants 5), 6), and 9). The optmal value of RP) for a gven set of Θ s and η s defnes the dual functon LΘ, η) and the dual problem s then gven by: 167

6 D) max Θ,η LΘ, η). The dual problem can be solved by usng a subgradent method: Gven ntal values Θ 0) and η 0) the terates are generated by Θ t + 1) = Θ t) + α t y g, z η t + 1) = η t) + β t w G ), 10) z ), G 11) where t s the teraton ndex and α t, β t are suffcently small postve parameters see [23]). Note that, each update step n ths approach uses data from all of the stes. Ths method naturally lends tself to a dstrbuted mplementaton: Each ste updates ts prmal varables z whch n turn are broadcasted toward other stes. Then, each ste updates ts dual varables Θ, η usng only local sub-gradent) nformaton. The scheme for the soluton of each sub problem LR ) s reported n Algorthm 1. The soluton of each sub-problem SUB ) can be obtaned also very effcently by closed formula derved by the KKT condtons. Detals are omtted here for space lmtatons and can be found n [4]. The procedure reported n Algorthm 1 s stopped when the percentage dfference of the obectve functon of two consecutve teratons s wthn a gven precson. Algorthm 1: Lagrangan Dstrbuted Optmzaton Procedure 1) Intalzaton: Set t = 0, Θ0) and η0) equal to some values; 2) Each ste solves ts problem SUB ) and broadcast the soluton z not the auxlary varables y and w ); 3) Prce updatng: Each ste terates the consstency prces wth the terate n 10) and 11); 4) Set t t + 1 and go to Step 2 untl satsfyng stoppng crteron); VI. EXPERIMENTAL RESULTS The resource management algorthms proposed have been evaluated for a varety of system and worload confguratons. Secton VI-A presents the expermental settngs and the results on the scalablty of our algorthms. Secton VI-B presents a cost-beneft evaluaton of our soluton compared wth other heurstcs and state-of-the-art technques [13], [27], [26]. Fnally, Secton VI-C shows the results of the applcaton of our resource management algorthms n a real prototype envronment deployed n Amazon EC2. A. Algorthm Performance To evaluate the effcency of the proposed algorthms, we have used a large set of randomly generated nstances. All tests have been performed on VMWare vrtual machne based on Ubuntu 9.10 server runnng on an Intel Nehalem dual socet quad-core system wth 32 GB of RAM. The vrtual machne has a physcal core dedcated wth guaranteed performance and 4 GB of memory reserved. We used SNOPT as non lnear solver [20]. The number of cloud stes I has been vared between 20 and 60, the number of request classes K between 100 and We would le to remar that, even f the number of cloud stes s small n realty e.g., Amazon owns 14 avalablty zones spread over fve dfferent regons worldwde), we consder up to 60 stes. Recall from Secton II that n our approach a ste wth S VM confguratons s modelled by S stes wth a sngle VM confguraton. The maxmum servce rate of a capacty one VM for executng class requests, µ, s set R = 3/µ, as n [26]. Expermental results see [4] for further detals) have shown that the average executon tme requred to solve nstances of maxmum sze s lower than 3 mnutes and one mnute for the CA and LR problems, respectvely. Hence, both the CA and LR mechansms can be adopted at the consdered tme scales. B. Comparson wth alternatve lterature proposals We performed a cost-beneft evaluaton of our approach consderng other heurstcs wth a twofold am: On the one hand we compared our soluton wth other state-of-the-art technques whch explot the utlzaton prncple and determne the number of VM nstances accordng to an utlzaton threshold upper bound [13], [27], [26], [2]. On the other hand, the research queston we addressed was concernng the effectveness of the LR mechansm n the cloud. Indeed, n cloud systems the resource provsonng can be performed n very few mnutes and hence nstead of redrectng the load to other stes one can argue that the allocaton of addtonal VMs to manage pea of traffcs could be more effectve. In ths Secton we report the results of the comparson of our CA+LR mechansm wth a set of solutons whch perform a more fne graned CA at multple tme scales. In the remander of ths Secton the followng alternatve solutons wll be consdered: Heurstc 1: The CA s performed on a 5 mnutes tme horzon and the number of VMs s determned accordng to utlzaton thresholds as n other approaches proposed n the lterature [13], [27], [26] and currently mplemented also by IaaS provders see, e.g., the very recent release of Amazon AWS Elastc Beanstal [2]). In the evaluaton, a lfe span of one hour for each nstantated VM has been consdered. The number of VMs s determned such that the utlzaton of the VMs s equal to a gven threshold τ 1. The VM provsonng s further trggered f the predcton of the VMs utlzaton s hgher than a second threshold τ 2 > τ 1. Multple analyses have been performed by adoptng dfferent thresholds: τ 1, τ 2 ) = 40%, 50%), 50%, 60%), and 60%, 80%). Heurstc 2: Same as Heurstc 1 but the number of VMs s determned by optmally solvng the CA problem reported n Secton V-A every 5 mnutes. Heurstc 3: Same as Heurstc 2 but wth a 10 mnutes tme horzon. The performance parameters of the request classes have been randomly generated as n Secton VI-A, whle the local ncomng worload has been obtaned from the traces of a very large dynamc Web-based system mplementng a multter logcal archtecture descrbed n [11]. In our experments the followng daly traces have been consdered wth 5 mnutes sample tme nterval: Normal day scenaro: It descrbes the baselne worload 168

7 where the number of clents requests changes followng the b-modal requests profle shown n [7]. Heavy day scenaro: It exhbts a 40% ncrement n the number of the clent requests wth respect to the baselne worload. Nosy day scenaro: It s characterzed by the same request profle belongng to the heavy day scenaro wth an addtonal nose component we added a whte nose wth zero mean and standard devaton equal to 10% of the heavy day pea). In ths way, we ncrease the system varablty n order to prove the accuracy of the predcton model and the robustness of our overall soluton also n hghly varable contexts. All scenaros are representatve of the typcal Web-based worload that s characterzed by heavy-taled dstrbutons [14], [6]. Moreover, the heavy scenaros add burst arrvals and flash crowds [21] that contrbute to augment the request sew, and they represent a more stressful testbed for predcton models. The motvaton behnd ths choce s to demonstrate that our predcton algorthm wors even n crtcal scenaros and our CA+LR mechansm are robust to worload varablty, although the toughest goal of predctng hot spot events remans an open ssue beyond the scope of ths paper. In partcular, the predcton model consdered n ths paper s able to provde an accurate predcton qualty that, n terms of mean square error [12], s always lower than 10%. Overall we have consdered 12 stes, whch we assume are located n 12 dfferent tme zones wth a one hour tme lag and the normal, heavy, and nosy traces have been sewed accordngly. In the followng quanttatve analyss we set T 1 = 1 hour, and T 2 = 5 mnutes. Fgure 3 plots, as an example, the VM costs over the 24 hours for the nosy day scenaros the normal and heavy cases are very smlar), whle Table II reports the percentage savngs of our approach wth respect to the other heurstcs consderng the total costs over the whole day. Fgures 4 and 5 reports, as an example, the plot of the rato R of the average response tme wth respect to the response R tme threshold of a class consdered as a reference example at ste 1. The plot shape s pretty general and s ndependent of the consdered ste and request class. As the results show, the Heurstc 1 s very senstve to the thresholds adopted. The 40%, 50%) case s very conservatve, t s around 35% more expensve than our approach but always allows to guarantee the response tme threshold the rato s strctly lower than 1). Vce versa the 60%, 80%) case provdes costs close to our soluton only 2-4% hgher) but ntroduces a very large number of SLA volatons especally n the nosy day scenaro see Fgure 5). Vce versa, our soluton ntroduced overall only 37 volatons over the 3456 tme ntervals consdered n the 12 stes, over the whole day. Furthermore, Heurstcs 1 s more senstve to traffc varablty. Heurstcs 2 and 3 perform better than Heurstcs 1, snce the number of VMs s optmally determned by the CA) problem solutons. However, the LR mechansm s stll effectve snce allows to reduce costs by Alternatve soluton % Savngs Normal day Heavy day Nosy day Heurstc 1-40%, 50%) Heurstc 1-50%, 60%) Heurstc 1-60%, 80%) Heurstc Heurstc TABLE II VM PERCENTAGE COST SAVINGS OVER THE 24 HOURS OBTAINED BY OUR APPROACH. 4-12%. The fne graned resource allocaton ntroduced by Heurstcs 2 and 3 ndeed ends nto an over-provsonng and better performance see Fgures 4 and 5), whle the LR mechansm allows to forward traffc spes to other locatons wthout overcomng n any addtonal capacty allocaton or sgnfcant SLA volatons. C. Amazon EC2 Test The effectveness of our resource management algorthms has been also evaluated on Amazon EC2 performng experments runnng the JSP mplementaton of the SPECweb benchmar. In partcular, we have consdered the banng worload, whch smulates the access to an on lne banng Web ste mplementng a full HTTPS load. The Web server Apache Tomcat n our setup) has been deployed on a large nstance, whle the load generators, the clent coordnator, and the bac-end smulator have been hosted by extra-large Amazon nstances n ths way we are guaranteed that they are not the system bottlenec). The test s performed deployng VM nstances n Vrgna and North Calforna Amazon regons. We have obtaned an estmate of the maxmum servce rate parameters and the networ delay among dfferent Amazon stes by performng an extensve off lne proflng along the lnes of [22]. We set R = 0.7 seconds as threshold for the average response tme and the overall test lasts one hour. We have generated an approprate traffc profle and run the CA algorthm at tme 0 and at tme 40 mnutes. The LR algorthm s run every 10 mnutes. Durng the frst 40 mnutes the CA soluton allocates two on demand Web server nstances at the two Amazon stes. Durng the last 20 mnutes the load s evenly shared by ntroducng the Amazon Elastc Load Balancer among three on demand Web server nstances n the Vrgna regon and fve n North Calforna. The Vrgna local ncomg worload s redrected to North Calforna from mnute 30 to 40, whle t s redrected from North Calforna to Vrgna durng the last 20 mnutes. Fgures 6 and 7 show the the overall traffc served at the two stes. Fgure 8 reports the end users average response tme and shows that our CA+LR algorthms are effectve snce the system provde performance accordng to the SLA for most of the tme and t s able to react to abrupt worload varatons. VII. CONCLUSIONS We proposed predcton-based dstrbuted CA and LR algorthms for IaaS cloud system mnmzng the cost of the 169

8 Fg. 3. scenaro. VM nstances costs for the nosy day Fg. 4. Response tme threshold rato for a reference Fg. 5. Response tme threshold rato for a reference class, normal day scenaro. class, nosy day scenaro. Fg. 6. Overall traffc served at Vrgna EC2 ste. Fg. 7. Overall traffc served at North Calforna EC2 ste. Fg. 8. Average response tme measured for the SPECweb2005 banng worload. runnng VMs. Expermental results shown that our solutons sgnfcantly mprove other heurstcs proposed n the lterature 5-35% on average), wthout ntroducng sgnfcant QoS volatons. Future wor wll extend the valdaton of our soluton consderng a larger expermental setup. REFERENCES [1] B. Abraham and J. Ledolter. Statstcal Methods for Forecastng. John Wley and Sons, [2] Amazon Inc. Amazon Elastc Cloud. [3] M. Andreoln, S. Casolar, and M. Colaann. Autonomc request management algorthms for geographcally dstrbuted nternet-based systems. In SASO, [4] D. Ardagna, S. Casolar, and B. Pancucc. Flexble dstrbuted capacty allocaton and load redrect algorthms for cloud systems. Poltecnco d Mlano, Tech. Report [5] D. Ardagna, B. Pancucc, M. Truban, and L. Zhang. Energy-Aware Autonomc Resource Allocaton n Mult-ter Vrtualzed Envronments. IEEE Trans. on Servces Computng, avalable on lne. [6] M. Arltt, D. Krshnamurthy, and J. Rola. Characterzng the scalablty of a large Web-based shoppng system. 11):44 69, Aug [7] Y. Baryshnov, E. Coffman, G. Perre, D. Rubensten, M. Squllante, and Y. Ymwadsana. Predctablty of web server traffc congeston. In WCW Proc., [8] M. Bennan and D. Menascé. Resource Allocaton for Autonomc Data Centers Usng Analytc Performance Models. In IEEE Int l Conf. Autonomc Computng Proc., [9] D. Bertseas. Nonlnear Programmng. Athena Scentfc, [10] G. Bolch, S. Grener, H. de Meer, and K. Trved. Queueng Networs and Marov Chans. J. Wley, [11] H. W. Can, R. Rawar, M. Marden, and M. H. Lpast. An archtectural evaluaton of Java TPC-W. In HPCA Proc., [12] S. Casolar and M. Colaann. On the selecton of models for runtme predcton of system resources. Autonomc Systems, Sprnger Eds. Danlo Ardagna, L Zhang), [13] L. Cherasova and P. Phaal. Sesson-Based Admsson Control: A Mechansm for Pea Load Management of Commercal Web Stes. IEEE Transactons on Computers, 516), June [14] M. E. Crovella, M. S. Taqqu, and A. Bestavros. Heavy-taled probablty dstrbutons n the World Wde Web. In A Practcal Gude To Heavy Tals, pages Chapman and Hall, New Yor, [15] M. D. Daaos, D. Katsaros, P. Mehra, G. Palls, and A. Vaal. Cloud Computng: Dstrbuted Internet Computng for IT and Scentfc Research. IEEE Internet Computng, 135):10 13, [16] H. Erdogmus. Cloud computng: Does nrvana hde behnd the nebula? IEEE Softw., 262):4 6, [17] S. Everette and J. Gardner. Exponental smoothng: State of the art. Journal of Forecastng, 4, [18] P. Felber, T. Kaldewey, and S. Wess. Proactve hot spot avodance for web server dependablty. Relable Dstrbuted Systems, IEEE Symposum on, pages , [19] H. Feng, Z. Lu, C. H. Xa, and L. Zhang. Load sheddng and dstrbuted resource control of stream processng networs. Perform. Eval., ): , [20] P. E. Gll, W. Murray, and M. A. Saunders. SNOPT: An SQP algorthm for large-scale constraned optmzaton. SIAM Journal of Optmzaton, 12: , [21] J. Jung, B. Krshnamurthy, and M. Rabnovch. Flash crowds and denal of servce attacs: Characterzaton and mplcatons for CDNs and Web stes. In WWW2002 Proc., Honolulu, HW, May [22] G. Pacfc, W. Segmuller, M. Spretzer, and A. Tantaw. Cpu demand for web servng: Measurement analyss and dynamc estmaton. Perform. Eval., 656-7): , [23] D. P. Palomar and M. Chang. A tutoral on decomposton methods for networ utlty maxmzaton. IEEE J. Sel. Areas Commun, 24: , [24] D. Trgg and A. Leach. Exponental smoothng wth an adaptve response rate. Operatonal Research Quarterly, 18, [25] B. Urgaonar, G. Pacfc, P. J. Shenoy, M. Spretzer, and A. N. Tantaw. Analytc modelng of multter Internet applcatons. ACM Transacton on Web, 11), January [26] A. Wole and G. Mexner. Twospot: A cloud platform for scalng out web applcatons dynamcally. In ServceWave, [27] X. Zhu, D. Young, B. Watson, Z. Wang, J. Rola, S. Snghal, B. McKee, C. Hyser, D.Gmach, R. Gardner, T. Chrstan, and L. Cherasova: slands: An ntegrated approach to resource management for vrtualzed data centers. Journal of Cluster Computng, 121):45 57, ACKNOWLEDGEMENT The wor of Danlo Ardagna and Barbara Pancucc has been partally supported by the GAME-IT and IDEAS- ERC Proect SMScom research proects. Sara Casolar acnowledges the support of MIUR-PRIN proect DOTS-LCCI. Thans are expressed to Prof. Mchele Colaann for hs frutful comments on the prelmnary versons of ths paper. 170