2. SYSTEM MODEL. the SLA (unlike the only other related mechanism [15] we can compare it is never able to meet the SLA).

Transcription

1 Managng Server Energy and Operatonal Costs n Hostng Centers Yyu Chen Dept. of IE Penn State Unversty Unversty Park, PA yzc107@psu.edu Anand Svasubramanam Dept. of CSE Penn State Unversty Unversty Park, PA anand@cse.psu.edu Amtayu Das Dept. of CSE Penn State Unversty Unversty Park, PA adas@cse.psu.edu Qan Wang Dept. of ME Penn State Unversty Unversty Park, PA quw6@psu.edu Wub Qn Dept. of ME Penn State Unversty Unversty Park, PA wqn@psu.edu Natarajan Gautam Dept. of IE Penn State Unversty Unversty Park, PA ngautam@psu.edu ABSTRACT The growng cost of tunng and managng computer systems s leadng to out-sourcng of commercal servces to hostng centers. These centers provson thousands of dense servers wthn a relatvely small real-estate n order to host the applcatons/servces of dfferent customers who may have been assured by a servce-level agreement (SLA). Power consumpton of these servers s becomng a serous concern n the desgn and operaton of the hostng centers. The effects of hgh power consumpton manfest not only n the costs spent n desgnng effectve coolng systems to ward off the generated heat, but n the cost of electrcty consumpton tself. It s crucal to deploy power management strateges n these hostng centers to lower these costs towards enhancng proftablty. At the same tme, technques for power management that nclude shuttng down these servers and/or modulatng ther operatonal speed, can mpact the ablty of the hostng center to meet SLAs. In addton, repeated on-off cycles can ncrease the wear-and-tear of server components, ncurrng costs for ther procurement and replacement. Ths paper presents a formalsm to ths problem, and proposes three new onlne soluton strateges based on steady state queung analyss, feedback control theory, and a hybrd mechansm borrowng deas from these two. Usng real web server traces, we show that these solutons are more adaptve to workload behavor when performng server provsonng and speed control than earler heurstcs towards mnmzng operatonal costs whle meetng the SLAs. Categores and Subject Descrptors: I.6.5 [Model Development]: Modelng methodologes; D.2.8 [Metrcs]: Performance measures Contact author Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. To copy otherwse, to republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. SIGMETRICS 05, June 6 10, 2005, Banff, Alberta, Canada. Copyrght 2005 ACM /05/ $5.00. General Terms: Desgn, Performance Keywords: Energy Management, Performance Modelng, Feedback, Server Provsonng 1. INTRODUCTION The motvaton for ths paper stems from two growngly mportant trends. On the one hand, the cost and complexty of system tunng and management s leadng numerous enterprses to offload ther IT demands to hostng/data centers. These hostng centers are thus makng a consderable nvestment n procurng and operatng servers to take on these demandng loads. The other trend s the growng mportance of energy/power consumpton of these servers at the hostng centers, n terms of the electrcty cost to keep them powered on, as well as n the desgn of extensve coolng systems to keep ther operatng temperatures wthn thermal stablty lmts for server components. A careful balance s needed at these centers to provson the rght number of resources to the rght servce/applcaton beng hosted, at the rght tme, n order to reduce ther operatonal cost whle stll meetng any performance based servce level agreement (SLA) decded upon earler. Focusng specfcally on the energy consumpton problem, ths paper presents three man technques - a pro-actve one, a reactve one, and a hybrd between the two - to dynamcally optmze operatng costs whle meetng performance-based SLAs. The growth of network-based commercal servces, together wth off-loadng of IT servces, s leadng to the growth of hostng/data centers that need to house several applcatons/servces. These centers provson thousands of servers, and data storage devces, to run these applcatons, earnng revenue from these applcaton provders (customers) n return - sometmes referred to as on-demand computng. Over-provsonng the servce capacty to allow for worst case load condtons s economcally not very attractve. Consequently, much of pror work (e.g. [9, 31, 36]) has looked at fndng the rght capacty, and dstrbutng ths capacty between the dfferent applcatons based on ther SLAs. However, there could stll be tme perods durng the executon when the overall server capacty s much hgher than the current demands across the applcatons. It should be noted that the hostng center s stll ncurrng operatonal costs, such as electrcty, durng such perods.

2 Energy consumpton of these hostng/data servers s becomng a serous concern. As several recent studes have ponted out [11, 12, 22, 23, 29] data centers can consume several Megawatt. It s not just the cost of powerng these servers, but we need to also nclude the cost of deployng coolng systems (whch n turn consume power) to ensure stable operaton [27]. We are at power denstes of around 100 Watts per square feet, and the coolng problem s expected to get worse wth shrnkng form factors. Fnally, one also needs to be concerned wth the envronmental ssues when generatng and delverng such hgh electrc power capactes. Statcally provsonng the number of servers for a gven cost (say of electrcty) and performance SLA can be conservatve or mss out on opportuntes for savngs, snce workloads are typcally varyng over tme. Dynamc power management s thus very mportant when deployng server farms/clusters, and hardware has started provdng assstance to acheve ths. For nstance, n addton to powerng down a server, most processors today allow dynamc voltage/frequency scalng (DVS), where the frequency (and voltage) can be lowered to produce much more savngs n power consumpton compared to how much one looses on performance. Whle ths may not provde as much savngs as shuttng down a server completely, the advantage s that t can stll servce requests (albet at a lower frequency), and does not ncur as hgh costs (whether t be tme for transtonng between frequences or n terms of wear-andtear assocated wth repeated server on-off cycles). Much of the earler work [11, 29] n ths area, used server turn off/on mechansms for power management. The only other work that consdered a combnaton of these two mechansms (turn off and DVS) was done at IBM [15]. However, ths work dd not (nor dd the others) consder the mpact of server off/on cycles on the long term relablty of server components due to wear-and-tear. Note that falng of components ncurs addtonal costs (hardware and personnel) for procurement and replacement. Further, none of the pror studes have really consdered the goal of meetng a response tme requrement (SLA). Rather, they have tred to optmze energy frst, at a slght degradaton n response tme. For a hostng center, t s more mportant to meet the requred SLA n terms of response tme (snce that s the revenue maker), and reducng the energy consumpton when meetng ths goal should be a desrable (rather than vce-versa). As our results wll show, not treatng the performance-based SLA n the dynamc optmzaton as a frst class ctzen causes these other schemes to fall short, even though they may provde more energy savngs. Fnally, most of the pror work has just looked at optmzng the energy n a sngle applcaton settng, assumng the exstence of a hgher level server provsonng mechansm across applcatons. Our framework on the other hand, ntegrates server provsonng across applcatons (whch automatcally tells us how many servers to turn on/off) wth energy management wthn an applcaton, n an elegant fashon whle stll allowng dfferent optons for these two steps. Ths paper presents the frst formalzaton to ths dynamc optmzaton problem of server provsonng and DVS control for multple applcatons, whch ncludes a response-tme SLA, and the costs (both power and wear-and-tear) of server shutdowns. We present three new onlne mechansms to ths problem. The frst s pro-actve n that t predcts workload behavor for the near future, and uses ths nformaton n a stochastc queung model to perform control. The second s reactve n that t uses feedback control to acheve the same goals. The thrd s a hybrd scheme where we use predctve nformaton from the frst for server provsonng, and feedback control of the latter for DVS. Usng real web-server workloads n a mult-applcaton settng, we show that all these schemes can provde good energy savngs wthout gnorng the SLA (unlke the only other related mechansm [15] we can compare t s never able to meet the SLA). 2. SYSTEM MODEL Our hostng center model contans a number of dentcal servers, called a server cluster, whch are all equally capable of runnng any applcaton. Whle most of our system models and optmzaton mechansms are applcable to a dverse spectrum of applcaton domans (from those n the scentfc doman to those n the commercal world), the mplementaton and evaluatons are manly targeted towards server based applcatons (web servers n partcular) that servce clent requests. Each HTTP request s drected to one of the servers runnng the applcaton, whch processes the request n a tme (servce tme) that s related to a request parameter (e.g. fle sze). Consequently, one can mpact the throughput/response tme for these requests by modulatng the number of servers allocated to an applcaton at any tme. Dense blade systems are beng ncreasngly deployed n hostng centers because of ther smaller form factors (makng t easer to rack them up), and power consumpton characterstcs. Such systems provde a farly powerful (server-class) processor such as an Intel Xeon, 1-2 GB of memory, and perhaps a small SCSI dsk. There can be several hundreds/thousands of these blade servers n a hostng center, all consumng electrc power dependng on whether they are turned on, and f so at what frequency. From the vewpont of the customers (applcatons) of the hostng center, t s mportant to have at least as many servers as necessary for that applcaton n order to meet a desred level of performance (SLA) for ts clent requests. In ths exercse, we use a smple SLA that tres to bound the average response tme for requests, and a more extensve SLA could have been used as well. From the vewpont of the hostng center, the goal s to consume as lttle electrcal energy as possble whle stll ensurng that the ndvdual applcatons are able to meet ther end-user SLAs for clent requests. A schematc of the system envronment s gven n Fgure 1. Fgure 1: System Model The workload mposed on a server s resources when runnng an applcaton expends electrc power, thereby ncurrng a cost n ts operaton. The mportant resources of concern nclude semconductor components such as the server CPU, caches, DRAMs, and system nterconnects, as well as electro-mechancal components such as dsks. There are two mechansms avalable today for managng the power consumpton of these systems: One can temporarly power down the blade, whch ensures that no electrcty flows to any component of ths server. Whle ths can provde the most power savngs, the downsde s that ths blade s not avalable to serve any requests. Brngng up

3 the machne to serve requests would ncur addtonal costs, n terms of () tme and energy expended to boot up the machne durng whch requests cannot be served, and () ncreased wear-and-tear of components (the dsks n partcular) that can reduce the mean-tme between falures (MTBF) leadng to addtonal costs for replacements and personnel. Another common opton for power management s dynamc voltage/frequency scalng (DVS). As wll be shown n secton 4, the dynamc power consumed n crcuts s proportonal to the cubc power of the operatng clock frequency. Slowng down the clock allows the scalng down of the supply voltages for the crcuts, resultng n power savngs. Even though not all server components may be exportng software nterfaces to perform DVS, CPUs [2] - even those n the server market - are startng to allow such dynamc control. Wth the CPUs consumng the bulk of the power n a blade server (note that an Intel Xeon consumes between Watts at full speed, whle the other blade components ncludng the dsk can add about Watts), DVS control n our envronment can provde substantal power savngs. Our framework allows the employment of both these optons towards enhancng energy savngs, wthout compromsng on enduser SLAs. In our model, when a machne s swtched off, t consumes no power. When the machne s on, t can operate at a number of dscrete frequences f1 < f2 <... < fl, where the relatonshp between the power consumpton and these operatng frequences s of the form P = P fxed + P f.f 3 (see secton 4), so that we capture the cubc relatonshp wth the CPU frequency whle stll accountng for the power consumpton of other components that do not scale wth the frequency. DVS mplementaton for a server cluster of these blades can be broadly classfed based on whether () the control s completely decentralzed, where each node ndependently makes ts scalng choce purely on local nformaton, or () there s a coordnated (perhaps centralzed) control mechansm that regulates the operaton of each node. Though decentralzed DVS control s attractve from the mplementaton vewpont, prevous research [15] has shown that a coordnated voltage scalng approach can provde substantally hgher savngs. In our system model, we use a coordnated DVS strategy where a controller s assgned for each applcaton and t perodcally assgns the operatng frequency/voltage for the servers runnng that applcaton. We perform ths at an applcaton granularty snce the load for each applcaton, and correspondng SLA requrements, can be qute dfferent. Further, n our model, each server contnues to be entrely devoted to a sngle applcaton untl the tme of server reallocaton. Wth ths model of the hostng center, the two soluton steps to the problem at hand are to () perform server provsonng to decde how many servers to allocate to each applcaton, and () decde what should be the operatng frequency for the servers allocated to each applcaton. These steps need to be done perodcally to account for tme-varyng behavor. Note that the frst step may requre brngng up/down servers and/or re-assgnng servers. The tme cost for such mgratons of applcatons (T mgrate) and/or reboots (T reboot ) are ncorporated n our model. Further, there s a long term mpact of server reboots due to wear-and-tear of components such as the dsk, and we nclude ther dollar cost. 3. RELATED WORK strngent servce-level agreements to be met, there have been several nvestgatons nto resource capacty plannng and dynamc provsonng ssues for QoS control (e.g. [5, 31, 34, 38]). As many studes have ponted out, over-provsonng of resources can be economcally unattractve, and t s mportant to accommodate transent overload stuatons (whch are qute common for these servces) wth the exstng resources [9]. Dynamc load montorng (e.g. [30]), transent-load based optmzaton (e.g. [9, 36]) and feedback-based technques (e.g. [3, 18, 26]) are beng examned to handle these stuatons. However, these studes have manly focused on performance (and revenue based on an SLA), and have not examned the power consumpton ssues. Energy Management n Moble Devces:. Energy management has tradtonally been consdered mportant for moble and resource-constraned envronments that are lmted by battery capactes. One common technque s to shut off components (e.g. the dsk [21],[24]) durng perods of nactvty. The nfluence that the frequency (whch allows the voltage to be scaled down) has on the power consumpton has been exploted to mplement Dynamc Voltage Scalng (DVS) mechansms for energy management of ntegrated crcuts [10], [32]. Consequently, many processors - not only those for the moble market, but even those n the server space [2] - today are startng to offer software nterfaces for DVS. There have been numerous studes on explotng DVS for power savngs wthout compromsng on performance (e.g. [17], [19],[25], [28], [37]). However, all these studes have been for moble devces that have very dfferent workload patterns than servers, and/or for embedded envronments where soft/hard real-tme constrants need to be met wthn a power budget. Energy Management n Servers and Data Centers:. It s only recently that energy management for server clusters found n data/hostng centers has ganed much attenton [8, 12, 16, 20, 23, 29]. Amongst these early studes, [11, 12, 29] show that resource allocaton and energy management can be ntertwned by developng technques for shuttng down servers that are not n use, or have low load (by offloadng ther dutes to other servers). A detaled study of the power profle of a real system by researchers at IBM Austn [6] ponts out that the CPU s the largest consumng component for typcal web server confguraton. Subsequent studes [15, 33] have looked at optmzng ths power, by montorng evolvng load and performance metrcs to dynamcally modulate CPU frequences. A categorzaton of these most closely related nvestgatons s summarzed n Table 1, n terms of whether () the schemes consder just server shutdowns or allow DVS, () they focus on energy management of just 1 server (whch can be thought of as completely ndependent management of a server wthout regard to the overall workload across the system), () they consder dfferent applcatons to be concurrently executng across these servers wth dfferent loads, v) they try to meet SLAs mposed by these applcatons, and (v) they ncorporate the cost of rebootng servers (n terms of both tme and MTBF). Of the four related studes shown here, only two [15, 33] have consdered the possblty of DVS for server applcatons. Of these two, only the IBM work [15] has examned the ssues n the context of a server cluster, wth the other focusng on a sngle machne/server settng. However, even the IBM work has not consdered the ssues n the context of multple applcatons beng hosted on these server clusters (.e. the server provsonng problem n conjuncton wth energy management), nor have they consdered the long term mpact of machne reboots on operatng costs. Resource Provsonng n Hostng/Data Centers:. Snce many of the servces/applcatons hosted at these centers can have

4 DVS? Multple Multple SLA? Reboot servers? applns.? cost? Duke [11] No Yes(5) Yes(2) Yes Yes(Tme) Rutgers [29] No Yes(8) No No Yes(Tme) Vrgna [33] Yes No No No No IBM [15] Yes Yes(10) No No No Ours Yes Yes Yes Yes Yes Table 1: Comparng the most related studes wth ths work. The numbers gven n parenthess ndcate number of servers/applns. used n the study. The Tme for the reboot cost ndcates that only the tme overhead of rebootng systems was consdered (and not the wear-and-tear). 4. PROBLEM FORMULATION We model our hostng centers to have M dentcal servers, all equally capable of runnng any applcaton. Our model allows a maxmum of N dfferent applcatons to be hosted at any tme across these servers, wth each applcaton beng allocated m servers at any tme ( P m M). When a server s operatonal, t can run between a maxmum frequency f max (consumng the hghest power) and a mnmum frequency f mn, wth a range of dscrete operatng frequency levels n-between. The servce tme of a request that comes to an applcaton s mpacted (lnearly) by the frequency, and the overall servce capacty allocated to an applcaton s also drectly related to the number of servers allotted to t. A server CPU operatng at a frequency f consumes dynamc power (P ) that s proportonal to V 2 f, where V s the operatng voltage [10]. Further, underlyng crcut desgn nherently mposes a relatonshp between the operatng voltage and crcut frequency. For nstance, as we lower the voltage (to reduce power), the frequency needs to be reduced n order to allow for the crcuts to stablze. The common rule of thumb for ths relatonshp s gven by V f. Snce the power consumpton of all other components (except the CPU) s ndependent of the frequency, we have the followng smple model of power consumed by one cluster node runnng at frequency f: P = P fxed + P f f 3. Ths cubc relatonshp between operatng frequency and power consumpton has been used n other related work as well [15]. We can then calculate the energy consumpton of the servers at the hostng center operatng at a constant frequency f, over tme t,as Z E = M P dt = M (P fxed + P f f 3 ) t t Snce the real workload s expected to be tme varyng, we need to control M and f over tme to manage the energy consumpton whle adherng to any performance goals. Let us say that these are controlled perodcally at a granularty of tme t. Electrcty Cost of Operaton:. Over a duraton of Z such P tme unts of duraton t, the overall energy consumpton s Z P N z=1 = m(z) (P fxed + P f f (z) 3 ) t, where m (z) s the number of servers allocated to applcaton durng the z-th tme perod, whch are all runnng at a frequency f (z). IfK $ s the electrcty charge expressed as dollars per unt of energy consumpton (e.g. klowatt hour), then we have the total electrcty P cost (n dollars) for operaton of the hostng center to be Z P N z=1 K $ m (z) (P fxed + P f f (z) 3 ) t Cost of Server Turn-ons:. One mportant pont to consder s the mpact of turnng on/off servers. Whle frequency scalng s tself relatvely smple (takng only a few cycles), and may not sgnfcantly affect the long term relablty of the system, the effects of turnng off/on the servers should not be overlooked. Machne reboots can last several seconds/mnutes, consumng power n the process. Further, t s well understood that server components (dsks n partcular) can be susceptble to machne start-stop cycles [14], thus reducng the MTBF when we perform server shutdowns. If B o s the dollar cost of a sngle server turn-on cycle, then the total amount of dollars expended by any mechansm usng ths approach can be calculated as P Z z=1 Bo h PN m(z) P N m(z 1) + Note that the term x + = x when x 0 and x + = 0 otherwse. B o tself needs to account for the dollar energy cost for brngng up the machne (P T reboot K $ ), together wth the dollar cost ncurred by a smaller MTBF (denoted as C r),.e. B o = P max T reboot K $ + C r, where P max denotes the server power consumpton when runnng at f max. Puttng these together, we get the dollar cost of operaton to be ZX NX K $ m (z) (P fxed + P f f (z) 3 ) t z=1 " N # +! X NX +B o m (z) m (z 1) whch s the objectve functon to mnmze. Note that one could choose to mnmze ths objectve by nordnately slowng down the frequences, or shuttng down a large number of servers. Ths can result n volatng any servce level agreements (to meet a response tme requrement) to customers/applcatons. We use a smple SLA, where the requrement s to meet an average response tme target. Consequently, when mnmzng the above objectve functon, we need to obey the followng constrants: W W, whch s the SLA to constran the average response tme of applcaton to a target W. f (z) F=(f1,f2,...fl) where f1 <f2 <... < fl, whch says that a runnng server has to operate at one of the l dscrete frequences. We use f mn to denote f1 and f max to denote fl. P N m (z) <= M, snce the total number of allocated servers has to be less than total capacty. 5. METHODOLOGY We next dscuss three mechansms to determne the number of servers (m ) allocated to each applcaton and ther frequency (f ) at any nstant. Note that all m servers of applcaton run at the same frequency f as explaned n secton 2. To accommodate tme-varyng workload behavor, we assume that every T mnutes, servers are allocated for the N applcatons (.e. for all, we determne m ). Lkewse, every t mnutes, we determne the frequency f for the m servers of applcaton. Although not requred for modelng, for logcal and mplementaton reasons we assume that T s an nteger multple of t. Thereby at larger tme granulartes T, we perform server allocaton/provsonng, and at smaller tme granulartes (t), we tune the servers to dfferent frequences. The basc premse for dong ths at 2 granulartes, wth T > t s that server re-allocaton (machne reboots and applcaton mgraton) s much more tme-consumng than changng the frequency. There are U ntervals of length T present n the entre duraton for whch we want to optmze, and each of these (denoted by u =1to U) has S ntervals of length t (.e. s =1to S). Ths s pctorally shown

5 Fgure 2: Granularty (T,t) for server allocaton and frequency modulaton n fgure 2. We frst propose two technques - non-lnear optmzaton based on steady-state queung analyss, and a technque based on control theory - to determne f and m at every t and T mnutes respectvely. Then, we llustrate a hybrd approach ntegratng these technques. 5.1 Queung Theory Based Approach We can model the system under consderaton as a set of N parallel queues denotng the N applcatons. Clent requests for applcaton arrve one by one nto ts correspondng nfnte capacty queue (queue-). Settng m and f by analyzng ths queung system nvolves three phases. Frst, we need to predct request arrval pattern and servce tme requrements. Next, we can use the predcted arrval and servce tme nformaton n a queung model to determne the mean response tme for each applcaton. Fnally, we can use the response tme nformaton n our optmzaton problem, for whch we descrbe a tractable soluton strategy Predcton To perform queung analyss n each nterval, we requre estmates of mean arrval rate of requests (λ), squared coeffcent of varaton of request nter-arrval tmes (Ca), 2 mean fle sze n bytes (φ), and squared coeffcent of varaton of fle sze (Cs 2 ) for each applcaton. Notce that all four of the above parameters are tme varyng as well as stochastc, requrng a predcton of ther values at each nterval. Hstorcally, researchers have used ether selfsmlar traffc patterns or used determnstc tme-varyng parameters. On one hand, self-smlar models are attractve for descrbng the nature of the traffc and generatng synthetc workloads, but they do not lend themselves well for analyss. On the other hand, tme-varyng models requre Posson assumptons and full knowledge of the tme-varyng behavor. It s mportant to note that t s not our goal here to determne the best predcton technque. However, our contrbuton s that f the parameters are well predcted, we can approxmate the nter-arrval tmes and fle sze requrements as ndependent and dentcally dstrbuted (..d.) nsde each nterval and use ths convenently for onlne optmzaton. We present a smple predcton technque wth the dsclamer that ths can surely be mproved upon. From one tme nterval to another, for the mean nter-arrval tmes and standard devaton of nter-arrval tmes, we use a multplcatve S-ARMA (seasonal autoregressve movng average) [35]. We use several ntervals of tranng data to tune the S-ARMA model. Then, based on the cumulatve average, seasonalty effects, as well as the value at a few prevous ntervals, we dentfy the parameters for a gven nterval. For mean and standard devaton of fle sze we use a smple decomposed model or Wnter s smoothng method [35]. In general, we found that the predcton works reasonably well for arrval rates, but the errors can be hgher n terms of fle szes n certan cases. However, as we mentoned, our goal here s not necessarly to derve the best predcton mechansm. Note that the predcton values wll be computed dynamcally at the begnnng of each nterval durng the actual executon wth the real data Queung Analyss Usng the predctons of λ(), Ca(), 2 φ(), and Cs 2 () for a gven nterval for each applcaton, we next obtan an expresson for the predcted average response tmes (W ) n that gven nterval. We approxmate queue- to be a G/G/m queue wth..d. nterarrval tmes and..d. servce tmes. Further, we assume that the tme nterval s large enough that steady state results can be used (note that ths s one of the reasons why results wth ths technque may not be very good for small tme granulartes of control as our evaluatons wll show). There are several approxmatons for response tme n the lterature, of whch many are emprcal. In ths study, we use the method n Bolch et al [7], whch states that W = φ() β f + αm φ() β f 1 1 ρ «C 2 a ()+C 2 s () 2m «, (1) where β f s the bandwdth of the server n bytes served per second for applcaton (β s a constant that s calculated by measurng the servce tme of HTTP requests of dfferent szes on an m +1 2 actual system), ρ = λ()φ() m β f, and α m = ρ. Notce that W n Equaton (1) s non-lnear wth respect to the decson/control varables f and m. Also, we do not need to wrte down the constrant ρ < 1 explctly n the optmzaton problem as satsfyng the SLA constrant would automatcally ensure ths. Note that W decreases wth respect to both m and f. Therefore the SLA constrant would be bndng f f and m were contnuous. However n the dscrete case, we can obtan an effcent fronter (.e. for every f we can fnd the smallest m that wll render the SLA constrant feasble). As there are l levels for f,we wll have to consder only l pars of m and f for each applcaton n each nterval, as s exploted below Solvng the optmzaton problem We use s and u n all the parameters to denote the correspondng subscrpts for S and U ntervals as shown n Fgure 2. For example, m (u) s the number of servers n queue- at the u th nterval of duraton T, and f (u, s) s the frequency of a server n queue- at the s th nterval of duraton t for ths gven u. The optmzaton problem can then be re-wrtten n terms of the decson varables f (u, s) and m (u) (assume for all, m (0) = 0) as: mn f (u,s),m (u) UX u=1 SX s=1 NX K $ m (u)(p fxed + P f f (u, s) 3 )t " N # +! X NX + B o m (u) m (u 1) subject to W W for all NX m (u) M for all u, and m (u) s a non-negatve nteger f F Clearly, the above optmzaton s non-lnear and dscrete n terms of the decson varables for both the objectve and the constrants. Hence standard optmzaton technques are not enough. Snce we only consder a fnte number of frequency alternatves, t s temptng to do a complete enumeraton to determne the optmal soluton. However, a complete enumeraton would requre comparng O(l S M U N) values - and performng these perodcally durng executon. Therefore we resort to a heurstc as explaned below.

6 We revse the predcton model descrbed above dynamcally based on the prevous observaton (for u 1), and then run the followng two steps at the begnnng of each u. We explan the steps brefly wthout gong nto detals. We frst consder the case t = T, and therefore for all ntervals s = 1. Under ths framework, we use f (u, s) =f (u) for ease of notaton. Step 1: We frst obtan a feasble soluton that would result n an upper bound to the objectve functon. We consder only a sngle nterval and optmze the parameters f (u) and m (u) for the smplfed objectve as below: mn f (u),m (u) P N K $m (u)(p fxed + P f f (u) 3 )t +B 0 P N m(u) There are two ways of solvng ths, one s to assume the decson varables (m (u) and f (u)) are contnuous and use standard nonlnear programmng technques. Ths s especally useful when the problem s scaled to a large number of applcatons. However, we use a second method where we explot the monotoncty propertes of the objectve functon and constrants as well as the fact that the frequences take only a small number of dscrete values. For that purpose, we use estmates of moments of nter-arrval tmes and fle szes, across all applcatons and across all ntervals. Then, for each nterval u we start by fndng the mnmum number of servers m (u) for all applcatons so that the constrant W (u) W can be satsfed usng the hghest frequency for the servers. Note that the total number of servers n the hostng center M s large enough that ths would guarantee the constrant P N m(u) M to be automatcally satsfed - otherwse t mples the SLA has been chosen poorly snce t would have been volated even wthout any energy management. Ths soluton can be mproved by recognzng that the objectve s a cubc n terms of the frequency that needs to be reduced. Therefore by sutably decreasng f (u) and ncreasng m (u), as long as the constrants are satsfed, we can obtan a soluton that would be close to the optmal soluton,.e. we are tryng to fnd the number of servers that are needed when they are all operatng at lower frequences that can gve the most power savngs. When reducng f (u), we select applcatons n decreasng order of f (u) for each nterval. Step 2: We next consder all the ntervals together. The actual objectve functon that s the total cost across the entre executon, s used for ths purpose. We use the upper bound soluton from the prevous step and work our way acceptng feasble solutons that reduce the objectve functon value. If a pont n ths feasble space mproves the objectve, we accept that soluton and search further n ts neghborhood. In our approach we consder one nterval at a tme (gong from the frst to the last) and n each nterval we select the applcatons n the decreasng order of ther frequences. For each applcaton we compare the number of servers n that nterval aganst those of the next nterval. We try to level off the number of servers (to the extent possble) whenever the resultng soluton mproves. We search greedly gvng mportance to the ponts where the number of servers for ths nterval are close to the number for the prevous nterval (to reduce turn-on costs). Thereafter, we tune the frequences so that the resultng soluton remans feasble. Note that T = t s beng assumed n the above two steps. To handle the cases where T > t, we frst perform the above optmzaton (assumng t = T and usng average values of arrval and fle sze parameters over every nterval) at each of u =1to U, to frst determne the m (u). Subsequently, usng the predcton nformaton for each s, we determne the approprate f (u, s) usng ths pre-determned value of m (u). W1 W2 WN Target Response Tme W1 W1 W2 W2 WN WN Trackng Error of Aggregate Frequency F1 F2 FN m1 m2 Server mn Allocaton f1 f2 fn Smulator W1 W2 WN Measured Response Tme Fgure 3: Overvew of the control-theoretc approach 5.2 Theoretc Approach An alternatve strategy that we propose s based on feedback control theory. Compared to the prevous mechansm that reles on steady-state analyss of the system, control theory provdes a way of addressng the system s transent dynamcs usng feedback. It can allow fner granularty of controllng the system, to become more responsve to workload changes. In ths approach, we decompose the gven problem nto two sub-problems: 1. Dynamcally determne an aggregate frequency for each applcaton that can meet response tme guarantees when there s a sngle server per applcaton runnng at ths frequency; the objectve for ths subproblem s to meet response tme wth mnmal aggregate capacty. 2. Solve a server allocaton problem, whch determnes the number of servers for each applcaton n order to provde the aggregate frequency obtaned from the frst subproblem. The objectve for ths subproblem s to balance between the cost of turnng on new servers and the energy consumpton of runnng servers at a hgher frequency. Fgure 3 shows a schematc of ths approach. At each tme perod t, based on the feedback of trackng error that s the dfference between the measured response tme and ts target value, an aggregate frequency wll be computed for each applcaton usng control theory to provde response tme guarantees. The number of servers for each applcaton wll then be allocated through an on-lne optmzaton to provde the requred aggregate frequency. The frequency for each ndvdual server s calculated n terms of the aggregate frequency and the number of runnng servers. Snce we set all servers for each applcaton to be runnng at the same frequency f, the aggregate frequency provded for the -th applcaton s calculated as F (u, s) =m (u) f (u, s) for u =1,,U; s =1,,S. We use F (u, s) nterchangeably wth F (k), k =1,,U S when necessary. It should be noted that though the noton of aggregate frequency s based on approxmatng the response tme from multple servers by the response tme from a sngle server wth the same capacty, the feedback of real response tme wll hopefully make enough adjustments for ths approxmaton. The detals of the feedback control block and the on-lne optmzaton block are gven next Feedback of Aggregate Frequency For the frst subproblem, we apply optmal control theory to dynamcally determne the aggregate server frequency for each applcaton. The orgnal formulaton n Secton 4 mnmzes operaton cost subject to a constrant on response tme. In ths secton, we ncorporate the response tme SLA nto the objectve functon to meet response tme wth as low cost as possble. We modfy the objectve functon as follows, NX U S X (R F F 3 (k)+r W, (W (k) W ) 2 ) (2) k=1 where k s the ndex for tme nterval t; R F and R W, are weghts whose rato provdes tradeoffs between meetng target response

7 tme and havng lower energy cost. The weght parameters R F and R W, are chosen to gve prorty to the SLA. The aggregate frequency F s constraned by the total system capacty,.e. NX F (k) M f max (3) for k =1,,U S. The cost functon (Eq. 2) together wth constrant (Eq. 3) defne a constraned mult-nput (F ) mult-output (W ) optmal control problem. One way to deal wth the constrant (Eq. 3) s to ncorporate t nto the cost functon. In ths paper, snce we assume that there s enough computaton capacty to meet the response tme guarantee for all applcatons (otherwse, the SLA would not have been agreed upon), constrant (Eq. 3) s always satsfed. Consequently, we solve the aggregate server frequency for each applcaton separately, whch leads to solvng N ndependent sngle-nput sngle-output control desgn problem. From the optmal control lterature, the Lnear Quadratc (LQ) regulator [4] seems to be the most approprate way of solvng ths problem. However, n a general LQ formulaton, the cost functon depends on a quadratc term of the control varable. Therefore, we to make the LQ method applcable for our problem. Based on the above arguments and approxmatons, the cost functon for optmal control desgn s defned as follows defne a new control varable F = F 3/2 U S X J = (R F F 2 (k)+r W, (W (k) W ) 2 ) (4) k=1 for =1,,N. Next, we frst use system dentfcaton technques to buld a dynamc model between the new control nput F and the response tme W, after whch a LQ control law s derved. System Identfcaton. We approxmate the dynamc relaton from F to the response tme W by a lnear second-order ARX model as follows: W (k+2) = a,1w (k+1)+a,2w (k)+b,1 F(k+1)+e (k+2) (5) or n a state-space form, W(k +1) W (k +2) 0 1 where H = «= H W(k) W (k +1) a,2 a,1 «, G = ««0 +G F+ e (k +2) «(6) 0, and e b denotes the,1 nose n the system, k =1,,U S 2. A second-order model s chosen here as a balance between model complexty versus accuracy to ft the emprcal data. For system dentfcaton, a pseudo-random bnary sgnal s used to generate the server frequency trajectory F, and part of the nput workload s used to produce the correspondng response tme trajectory W ; then the data set ( F,W ) s used by standard system dentfcaton toolbox n Matlab to compute the model coeffcents n (Eq. 5). It should be noted that even though we use a real trace of HTTP requests for system dentfcaton (whch s the same as that used n generatng predcton models for queung n secton 5.1.1), ths trace s for a tme perod that s dfferent from what s used n the actual smulatons and evaluatons. Snce the coeffcent values n Equaton 5 depend on the samplng granularty of the emprcal data, for each tme granularty (T,t) that we use n our evaluatons, an ndvdual system model (Eq. 5) s constructed for control desgn. Detals on the accuracy of the model can be found n [13]. Law. Gven the cost functon (Eq. 4) together wth the lnear dynamc equaton (Eq. 6), the Lnear-Quadratc regulator provdes an optmal soluton that s always stablzng. The resultng LQ control nput F s represented as the product of an optmal feedback gan wth the trackng error n meetng response tme, F (k +1)= R 1 W(k) F Π W «G W (k 1) W (7) In terms of the LQ control theory, the negatve feedback gan R 1 F ΠG s determned by solvng Π from the followng Rcatt equaton, H T Π H Π (H T Π G )(R F + G T Π G ) 1 (G T Π H ) + R W, =0 (8) The LQ soluton can be easly calculated usng exstng Matlab toolbox. The aggregate frequency F for the -th applcaton s then 2/3 calculated by F = F. In the mplementaton of the LQ regulator, an ntegrator s appended to reduce the steady-state trackng error n meetng response tme SLA. Thus overall controller s F (k + 1) = F (k)+ = F (k) R 2/3 F F 2/3 (k +1) j Π G W(k) W «ff 2/3 W (k 1) W (9) for k = 1,,U S 1. The weghts R F and R W, n (Eq. 4) are desgn parameters. The rule of thumb s to set R F as 1/( F,max) 2 and R W, as 1/(W,max W ) 2, where the subscrpt max denotes the maxmum possble value Server Allocaton Gven the aggregate frequency F (u, s), an on-lne optmzaton s formulated to allocate servers to each applcaton. Ths s based on the tradeoff between the energy cost of runnng servers wth hgher frequences versus the cost of turnng on more servers. Defne m(u) = P N m(u), whch denotes the total number of servers beng turned on across all applcatons, and defne F (u) = P N (maxs=1,,s F(u, s)), whch denotes the total capacty that m(u) should provde at each tme nterval of T.We allocate the number of servers for each applcaton m (u) proportonal to ts aggregate frequency,.e., maxs F(u, s) m (u) = m(u) (10) F (u) Snce server allocaton occurs at u =1,,U, by usng f (u, s) = F (u, s)/m (u) F (u)/m(u) n the cost functon defned n Secton 4, we mnmze the followng cost, UX (K $ m(u) T (P fxed + P f (F (u)/m(u)) 3 )) u=1 + UX B 0 [m(u) m(u 1)] + (11) u=1 Each server s frequency should be operated wthn [f mn,f max]. Consequently, for provdng the aggregate frequency F (u), the total number of servers that are allowed to run across all applcatons at u s constraned by m(u) m(u) m(u), where m(u) = F (u) f max and m(u) =mn( F (u) f mn,m). Equaton (11) s a cost functon wth decson varable m(u). We compute m(u) at the begnnng of each u, but at that tme the value of F calculated based on the feedback control n secton s

8 only up to F (u 1) and the values of F (u) to F (U) are not avalable. Therefore, an on-lne greedy algorthm s desgned to solve m(u) n mnmzng Equaton (11). Further, we use the capacty from the prevous nterval F (u 1,s) and F (u 1) to calculate m (u) n Equaton (10). A pseudo-code of ths on-lne algorthm can be found n [13]. The general dea behnd ths on-lne server allocaton algorthm s explaned as follows. If we gnore the server turn-on costs n Equaton (11), mnmzng the number of servers (denoted by m (u)) for the energy cost can be acheved by settng the frst dervatve of Equaton (11) wth respect to m(u) as zero,.e. δ δm(u) (K $ m(u) T (P fxed + P f (F (u)/m(u)) 3 ) whch gves = K $ T (P fxed 2P f F 3 (u) )=0 (12) m 3 (u) m (u) = F (u) ( 2P f ) 1/3 (13) P fxed Ths m (u) gves the break even number of servers based on the trade-off between the ncrease n cost due to turnng on more servers, and the correspondng decrease n power expended because of allowng them to operate at a lower frequency. If we add one more server beyond m (u), the ncrease n cost due to the addton wll outwegh the cost reducton due to the decrease of server frequency. Therefore, the number of servers beng turned on m(u) should be always less than m (u) even wthout consderng the cost of server turn-ons. If we take nto account the cost of turnng on servers (B o), at any u, u =1,U, there are two cases: 1. If the number of runnng servers n the prevous tme perod m(u 1) s hgher than the mnmum of the break-even number m (u) and the upper-bound m(u), the number of servers m(u) should be brought down to mn(m (u), m(u)) - there s no assocated boot cost for ths. 2. Otherwse, addtonal servers beyond m(u 1) could be turned on, but the total number of runnng servers s stll upper-bounded by mn(m (u), m(u)). For the second case, the number of servers s determned teratvely. Assumng that the number of servers has been ncreased from m(u 1) to, turnng on one more server beyond wll add a one-tme boot cost B 0, whle t wll reduce power cost by K $ (j u +1) T ( P fxed + 2P f F 3 (u) ) (whch s the dervatve of the frst term n Equaton (11) at m(u) 3 =) only f no server wll be turned off durng tme u to tme j. The latter condton s checked by verfyng f s less than the mnmum of m (u) and m(u) up to tme j. Note that at current tme perod u, wedonot have future aggregate frequency F (j) for j u to calculate m(j) - we use an estmated value m(j) to restrct - whch could cause the algorthm to become more greedy. Based on ths argument, the algorthm searches f there exsts such j wthn the whole U tme perods durng whch the savng n power cost s larger than the one-tme boot cost for turnng on one more server. More servers wll be turned on when such j exsts untl the number of runnng servers touches ts upper bound. After the aggregate frequency and number of servers for each applcaton (Equaton (10)) has been determned, the server frequency s calculated as F(u, s) f (u, s) =D( m ) (14) (u) for u =1,,U and s =1,,S, where D( ) denotes roundng up the frequency to dscrete levels n F =(f1,f2,...fl). 5.3 Hybrd Approach The queung approach, descrbed above, can be vewed to be pro-actve, snce t predcts workload behavor for the near future and tunes server allocaton and frequency settng approprately. Whle t can do well when the predctons are farly accurate, and the steady state stochastc behavor s obeyed (requrng a farly coarse-graned wndow), t may not be adaptve enough for fnegraned transent behavor. On the other hand, the control theoretc approach can be vewed to be reactve snce t bases ts decsons on feedback (we are usng feedback control here rather than feed-forward), and s thus expected to be better at fner granulartes, though t can possbly mss out on the predctve nformaton for a longer granularty of optmzaton. Ths leads us to consder a hybrd scheme, where we use the predctve nformaton of the queung approach to determne server allocaton (secton 5.1) at a granularty of T, and the feedback-based control theoretc approach for frequency settng (secton 5.2.1) at the smaller granularty of t. Note that ths strategy s also n agreement wth our underlyng phlosophy where server provsonng costs (n terms of tme for server turn-on and applcaton mgraton) are antcpated to be much hgher than frequency control, and are thus expected to be done less frequently (.e. T>t). 6. EXPERIMENTAL EVALUATION 6.1 Expermental set up We used real HTTP traces obtaned from [1] durng the last week of September and early October 2004, for our evaluaton, whose arrval tme characterstcs are shown n Fgure 4. There are 3 traces n all, denoted as Applcatons 1 to 3, wth each trace beng for a 3 day duraton. We use the data of the frst 2 days (called tranng data) to buld the predcton model for the Queung approach and the system dentfcaton model for the control-theoretc approach, whle only the 3rd day s trace s used n the actual smulaton/evaluaton. We mx these three applcatons n our experments to get dfferent workloads, WL1 to WL3: WL1 contans only one applcaton, namely Applcaton 2; WL2 ncludes both Applcatons 1 and 2; whle WL3 ncludes all 3 applcatons. The technques have been evaluated usng a smulator bult on top of the CSIM smulaton package. The smulator takes the server frequency f and the number of servers m for each tme perod as nputs, and generates response tme W as system output for a gven statc HTTP request trace. Durng the smulaton, t also calculates the energy consumed, number of reboots, and cost of operaton. The cost of reboots and mgratng a server from one applcaton to another are nputs to the smulator. In the nterest of space, we gve a bref descrpton of the smulaton model below, and the reader s referred to [13] for further detals. Snce we are prmarly nterested n the CPU power, whch as explaned n secton 2 domnates over the other components, we assume that the statc HTTP requests ht n the cache. Further, as ponted out n [15], 99% of the HTTP requests can be easly served from the cache. We ran mcrobenchmarks of requests wth dfferent fle szes on a server machne that ht n the cache and used ths as the servce tmes for the requests at the hghest operatng frequency (2.6 GHz). As expected, the relatonshp between fle szes and servce tmes was more or less lnear. We also conducted smlar experments on a laptop wth DVS capabltes to confrm that the servce tme s nversely proportonal to the operatng frequency of the CPU, and approprately adjusted the servce tmes n our sm-

9 ulaton model for the server class CPU. Each node n the smulated cluster uses these servce tmes n servng requests n FCFS order. The maxmum power consumpton at the hghest frequency n our smulator s 100 W, whch roughly matches the consumpton on current server CPUs. We then use the well-known cubc relatonshp between frequency and power, to model the power at each dscrete frequency, whch s smlar to the technque n [15]. We restrct the results presented here to the parameter values shown n Table 2 n the nterest of space. The table gves the dfferent possble CPU frequences, and ther assocated power consumpton values. The electrcty cost (K $ ) s close to that beng charged currently, and the dollar cost assocated wth wear-and-tear per reboot s based approxmately on the cost of a dsk replacement (say $200 ncludng personnel) and the rated MTBF (say startstop cycles). Though our model allows per applcaton W, n the nterest of clarty we use the same target response tme for all applcatons of 6 ms. We have ensured that ths s achevable for each workload by provsonng an approprate number of servers (M)as gven n the table. We consder dfferent combnatons of T and t, wth t<t,.e. DVS beng done much more frequently than server allocaton, wth (T,t) beng used to denote the experment. Parameter Value F (n GHz) (1.4, 1.57, 1.74, 1.91, 2.08, 2.25, 2.42, 2.6) P (n Watts for each f) (60, 63, 66.8, 71.3, 76.8, 83.2, 90.7, 100) K $ (cents/kwh) 10 C r (cents/boot) 20,000/40,000 = 0.5 B o (cents/boot) C r = T reboot, T mgrate 90 secs, 20 secs M 13 (for WL1), 31 (for WL2), 39 (for WL3) W (n msec) 6 T (n mnutes) (60, 20, 10) t (n mnutes) (60, 20, 10, 5) wth t T Table 2: Default Parameter Values for Smulaton 6.2 Results Need for Sophstcated Strateges Before evaluatng our schemes, we frst present some results to motvate ther need, by examnng the followng three smple schemes runnng WL1: 1. Fxed servers, constant frequency durng the entre experment: Results wth such a scheme for the maxmum servers (=13) by runnng the experment n a brute-force fashon wth each frequency level s shown n Table 3 (a). If we want any vable opton wth ths scheme (.e. RT < 6.6), the lowest cost we can get s 196 cents, whch s around 15% hgher than what Queung can provde at (60,60) - see Table 4 - and we can do even better at other granulartes. 2. Hghest frequency, constant number of servers durng the entre experment: Ths corresponds to statcally confgurng the hostng center wth the mnmum number of servers needed to meet the SLA f they were all to run at the maxmum speed. Lookng at the results for ths scheme n Table 3 (b), we see that t takes at least 8 servers operatng at full frequency, before ths becomes a vable opton. Even at ths number of servers, the energy consumpton and cost are hgher than for our schemes at dfferent tme granulartes (compare wth Table 4). 3. Maxmum number of servers, swtch between f mn and f max: Whenever there s no request, a server swtches to the mnmum frequency (no cost s assumed for such swtchng), and whenever a request arrves t swtches to the maxmum frequency untl t has fnshed servng the request. Clearly, the response tme s not any dfferent from operatng at maxmum frequency. However, the energy (72.9% of havng them all operate at hghest frequency) and cost ( cents) are much hgher than what we can get wth our schemes. Arrval rate App 1 App 2 App Tme (hour) (a) Tranng Data Arrval rate Tme (hour) (b) Experment Data Fgure 4: Tme Varyng Arrval Rates App 1 App 2 App 3 Metrcs:. The man statstcs that we present here nclude () the objectve functon (.e. Cost), () the energy consumpton (Energy %) expressed as a percentage of the energy that would be consumed f all the servers were runnng all the tme at the hghest frequency,.e. wthout any power management, () the number of reboots (#reboots), and (v) the mean response tme (RT). When evaluatng any scheme we want to frst check whether t s able to meet the response tme SLA (.e. a target of 6 ms), before checkng ts energy consumpton or cost. However, t should be noted that some schemes may very margnally volate ths SLA whle savng substantal cost. In order to not penalze such a scheme, we allow a 10% slack n the SLA,.e. we consder schemes that can go as hgh as 6.6 ms n ts mean response tme when comparng them, and we term such a scheme to be a vable opton. Note that the frst two schemes requre a brute-force evaluaton of all alternatves, and are thus not sutable n practce. Our pont n showng ther results was to pont out that even f one had a very good statc server capacty allocaton mechansm for the hostng center, results wth a dynamc power management strategy can provde better savngs due to tme-varyng behavor of the workload. One could envson the above thrd scheme to be a nobraner power management strategy f there are no costs to transton between frequences. However, wth reasonable load condtons, there s not that much scope for operatng at lower frequences. The reader should note that we could use any of our three proposals n conjuncton wth ths thrd smple mechansm n order to set the maxmum frequency at whch to serve a request dynamcally (and to otherwse brng t down to f mn). However, we do not consder that n the evaluatons. Freq. Cost RT Energy (GHz) (cents) (ms) (%) (a) Fxed Servers = 13, Const. Freq. Serv. Cost RT Energy (#) (cents) (ms) (%) (b) Max. Freq. = 2.6 GHz, Const. Servers Table 3: Results for the smple strateges

10 WL1 WL2 WL3 (T,t) Schemes Cost RT Energy #reboots Cost RT Energy #reboots Cost RT Energy #reboots (cents) (ms) (%) (cents) (ms) (%) (cents) (ms) (%) (60, 60) Queueng Hybrd IBM (60, 20) Queueng Hybrd (60, 10) Queueng Hybrd (60, 5) Queueng Hybrd (20, 20) Queueng Hybrd IBM (20, 10) Queueng Hybrd (20, 5) Queueng Hybrd (10, 10) Queueng Hybrd IBM (10, 5) Queueng Hybrd Table 4: Comparng the Schemes for the three workloads. Only the costs wth a next to them are vable Comparson wth IBM method The most closely related power management mechansm from IBM [15] uses a rather smple feedback control mechansm to determne the frequency and number of servers for the next nterval gven those of the current nterval and the observed utlzaton/response tme, and these two mechansms are ntegrated (.e. T = t). It s a purely feedback mechansm and does not use any predcton, and t s not ntended to manage server allocaton across applcatons. In the nterest of farness, we should manly compare results for WL1 (the sngle applcaton workload n Table 4), and even there we see that whle ther method gves better energy savngs (around 70%), t s not vable (.e. the response tmes are never meetng the requred SLA). Snce ther method treats response tme on a best effort bass, rather than a constrant to obey, energy optmzaton takes center-stage, leadng to much worse response tmes. As mentoned earler, the SLA s usually much more mportant (the revenue earner) to the hostng center. Our schemes are able to meet the SLA n many cases, whle stll provdng reasonable energy savngs (around 50%) Comparng our schemes Table 4 can be used to compare our three strateges for the three workloads. We manly focus on the vable executons,.e. those where the average response tme s less than 6.6 ms. As descrbed earler, the pro-actve Queung approach uses predctons to antcpate workload behavor and steady-state analyss, both of whch need larger tme granulartes for better accuracy. At these larger granulartes, the theoretc approach does not have any predcton nformaton to optmze for the next nterval, usng feedback from the last nterval that s agan coarse-graned. Consequently, we see that the Queung approach does much better than the reactve scheme at large (T,t) granulartes. For example, at (60,60), Queung s around 10% better n cost than for WL1. To understand the executon characterstcs of these experments at large (T,t), we gve an example executon of Applcaton 3 n WL3 n terms of the number of servers provsoned (m ), ther frequency (f ) and the correspondng response tme (W ) over tme n Fgure 5 (b). If we examne the response tme behavor of (60,60), we see that there s a spke at hour 7. Ths s because there was a load burst n ths hour, whch the mechansm could not detect at the end of hour 6 (because there s only feedback). On the other hand, Queung has advance knowledge of the workload for the next hour, and s able to make better judgments on server allocaton. Ths also reflects on the correspondng frequency that s beng set, resultng n the lower cost for Queung. When we move to the other extreme of small tme granulartes (say (10,5)), we see that the naccuraces of the Queung strategy sgnfcantly mpact ts vablty, and ths opton does not meet the SLA n any of the 3 workloads. However, by gettng very frequent feedback from the system, the reactve scheme s able to perform a better job of server allocaton and frequency modulaton to lower the costs. The result of naccuraces of Queung at small (T,t) s obvous n Fgure 5 (a), whch shows the tme varyng behavor for Applcaton 2 n WL3. The Queung scheme appears to under-estmate the requred capacty, thus turnng down more servers than. Ths results n a hgher response tme. The hybrd scheme that s ntended to beneft from the merts of these two schemes, does gve a cost between these two at ether extremes of tme granulartes. When we look at the large T and short t ranges (e.g. (60,5)), hybrd benefts from the pro-actve mechansm for server allocaton from the queung strategy (t uses the same number of servers as Queung), and the reactve feedback

11 Allocated servers Frequency (GHz) Mean resp tme (msec.) from the underlyng system to set frequences based on short term varatons to meet the response tme SLA # Servers # Servers Q(10,5) H(60, 5) C(10, 5) Tme (hour) Frequency Q(10,5) 1.6 H(60, 5) C(10, 5) 1.4 Tme (hour) Resp. Tme Q(10,5) H(60, 5) C(10, 5) 10 0 Tme (hour) Allocated servers Frequency (GHz) Mean resp tme (msec.) Q(60,60)/H(60, 5) C(60, 60) 4 Tme (hour) Frequency Q(60,60) 1.6 H(60, 5) C(60, 60) 1.4 Tme (hour) Resp. Tme Q(60,60) H(60, 5) C(60, 60) 10 0 Tme (hour) (a) Appln. 2 (b) Appln. 3 Fgure 5: Tme Varyng Behavor for Appln. 2 and Appln. 3 n WL3 6.3 Varyng the Parameters One would be nterested n fndng out how these schemes scale as we move to a larger envronment wth a lot more servers that host more applcatons. We have conducted an experment wth M = 285 servers runnng N =30applcatons. Snce t s dffcult to procure so many dfferent traces, we take the three traces used earler and replcate them by usng randomzed phase dfferences to synthetcally generate workloads. Results for ths workload are gven n Table 5 for representatve (T, t) granulartes, vz., (60,60), (60,5) and (10,5). As before, the Queung approach s more vable at the larger tme granularty, and the naccuraces cause a sgnfcant degradaton n response tmes at the fner granularty. In fact, these naccuraces at fne granulartes result n poor choce of server allocaton n Hybrd at (10, 5). However, at the (60, 5) granularty, Hybrd s agan able to beneft from the pro-actve predcton of Queung for server allocaton, and the feedback of for frequency modulaton, gvng the lowest response tmes. Note that by modulatng parameters of our system model we can target our optmzatons at a wde spectrum of systems, operatng condtons and cost functons. For nstance, settng T to mples conductng server provsonng once (.e. a statc confguraton of the number of servers), and then managng power only wth DVS. Smlarly, settng t to mples that only server turn off/on s em- (T,t) Schemes Cost RT #reboots (cents) (ms) (60, 60) Queueng Hybrd (60, 5) Queueng Hybrd (10, 5) Queueng Hybrd Table 5: Results for Workload wth 30 applcatons and 285 servers ployed for power management. We have also conducted experments varyng the dollar (B o) and tme (T reboot ) of reboots (B o), and the tme for applcaton mgraton (T mgrate). The reader s referred to [13] for these results. Further, we can gnore the current objectve functon, set F = f max, and fnd the mnmum number of servers needed to meet W SLA for each applcaton at any tme. Ths s the tradtonal dynamc server provsonng problem wthout consderng energy consumpton. We have conducted such an experment for WL3, and we fnd that the number of servers that are needed n all s 29 where the m keeps changng over tme (at 60 mnute granularty usng Queung). Ths s lower than = 33 servers that would be needed f each of the applcatons were to be run n solaton,.e. the server farm s parttoned between the applcatons to ensure SLA for each. Ths s because the peak demands across applcatons are not necessarly comng at the same tme. Note that we are able to optmze energy by consderng server provsonng and DVS smultaneously. On the other hand, schemes such as [15], rely on a hgher level provsonng mechansm after whch they can perform ther energy optmzatons. 7. CONCLUDING REMARKS Ths paper has presented the frst formalsm to the problem of reducng server energy consumpton at hostng centers runnng multple applcatons towards the goal of meetng performance based SLAs to clent requests. Though pror studes have shown energy savngs for server clusters by server turn-offs and DVS, these savngs come at a rather expensve cost n terms of volatng the performance-based SLAs (whch are extremely mportant to mantan the revenue stream). Further, prevous proposals have not consdered the cost of server turn-offs, not just n terms of tme overheads, but also n the wear-and-tear of components over an extended perod of tme. Our soluton strateges couple () server provsonng for dfferent applcatons, and () DVS towards enhancng power savngs, whle stll allowng dfferent mechansms for achevng these two actons, unlke any pror work. We have presented three new soluton strateges to ths problem. The frst s a pro-actve soluton (Queung) that predcts workload behavor for the near future, and uses ths nformaton to conduct non-lnear optmzaton based on steady state queung analyss. The second s a reactve soluton (), whch uses perodc feedback of system executon n a control-theoretc framework to acheve the goals. Whle Queung can be a better alternatve than when the workload behavor for the near future s dfferent from the recent past, the steady state assumptons may not hold over shorter granulartes. Feedback s preferable at these shorter tme granulartes, but the nformaton obtaned at the last montorng perod may not be very recent at larger tme granulartes. The Hybrd strategy, on the other hand, can use the predctve nformaton of Queung at coarse

12 tme granulartes for server provsonng, and the feedback of at short granulartes for DVS. Ths Hybrd scheme s also well suted for a more practcal settng where one would lke to perform server provsonng less frequently (due to the hgh tme overhead of brngng up servers or mgratng applcatons), and perform DVS control more frequently. We have demonstrated these deas usng real web server traces, showng how our generc framework can capture a wde spectrum of system confguratons, workload behavor, and the targets for optmzaton. Fnally, we would lke to menton that an mplementaton of the proposed schemes strategy s not tme-consumng. For nstance, n the 285 servers runnng 30 applcatons experment, each server allocaton nvocaton for Hybrd (whch s done once n 60 mnutes) takes around 30 seconds, and the frequency calculatons (done once n 5 mnutes) take less than a second, and these would not even need to run on the actual server nodes. By dong ths control, we are savng on the average around $35 per day n electrcty for the 285 server system. If we consder a more realstc hostng center wth 10 tmes as many servers, ths can work out to a savngs of over $125K per year. We are currently prototypng ths framework on a server cluster. We are also refnng our soluton strateges further, and evaluatng them wth a wder-spectrum of workloads. 8. ACKNOWLEDGEMENTS Ths research has been funded n part by NSF grants , , and an IBM Faculty Award. 9. REFERENCES [1] Web Cachng project. [2] Intel Outlnes Platform Innovatons for More Manageable, Balanced and Secure Enterprse Computng. Intel Press Release, February [3] T. Abdelzaher, K. G. Shn, and N. Bhatt. Performance guarantees for Web server end-systems: A control-theoretcal approach. IEEE Transactons on Parallel and Dstrbuted Systems, 13(1), [4] B. D. O. Anderson and J. B. Moore. Optmal : Lnear Quadratc Methods. Prentce Hall, [5] K. Appleby, S. Fakhour, L. Fong, G. Goldszmdt, M. Kalantar, S. Krshnakumar, D. Pazel, J. Pershng, and B. Rochwerger. Oceano-SLA Based Management of a Computng Utlty. In Proceedngs of the IEEE/IFIP Integrated Network Management, May [6] P. Bohrer, M. Elnozahy, M. Kstler, C. Lefurgy, C. McDowell, and R. Rajamony. The Case for Power Management n Web Servers. In R. Graybll and R. Melhem, edtors, Power Aware Computng. Kluwer Academc Publshers, [7] G. Bolch, S. Grener, H. Meer, and K. S. Trved. Queueng Networks and Markov Chans: Modelng and Performance Evaluaton wth Computer Scence Applcatons. John Wley, New York, [8] E. Carrera, E. Pnhero, and R. Banchn. Conservng Dsk Energy n Network Servers. In Proceedngs of the 17th Internatonal Conference on Supercomputng, [9] A. Chandra, P. Goyal, and P. Shenoy. Quantfyng the Benefts of Resource Multplexng n On-Demand Data Centers. In Proceedngs of Frst ACM Workshop on Algorthms and Archtectures for Self-Managng Systems, June [10] A. Chandrakasan and R. W. Brodersen. Low-Power CMOS Desgn. Wley-IEEE Press, [11] J. Chase, D. Anderson, P. Thakur, and A. Vahdat. Managng Energy and Server Resources n Hostng Centers. In Proceedngs of the 18th Symposum on Operatng Systems Prncples SOSP 01, October [12] J. Chase and R. Doyle. Balance of Power: Energy Management for Server Clusters. In Proceedngs of the 8th Workshop on Hot Topcs n Operatng Systems, May [13] Y. Chen, A. Das, W. Qn, A. Svasubramanam, Q. Wang, and N. Gautam. Managng Server Energy and Operatonal Costs n Hostng Centers. Techncal Report CSE , The Pennsylvana State Unversty, February [14] J. G. Elerath. Specfyng Relablty n the Dsk Drve Industry: No More MTBFs. In Proceedngs of the Annual Relablty and Mantanablty Symposum, pages , [15] M. Elnozahy, M. Kstler, and R. Rajamony. Energy-Effcent Server Clusters. In Proceedngs of the Second Workshop on Power Aware Computng Systems, February [16] M. Elnozahy, M. Kstler, and R. Rajamony. Energy Conservaton Polces for Web Servers. In Proceedngs of the 4th USENIX Symposum on Internet Technologes and Systems, March [17] K. Flautner, S. Renhardt, and T. Mudge. Automatc performance settng for dynamc voltage scalng. In Proceedngs of the 7th annual nternatonal conference on Moble computng and networkng, pages , [18] N. Gandh, S. Parekh, J Hellersten, and D. Tlbury. Feedback of a Lotus Notes Server: Modelng and Desgn. In Proceedngs of the Amercan Conference, [19] D. Grunwald, P. Levs, K. I. Farkas, C. B. Morrey III, and M. Neufeld. Polces for Dynamc Clock Schedulng. In Proceedngs of the Symposum on Operatng Systems Desgn and Implementaton, pages 73 86, [20] S. Gurumurth, A. Svasubramanam, M. Kandemr, and H. Franke. DRPM: Dynamc Speed for Power Management n Server Class Dsks. In Proceedngs of the Internatonal Symposum on Computer Archtecture, pages , [21] D. P. Helmbold, D. E. Long, T. L. Sconyers, and B. Sherrod. Adaptve dsk spndown for moble computers. Mob. Netw. Appl., 5(4): , [22] J. Jones and B. Fonseca. Energy Crss Pnches Hostng Vendors. /01/01/08/010108hnpower.xml. [23] C. Lefurgy, K. Rajaman, F. Rawson, W. Felter, M. Kstler, and T. W. Kelle. Energy Management for Commercal Servers. IEEE Computer, 36(12):39 48, [24] K. L, R. Kumpf, P. Horton, and T. E. Anderson. Quanttatve Analyss of Dsk Drve Power Management n Portable Computers. Techncal report, Unversty of Calforna at Berkeley, [25] J. R. Lorch and A. J. Smth. Improvng Dynamc Voltage Scalng Algorthms wth PACE. In Proceedngs of ACM SIGMETRICS, June [26] C. Lu, T. F. Abdelzaher, J. A. Stankovc, and S. H. Son. A Feedback Approach for Guaranteeng Relatve Delays n Web Servers. In Proceedngs of the Seventh Real-Tme Technology and Applcatons Symposum, page 51, [27] C. D. Patel, C. E. Bash, C. Belady, L. Stahl, and D. Sullvan. Computatonal Flud Dynamcs Modelng of Hgh Compute Densty Data Centers to Assure System Inlet Ar Specfcatons. In Proceedngs of the Pacfc Rm ASME Internatonal Electronc Packagng Techncal Conference and Exhbton (IPACK), [28] T. Perng, T. Burd, and R. Brodersen. The smulaton and evaluaton of dynamc voltage scalng algorthms. In Proceedngs of the 1998 nternatonal symposum on Low power electroncs and desgn, pages 76 81, [29] E. Pnhero, R. Banchn, E. V. Carrera, and T. Heath. Load Balancng and Unbalancng for Power and Performance n Cluster-Based Systems. In Proceedngs of the Workshop on Complers and Operatng Systems for Low Power, September [30] P. Pradhan, R. Tewar, S. Sahu, A. Chandra, and P. Shenoy. An Observaton-based Approach Towards Self-managng Web Servers. In Proceedngs of ACM/IEEE Intl Workshop on Qualty of Servce, May [31] S. Ranjan, J. Rola, H. Fu, and E. Knghtly. QoS-Drven Server Mgraton for Internet Data Centers. In Proceedngs of Internatonal Workshop on QoS, May [32] K. Roy and S. Prasad. Low-Power CMOS VLSI Crcut Desgn. John Wley and Sons, New York, [33] V. Sharma, A. Thomas, T. Abdelzaher, and K. Skadron. Power-aware QoS Management n Web Servers. In Proceedngs of the Real-Tme Systems Symposum, December [34] K. Shen, H. Tang, T. Yang, and L. Chu. Integrated resource management for cluster-based Internet servces. SIGOPS Oper. Syst. Rev., 36(11): , [35] D. S. Stoffer and R. H. Shumway. Tme Seres Analyss and Its Applcatons. Sprnger Verlag, New York, [36] B. Urgaonkar and P. Shenoy. Cataclysm: Handlng Extreme Overloads n Internet Servces. In Proceedngs of ACM Prncples of Dstrbuted Computng, July [37] M. Weser, B. Welch, A. J. Demers, and S. Shenker. Schedulng for Reduced CPU Energy. In Proceedngs of the Symposum on Operatng Systems Desgn and Implementaton, pages 13 23, [38] Q. Zhang, E. Smrn, and G. Cardo. Proft-drven Servce Dfferentaton n Transent Envronments. In Proceedngs of the 11th IEEE/ACM Internatonal Symposum on Modelng, Analyss and Smulaton of Computer Telecommuncatons Systems, 2003.