Dynamic right-sizing for power-proportional data centers Extended version

Transcription

1 1 Dynaic right-sizing for power-proportional data centers Extended version Minghong Lin, Ada Wieran, Lachlan L. H. Andrew and Eno Thereska Abstract Power consuption iposes a significant cost for data centers ipleenting cloud services, yet uch of that power is used to aintain excess service capacity during periods of low load. This paper investigates how uch can be saved by dynaically right-sizing the data center by turning off servers during such periods, and how to achieve that saving via an online algorith. We propose a very general odel and prove that the optial offline algorith for dynaic right-sizing has a siple structure when viewed in reverse tie, and this structure is exploited to develop a new lazy online algorith, which is proven to be 3-copetitive. We validate the algorith using traces fro two real data center workloads and show that significant cost-savings are possible. Additionally, we contrast this new algorith with the ore traditional approach of receding horizon control. I. INTRODUCTION Energy costs represent a significant fraction of a data center s budget [1] and this fraction is expected to grow as the price of energy increases in coing years. Hence, there is a growing push to iprove the energy efficiency of the data centers behind cloud coputing. A guiding focus for research into green data centers is the goal of designing data centers that are power-proportional, i.e., use power only in proportion to the load. However, current data centers are far fro this goal even today s energy-efficient data centers consue alost half of their peak power when nearly idle [2]. A proising approach for aking data centers ore powerproportional is using software to dynaically adapt the nuber of active servers to atch the current workload, i.e., to dynaically right-size the data center. Specifically, dynaic right-sizing refers to adapting the way requests are dispatched to servers in the data center so that, during periods of low load, servers that are not needed do not have jobs routed to the and thus are allowed to enter a power-saving ode (e.g., go to sleep or shut down). Technologies that ipleent dynaic right-sizing are still far fro standard in data centers due to a nuber of challenges. First, servers ust be able to sealessly transition into and out of power saving odes while not losing their state. There has been a growing aount of research into enabling this in recent years, dealing with virtual achine state [3], network state [4] and storage state [5], [6]. Second, such techniques ust prove to be reliable, since adinistrators we talk to worry about wear-and-tear consequences of such technologies. Third, and the challenge that this paper addresses, it is unclear how to deterine the nuber of servers to M. Lin and A. Wieran are with California Institute of Technology. L. L. H. Andrew is with Swinburne University of Technology. E. Thereska is with Microsoft Research, Cabridge. toggle into power-saving ode and how to control servers and requests due to the lack of knowledge about future workloads, which eans that a server that is put to sleep ay soon need to be woken again. Although Receding Horizon Control has been proposed for the online control [7], [8], as shown in Section III, its perforance really depends on the prediction window and the toggling cost of servers, which ay vary widely in different data centers. A goal of this paper is to provide a new algorith to address this challenge for general settings. To this end, we develop a general odel that captures the ajor issues that affect the design of a right-sizing algorith, including: the cost (lost revenue) associated with the increased delay fro using fewer servers, the energy cost of aintaining an active server with a particular load, and the cost incurred fro toggling a server into and out of a power-saving ode (including the delay, energy, and wear-and-tear costs). This paper akes three contributions: First, we analytically characterize the optial offline solution (Section IV). We prove that it exhibits a siple, lazy structure when viewed in reverse tie. Second, we introduce and analyze a novel, practical online algorith otivated by this structure (Section V). The algorith, naed Lazy Capacity Provisioning (LCP(w)), uses a prediction window of length w of future arrivals and iics the lazy structure of the optial algorith, but proceeding forward instead of backwards in tie. We prove that LCP(w) is 3-copetitive, i.e., its cost is at ost 3 ties that of the optial offline solution. This is regardless of the workload and for very general energy and delay cost odels, even when no inforation is used about arrivals beyond the current tie period (w = ). Further, in practice, LCP(w) is far better than 3-copetitive, incurring nearly the optial cost. Third, we validate our algorith using two load traces (fro Hotail and a Microsoft Research data center) to evaluate the cost savings achieved fro dynaic right-sizing in practice (Section VI). We contrast our new algorith with Receding Horizon Control and confir that our algorith provides uch ore stable cost saving. We show that significant savings are possible under a wide range of settings and that savings becoe draatic when the workload is predictable over an interval proportional to the toggling cost. The agnitude of the potential savings depends priarily on the peak-to-ean ratio (PMR) of the workload, with a PMR of 3 being enough to give 3% cost saving. In the context of these real traces, we also discuss when it does and when it does not ake sense to use dynaic right-sizing versus the alternative of valleyfilling, i.e., using periods of low load to run background/ aintenance tasks. We find that dynaic right-sizing provides

2 ore than 15% cost savings even when the background work akes up 4% of the workload when the PMR is larger than 3. II. MODEL FORMULATION Right-sizing probles vary between data centers: soe systes have flexible quality of service (QoS) requireent, others have hard service level agreeents (SLAs); the electricity price ay be tie-varying or fixed; workloads can be hoogeneous or heterogeneous, etc. We first introduce a general odel which captures the ain issues in any rightsizing probles in different systes. We then give exaples and show how they fit into this general odel. A. General odel We now describe the general odel used to explore the cost savings possible fro dynaic right-sizing. An instance consists of a constant β >, a horizon 1 T >, a sequence of non-negative convex functions g t ( ) for t = 1,..., T 1; g t ( ) can take on the value but cannot be identically. For notational siplicity, let g T (x) and x = x T =. The odel is: iniize x 1,...,x T 1 g t ( ) + β ( 1 ) + (1) subject to where (x) + = ax(, x), and { } are non-negative scalar variables. The solution to optiization (1) ay not be unique. By Corollary 1 in Appendix A, there exists a solution that is the eleentwise axiu. Unless otherwise stated, the solution refers to this axiu solution. We discuss the extension to vector in Section VIII. To apply this to data center right-sizing, recall the ajor costs of right-sizing: (a) the cost associated with the increased delay fro using fewer servers and the energy cost of aintaining an active server with a particular load; (b) the cost incurred fro toggling a server into and out of a power-saving ode (including the delay, igration, and wearand-tear costs). We call the first part operating cost and the second part switching cost and consider a discretetie odel where the tieslot length atches the tiescale at which the data center can adjust its capacity. There is a (possibly long) tie-interval of interest t {, 1,..., T } and the capacity (i.e., nuber of active servers) at tie t is. The operating cost in each tieslot is odelled by g t ( ), which presents the cost of using servers ( has a vector value if servers are heterogeneous) to serve requests at tieslot t. Moreover, g t ( ) captures any other factors in tieslot t, including the arrival rate, the electricity price, the cap on available servers, the SLA constraints. For the switching cost, let β be the cost to transition a server fro the sleep state to the active state and back again. We dee this cost to be incurred only when the server wakes up. Thus the switching cost for changing the nuber of active 1 If paraeters such as β are known in advance, then the results in this paper can be extended to a odel in which each instance has a finite duration, but the cost is sued over an infinite horizon. servers fro 1 to is β( 1 ) +. The constant β includes the costs of (i) the energy used, (ii) the delay in igrating connections/data/etc. (e.g., by VM techniques), (iii) increased wear-and-tear on the servers, and (iv) the risk associated with server toggling. If only (i) and (ii) atter, then β is either on the order of the cost to run a server for a few seconds (waking fro suspend-to-ram or igrating network state [4] or storage state [5]), or several inutes (to igrate a large VM [3]). However, if (iii) is included, then β becoes on the order of the cost to run a server for an hour [9]. Finally, if (iv) is considered then our conversations with operators suggest that their perceived risk that servers will not turn on properly when toggled is high, so β ay be any hours server costs. Throughout, denote the operating cost of a vector X = (x 1,..., x T ) by cost o (X) = T g t( ), the switching cost by cost s (X) = β T ( 1 ) +, and cost(x) = cost o (X) + cost s (X). We have seen that Forulation (1) captures the operating cost and switching cost of right-sizing probles. However, Forulation (1) ay look not that general at first glance due to the lack of constraints. Actually constraints in rightsizing probles can be handled iplicitly in functions g t ( ) by extended-value extension, i.e., defining g t ( ) to be outside its doain. This extension akes our description/ notation ore clear without introducing particular constraints in different data centers. Forulation (1) akes a siplification that it does not enforce that be integer valued. This is acceptable since the nuber of servers in a typical data center is large; oreover, preliinary investigation suggests that this can be enforced by an appropriate choice of cost functions, g t ( ). This odel also ignores issues surrounding reliability and availability. In practice, a solution that toggles servers ust still aintain the reliability and availability guarantees. Such issues are beyond the scope of this paper, however previous work shows that solutions are possible [5]. This optiization proble would be easy to solve offline, i.e., given functions g t ( ) for all t. However, our goal is to find online algoriths for this optiization, i.e., algoriths that deterine x using only inforation up to tie + w, where the prediction window w is part of the proble instance. Here, we assue that the predictions [g,..., g +w ] are known perfectly at tie, but we show in Section VI that our algorith is robust to this assuption in practice. We evaluate the perforance of an online algorith A using the standard notion of copetitive ratio. The copetitive ratio of A is defined as the supreu, taken over all possible inputs, of cost(a)/cost(op T ), where cost(a) is the objective function of (1) under A and OP T is the optial offline algorith. The analytic results of Sections IV and V assue that the service has a finite duration, i.e. T <, but hold for arbitrary sequences of convex functions g t ( ). Thus, the analytic results provide worst-case guarantees. However, to provide realistic cost estiates, we consider case-studies in Section VI where g t ( ) is based on real-world traces. 2

3 B. Special cases Now let us consider two choices of g t ( ) to fit two different right-sizing probles into Forulation (1). The discrete-tie odel consists of finitely any ties t {, 1,..., T }, where T ay be a year, easured in tieslots of 1 inutes. Assue the workload affects the operating cost at tieslot t only by its ean arrival rate, denoted λ t. (This ay have a vector value if there are ultiple types of work). This assuption is reasonable for any services such as web search, social network or eail service since the request interarrival ties and the response ties are uch shorter than the tieslots so that the provisioning can be based on the arrival rate. We odel a data center as a collection of hoogeneous servers (except Section VIII where heterogeneous systes are discussed) and focus on deterining, the nuber of active servers during each tie slot t. For notational siplicity, we assue a load balancer assigns arriving jobs to active servers uniforly, at rate λ t / per server (which is widely deployed and turns out to be optial for any systes). Assue the power of an active server handling arrival rate λ is e(λ), and the perforance etric we care about is d(λ) (e.g., average response tie or response tie violation probability). For exaple, a coon odel of the power for typical servers is an affine function e(λ) = e + e 1 λ where e and e 1 are constants; e.g., see [1]. The average response tie can be odeled using standard queuing theory results. If the server happens to be odeled by an M/GI/1 Processor Sharing queue then the average response tie is d(λ) = 1/(µ λ), where the service rate of the server is µ [11]. Other exaples include, for instance, using the 99th percentile of delay instead of the ean. In fact, if the server happens to be odeled by an M/M/1 Processor Sharing queue then the 99th percentile is log(1)/(µ λ), and so the for of d(λ) does not change [11]. Note that, in practice, e(λ) and d(λ) can be epirically easured by observing the syste over tie without assuing atheatical odels for the. Our analytical results work for any e( ) and d( ) as long as they result in convex g t ( ). But for siplicity, we will use e(λ) = e +e 1 λ and d(λ) = 1/(µ λ) for our nuerical experients. 1) Exaple 1: Services with flexible QoS requireent: Many Internet services, such as web search, eail service and social network are flexible in response tie in certain range. For such applications, it is hard to find a threshold to distinguish good services and bad services. Instead, the revenue is lost gradually as the response tie increases and thus our goal is to balance the perforance and the power cost [8], [12] [14]. One natural odel for the lost revenue is d 1 λ(d(λ) d ) + where d is the iniu delay users can detect and d 1 is a constant. This easures the perceived delay weighted by the fraction of users experiencing that delay. For the energy cost, assue the electricity price is p t at tieslot t, then the energy cost is p t e(λ). There ay be a tie-varying cap M t on the nuber of active server available due to other activities/services in the data center (e.g., backup activities). The cobination together with the dispatching rule at the load balancer gives iniize + β ( ( d 1 λ t d ( λt ) d ) + + p t e ( λt ) ) ( 1 ) + (2) subject to λ t M t Copared to Forulation (1), we have g t ( ) = ( ( ) ) + ( ) d x 1λ t λ t (d ) t d + λ pt e t if λ t M t, otherwise. We can see that g t ( ) is in the for of the perspective function of f(z) = d 1 z(d(z) d ) + + p t e(z), which is convex under coon odels of d( ) and e( ), such as M/G/1 queueing odels with or without speed scaling [15] of individual servers. Thus g t ( ) is convex under coon odels. 2) Exaple 2: Services with hard QoS constraints: Soe data centers have to support services with hard constraints such as SLA requireent or delay constraints for ultiedia applications. For such systes, the goal is to iniize energy cost while satisfy the constraints, as shown in [16], [17]. The optiization becoes ( ) λt iniize p t e + β ( 1 ) + (3) subject to λ t and d ( λt ) D t where d( ) can be average delay, delay violation probability or other perforance etrics. This constraint usually defines a convex doain for. Actually if we assue that d( ) is an increasing function of the load, then it is equivalent to iposing an upper bound on the load per server (i.e., λ t / is less than certain threshold) and thus results in a convex optiization proble. The corresponding g t ( ) can be defined as follows: g t ( ) = { ( ) λ xt p t e t if λ t and d otherwise. III. RECEDING HORIZON CONTROL ( λ t ) D t, As otivation for deriving a new algorith, let us first consider the standard approach. An online policy that is coonly proposed to control data centers [7], [8] (and other dynaic systes) is Receding Horizon Control (RHC) algoriths, also known as Model Predictive Control. RHC has a long history in the control theory literature [18] [2] where the focus was on stability analysis. In this section, let us introduce briefly the copetitive analysis result of RHC for our data center proble (which is proven in [21]) and see why we ay need a better online algorith. Inforally, RHC(w) works by solving, at tie, the cost optiization over the window (, +w) given the starting state 3

4 x 1, and then using the first step of the solution, discarding the rest. Forally, define X (x 1 ; g... g +w ) as the vector in R w+1 indexed by t {,..., + w}, which is the solution to iniize +w t= subject to Then, RHC(w) works as follows. +w g t ( ) + β t= ( 1 ) + (4) Algorith 1. Receding Horizon Control, RHC(w). For all t, set the nuber of active servers to =. At each tieslot 1, set the nuber of active servers to x = X (x 1 ; g... g +w ) Note that (4) need not have a unique solution. We define RHC(w) to select the solution with the greatest first entry. Define e the iniu cost per tieslot for an active server, then we have the following theore [21]: β Theore 1. RHC(w) is (1 + (w+1)e )-copetitive for optiization (1) but not better than ( 1 w+2 + β (w+2)e )-copetitive. This highlights that, with enough lookahead, RHC(w) is guaranteed to perfor quite well. However, its copetitive ratio depends on the paraeters β, e and w. These paraeters vary widely in different data centers since they host different services or have different SLAs. Thus, RHC(w) ay have a poor copetitive ratio for a data center with big switching cost or sall prediction window. This poor perforance is to be expected since RHC(w) akes no use of the structure of the optiization we are trying to solve. We ay wonder if there exists better online algorith for this proble that perfors well for a wider range of paraeters, even when the prediction window w = (no workload prediction except current tieslot). IV. THE OPTIMAL OFFLINE SOLUTION In order to exploit the structure of the specific proble (1), the first natural task is to characterize the optial offline solution, i.e., the optial solution given g t ( ) for all t. The insight provided by the characterization of the offline optiu otivates the forulation of our online algorith. It turns out that there is a siple characterization of the optial offline solution X to the optiization proble (1), in ters of two bounds on the optial solution which correspond to charging cost β either when a server coes out of powersaving ode (as (1) states) or when it goes in. The optial x can be viewed as lazily staying within these bounds going backwards in tie. More forally, let us first describe lower and upper bounds on x, denoted x L and x U, respectively. Let (x L,1,..., x L, ) be the solution vector to the optiization proble iniize g t ( ) + β ( 1 ) + (5) subject to where x =. Then, define x L = x L,. Siilarly, let (x U,1,..., x U, ) be the solution vector to the optiization iniize g t ( ) + β subject to ( 1 ) + (6) Then, define x U = x U,. Notice that in each case, the optiization proble includes only ties 1 t, and so ignores the future inforation for t >. In the case of the lower bound, β cost is incurred for each server toggled on, while in the upper bound, β cost is incurred for each server toggled into power-saving ode. Lea 1. For all, x L x x U. Given Lea 1, we now characterize the optial solution x. Define (x) b a = ax(in(x, b), a) as the projection of x into [a, b]. Then, we have: Theore 2. The optial solution X = (x 1,..., x T ) of the data center optiization proble (1) satisfies the backward recurrence relation x =, > T ; ( x )x U +1 x L, T. Theore 2 and Lea 1 are proven in Appendix A. An exaple of the optial x t can be seen in Figure 1(a). Many ore nueric exaples of the perforance of the optial offline algorith are provided in Section VI. Theore 2 and Figure 1(a) highlight that the optial algorith can be interpreted as oving backwards in tie, starting with x T = and keeping x = x +1 unless the bounds prohibit this, in which case it akes the sallest possible change. This interpretation hightlights that it is ipossible for an online algorith to copute x since, without knowledge of the future, an online algorith cannot know whether to keep x constant or to follow the upper/lower bound. V. LAZY CAPACITY PROVISIONING A ajor contribution of this paper is the presentation and analysis of a novel online algorith, Lazy Capacity Provisioning (LCP(w)). At tie, LCP(w) knows only g t ( ) for t + w, for soe prediction window w. As entioned before, we assue that these are known perfectly, but we show in Section VI that the algorith is robust to this assuption in practice. The design of LCP(w) is otivated by the structure of the optial offline solution described in Section IV. Like the optial solution, it lazily stays within upper and lower bounds. However, it does this oving forward in tie instead of backwards in tie. Before defining LCP(w) forally, recall that the bounds x U and x L do not use knowledge about the loads in the prediction window of LCP(w). To use it, define refined bounds x U,w and x L,w such that x U,w = x U +w, in the solution of (6) and x L,w = x L +w, in that of (5). Note that w = is allowed (no future prediction) and x U, = x U and x L, = x L. The following generalization of Lea 1 is proven in Appendix B. (7) 4

5 nuber of servers nuber of servers bounds tie t (hours) (a) Offline optial LCP() bounds tie t (hours) (b) LCP() load load tie (hours) (a) Hotail tie (hours) (b) MSR Fig. 1. Illustrations of (a) the offline optial solution and (b) LCP() for the first day of the MSR workload described in Section VI with a sapling period of 1 inutes. The operating cost is defined in (2) with d = 1.5, d 1 = 1, µ = 1, p t = 1, e = 1 and e 1 = and the switching cost has β = 6 (corresponding to the energy consuption for one hour). Lea 2. x L x L,w x x U,w x U for all w. Now, we are ready to define LCP(w) using x U,w x = and x L,w. Algorith 2. Lazy Capacity Provisioning, LCP(w). Let X = (x,..., xt ) denote the vector of active servers under LCP(w). This vector can be calculated using the following forward recurrence relation, ; ( x 1 )x U,w x L,w, 1. Figure 1(b) illustrates the behavior of LCP(). Note its siilarity with Figure 1(a), but with the laziness in forward tie instead of reverse tie. The coputational deands of LCP(w) ay initially see prohibitive as grows, since calculating x U,w and x L,w requires solving convex optiizations of size + w. However, it is possible to calculate x U,w and x L,w without using the full history. Lea 1 in Appendix B iplies that it is enough to use only the history since the ost recent point when the solutions of (5) and (6) are either both increasing or both decreasing, if such a point exists. In practice, this period is typically less than a day due to diurnal traffic patterns, and so the convex optiization, and hence LCP(w), reains tractable even as grows. In Figure 1(b), each point is solved in under half a second. Next, consider the cost incurred by LCP(w). Section VI discusses the cost in realistic settings, while in this section we focus on worst-case bounds, i.e., characterize the copetitive ratio. The following theore is proven in Appendix B. Theore 3. cost(x ) cost(x ) + 2cost s (X ). Thus, LCP(w) is 3-copetitive for optiization (1). Further, (8) Fig. 2. Illustration of the traces used for nuerical experients. for any finite w and ɛ > there exists an instance, with g t of the for f(λ t / ), such that LCP(w) attains a cost greater than 3 ɛ ties the optial cost. Note that Theore 3 says that the copetitive ratio is independent of the prediction window size w, the switching cost β, and is uniforly bounded above regardless of the for of the operating cost functions g t ( ). Surprisingly, this eans that even the yopic LCP() is 3-copetitive, regardless of {g t ( )}, despite having no inforation beyond the current tieslot. It is also surprising that the copetitive ratio is tight regardless of w. Seeingly, for large w, LCP(w) should provide reduced costs. Indeed, for any particular workload, as w grows the cost decreases and eventually atches the optial, when w > T. However, for any fixed w, there is a worstcase sequence of cost functions such that the copetitive ratio is arbitrarily close to 3. Moreover, there is such a sequence of cost functions having the natural for f(λ t / ), which corresponds to a tie varying load λ being shared equally aong servers, which each has a tie-invariant (though pathological) operating cost f(λ) when serving load λ. Finally, though 3-copetitive ay see like a large gap, the fact that cost(x ) cost(x )+2cost s (X ) highlights that the gap will tend to be uch saller in practice, where the switching costs ake up a sall fraction of the total costs since dynaic right-sizing would tend to toggle servers once a day due to the diurnal traffic. VI. CASE STUDIES In this section the goal is two-fold: The first is to evaluate the cost incurred by LCP(w) relative to the optial solution and RHC(w) for realistic workloads. The second is ore generally to illustrate the cost savings and energy savings that coe fro dynaic right-sizing in data centers. To accoplish these goals, we experient using two real-world traces. 5

6 A. Experiental setup Throughout the experiental setup, the ai is to choose paraeters that provide conservative estiates of the cost savings fro LCP(w) and right-sizing in general. Cost benchark: Current data centers typically do not use dynaic right-sizing and so to provide a benchark against which LCP(w) is judged, we consider the cost incurred by a static right-sizing schee for capacity provisioning. This chooses a constant nuber of servers that iniizes the costs incurred based on full knowledge of the entire workload. This policy is clearly not possible in practice, but it provides a conservative estiate of the savings fro right-sizing since it uses perfect knowledge of all peaks and eliinates the need for overprovisioning in order to handle the possibility of flash crowds or other traffic bursts. Cost function paraeters: The cost is of the for (2), characterized by the paraeters d, d 1, µ, p t, e and e 1, and the switching cost β. We noralize µ = 1, p t = 1 and choose units such that the fixed power is e = 1. The load-dependent power is set to e 1 =, because the energy consuption of current servers is doinated by the fixed costs [2]. The delay cost d 1 reflects revenue lost due to custoers being deterred by delay, or to violation of SLAs. We set d 1 /e = 1 for ost experients but consider a wide range of settings in Figure 8. The iniu perceptible delay is set to d = 1.5 ties the tie to serve a single job. The value 1.5 is realistic or even conservative, since valley filling experients siilar to those of Section VI-B show that a saller value would result in a significant added cost when using valley filling, which operators now do with inial increental cost. The noralized switching cost β/e easures the duration a server ust be powered down to outweigh the switching cost. We use β = 6e, which corresponds to the energy consuption for one hour (six saples). This was chosen as an estiate of the tie a server should sleep so that the wearand-tear of power cycling atches that of operating [9]. Workload inforation: The workloads for these experients are drawn fro I/O traces of two real-world data centers. The first set of traces is fro Hotail, a large eail service running on tens of thousands of servers. We used traces fro 8 such servers over a 48-hour period, starting at idnight (PDT) on Monday August 4 28 [5]. The second set of traces is taken fro 6 RAID volues at MSR Cabridge. The traced period was 1 week starting fro 5PM GMT on the 22nd February 27 [5]. These activity traces represent respectively a service used by illions of users and a sall service used by hundreds of users. The traces are noralized such that the peak load is 1, which are shown in Figure 2. Notice that this noralization does not affect the experient results since only the shape of trace atters for cost saving. Both sets of traces show strong diurnal properties and have peak-to-ean ratios (PMRs) of 1.64 and 4.64 for Hotail and MSR respectively. Loads were averaged over disjoint 1 inute intervals. The Hotail trace contains significant nightly activity due to aintenance processes (backup, index creation etc) which is not shown fully in Figure 2(a). The data center, however, is provisioned for the peak foreground activity. This creates 1 5 LCP(w) RHC(w) prediction window, w Fig. 3. (a) Hotail LCP(w) RHC(w) prediction window, w (b) MSR Ipact of prediction window size on cost incurred by LCP(w). a dilea: should our experients include the aintenance activity or to reove it? Figure 5 illustrates the ipact of this decision. If the spike is retained, it akes up nearly 12% of the total load and forces the static provisioning to use a uch larger nuber of servers than if it were reoved, aking savings fro dynaic right-sizing uch ore draatic. To provide conservative estiates of the savings fro rightsizing, we chose to tri the size of the spike to iniize the savings fro right-sizing. This triing akes the nightly background spike have value roughly equal to 1 (the peak foreground activity). B. When is right-sizing beneficial? Our experients are organized in order to illustrate the ipact of a wide variety of paraeters on the cost-savings provided by dynaic right-sizing with LCP(w). The goal is to better understand when dynaic right-sizing can provide large enough cost-savings to warrant the extra ipleentation coplexity. Reeber that throughout, we have attepted to choose experiental settings so that the benefit of dynaic right-sizing is conservatively estiated. The analytic results in Theore 1 and Theore 3 show that the perforance of RHC(w) is ore sensitive to the prediction window w and the switching cost β/e while LCP(w) works well for general settings. The first two experients consider the realistic cost saving fro RHC(w) and LCP(w) with varying w and β/e under real-world traces. Ipact of prediction window size: The first paraeter we study is the ipact of the predictability of the workload. In particular, depending on the workload, the prediction window w for which accurate estiates can be ade could be on the order of tens of inutes or on the order of hours. Figure 3 illustrates its ipact on the cost savings of RHC(w) and LCP(w), where the unit of w is one tieslot of is 1 inutes. The first observation fro Figure 3 is that the savings possible ( ) in the MSR trace are larger than in the Hotail trace. Second, the cost saving of LCP(w) is siilar to RHC(w) when w 3. However, RHC(w) can be uch worse when w 2, i.e., LCP(w) has ore stable cost saving than RHC(w), which supports the analytic results very well. In both traces, a big fraction of the optial cost savings is achieved by LCP(), which uses only workload predictions about the current tieslot (1 inutes). The fact that this yopic algorith provides significant gain over static provisioning is encouraging. Further, a prediction window that 6

7 2 2 4 LCP(3) LCP() RHC(3) RHC() in hour day β / e 5 5 LCP(3) LCP() RHC(3) RHC() in hour day β / e w= % of load due to spike w= Mean prediction error (% ean load) Fig. 4. (a) Hotail (b) MSR Ipact of switching cost, against tie on a logarithic scale. Fig. 5. Ipact of overnight peak in Fig. 6. Ipact of prediction error on the Hotail workload. LCP(w) under Hotail workload. is approxiately the size of β = 6 (i.e. one hour) gives nearly the optial cost savings. Ipact of switching costs: One of the ain worries when considering right-sizing is the switching cost of toggling servers, β, which includes the delay costs, energy costs, costs of wear-and-tear, and other risks involved. Thus, an iportant question to address is: How large ust switching costs be before the cost savings fro right-sizing disappears? Figure 4 shows that significant gains are possible provided β is saller than the duration of the valleys. Given that the energy costs, delay costs, and wear-and-tear costs are likely to be on the order of an hour, this iplies that unless the risks associated with toggling a server are perceived to be extree, the benefits fro dynaic right-sizing are large in the MSR trace (high PMR case). Though the gains are saller in the Hotail case for large β, this is because the spike of background work splits an 8 hour valley into two short 4 hour valleys. If these tasks were shifted or balanced across the valley, the Hotail trace would show significant cost reduction for uch larger β, siilarly to the MSR trace. Another key observation is that the cost saving of LCP(w) is siilar to that of RHC(w) when β is sall or oderate. However, when β becoes big, LCP(w) is uch better than RHC(w), which confirs that LCP(w) provides ore stable cost saving than RHC(w). Since we use oderate prediction window and oderate switching cost as the default setting in our experients, we will show only the LCP(w) results but not RHC(w) results for the reaining experients. Prediction error: The LCP(w) algorith depends on having estiates for the arrival rate during the current tieslot as well as for w tieslots into the future. Our analysis in Section V assues that these estiates are perfect, but of course in practice there are prediction errors. Figure 6 shows the cost reduction for the Hotail trace when additive white Gaussian noise of increasing variance is added to the predictions used by LCP(w), including the workload in current tieslot. A variance of corresponds to perfect knowledge of the near-future workload, and this figure shows that the perforance does not degrade significantly when there is oderate uncertainty, which suggests that the assuptions of the analysis are not probleatic. The plot for the MSR trace is qualitatively siilar, although the actual cost savings are significantly larger. Even very inexact knowledge of future workloads can be beneficial. Note that LCP() s upper and lower bounds x U and w= peak/ean ratio (a) Total cost % energy cost reduction w= peak/ean ratio (b) Energy cost Fig. 7. Ipact of the peak-to-ean ratio of the workload on the total cost and energy cost incurred by LCP(w) in the Hotail workload. x L iplicitly assue that future workloads will be infinite and, respectively. Perforance can be iproved if either of these bounds can be tightened. For exaple, it ay be possible to use daily trends to predict a lower bound on the load, while the upper bound is still taken to be infinite to allow for flash crowds. Given that prediction errors for real data sets tend to be sall [8], [22], based on these plots, to siplify our experients we allow LCP(w) perfect predictions. Ipact of peak-to-ean ratio (PMR): Dynaic right-sizing inherently exploits the gap between the peaks and valleys of the workload, and intuitively provides larger savings as that gap grows. To illustrate this, we artificially scaled the PMR of each trace and calculated the resulting savings. The results, in Figure 7, illustrate that the intuition holds for both cost savings and energy savings. The gain grows quickly fro zero at PMR=1, to 5 1% at PMR 2 which is coon in large data centers, to very large values for the higher PMRs coon in sall to ediu sized data centers. This shows that, even for sall data centers where the overhead of ipleenting right-sizing is aortized over fewer servers, there is a significant benefit in doing so. To provide soe context for the onetary value of these savings, consider that a typical 5, server data center has an electricity bill of around $1 illion/onth [1]. The workload for the figure is generated fro the Hotail workload by scaling λ t as ˆλ t = k(λ t ) α, varying α and adjusting k to keep the ean constant. Note that though Figure 7 includes only the results for Hotail, the resulting plot for the MSR trace is nearly identical. This highlights that the difference in cost savings observed between the two traces is priarily due to the fact that the PMR of the MSR trace is so uch larger than that of the Hotail trace. 7

8 w= energy cost / delay cost w= energy cost / delay cost nuber of servers Static 2 constrained tie t (hours) (a) Hotail (b) MSR (a) Offline optial Fig. 8. Ipact of increasing energy costs. 1 Ipact of energy costs: Clearly the benefit of dynaic right-sizing is highly dependent on the cost of energy. As the econoy is forced to ove towards ore expensive renewable energy sources, this cost will inevitably increase and Figure 8 shows how this increasing cost will affect the cost savings possible fro dynaic right-sizing. Note that the cost savings fro dynaic right-sizing grow quickly as energy costs rise. However, even when energy costs are quite sall relative to delay costs, we see iproveent in the case of the MSR workload due to its large PMR. Ipact of valley filling: A coon alternative to dynaic right-sizing that is often suggested is to run very delayinsensitive aintenance/background processes during the periods of low load, known as valley filling. Soe applications have a huge aount of such background work, e.g., search engines tuning their ranking algoriths. If there is enough such background work, then the valleys can in principle be entirely filled and so the PMR 1 and thus dynaic rightsizing is unnecessary. Thus, an iportant question is: How uch background work is enough to eliinate the cost savings fro dynaic right-sizing? Figure 1 shows that, in fact, dynaic right-sizing provides cost savings even when background work akes up a significant fraction of the total load. For the Hotail trace, significant savings are still possible when background load akes upwards of 1% of the total load, while for the MSR trace this threshold becoes nearly 6%. Note that Figure 1 results fro considering ideal valley filling, which results in a perfectly flat load during the valleys, but does not give background processes lower queueing priority. Ipact of capacity liits: Energy-related expenses account for alost half of the cost of a data center. However, any of those, such as power supply and cooling infrastructure, depend on the peak power rather than the total energy. This raises the question of whether LCP increases the peak nuber of servers used, or equivalently the peak power consuption. The dotted line of Figure 9 shows the optial static nuber of servers to use, which we call M, while the dashed line in Figure 9(a) shows the optial nuber and that in Figure 9(b) shows the nuber used by LCP. Dynaic provisioning uses a higher peak nuber of servers, which would increase the cost. However, the LCP algorith (and off-line optiization) can ipose an additional constraint M t = M to ensure that the infrastructure cost is not increased. Indeed, when M t is independent of t the constrained optiu appears to have the nuber of servers Static 2 LCP constrained tie t (hours) (b) LCP() Fig. 9. Illustrations of (a) the offline optial solution and (b) LCP() for the Hotail workload described in Section VI with a sapling period of 1 inutes. The operating cost is as in Figure 1. The dotted line shows the optial static nuber of servers to use, the dashed line shows the optial or LCP provisioning without constraints, and the solid line shows the constrained optiu or constrained LCP solution. Notice that the constrained optiu turns out to correspond to siple clipping of the unconstrained optiu. very siple for in(m, x t ) the unconstrained optiu clipped to M though this structure does not hold for general tie-varying bounds. An exaple of is shown by the solid lines in Figure 9. Despite the additional constraint, ost of the potential gains are still realized by dynaic right sizing. In this case, the gain of the optial solution drops fro 8% to 6.5%, and that of LCP drops fro 5.1% to 4.4%. VII. RELATED WORK Interest in right-sizing has been growing since [14] and [23] appeared early last decade. Approaches range fro very analytic work focusing on developing algoriths with provable guarantees to systes work focusing on ipleentation. Early systes work such as [23] achieved substantial savings despite ignoring switching costs in their design. Other designs have focused on decentralized fraeworks, e.g., [24], [25] and [26], as opposed to the centralized fraework considered here. A recent survey is [27]. Related analytic work focusing on dynaic right-sizing includes [28], which reallocates resources between tasks within a data center, and [29], [3], which considers sleep of individual coponents, aong others. Typically, approaches have applied optiization using queueing theoretic odels, e.g., [31], [32], or control theoretic approaches, e.g., [33] [35]. A recent survey of analytic work focusing on energy efficiency in general is [36]. Our work is differentiated fro this literature by the generality of the odel considered, which subsues ost coon energy and delay cost odels used by analytic researchers, and the fact that we provide worst-case guarantees for the cost of the algorith, which is typically not possible for queueing or control theoretic based algoriths. 8

9 The odel and algorith introduced in this paper ost closely ties to the online algoriths literature, specifically the classic rent-or-buy (or ski rental ) proble [37]. The best deterinistic strategy for deciding when to turn off a single idle server (i.e., to stop renting and to buy ) is 2- copetitive [38]. Additionally, there is a randoized algorith which is asyptotically e/(e 1)-copetitive [39]. These algoriths have been directly translated to the data-center context in [4], [41]. These papers assue that the power requireents of servers are fixed when they are on, and that there is no perforance penalty for increasing the load on each server up to the axiu load it can support. In this setting, a judicious allocation of jobs to servers allows each server to perfor an independent rent-or-buy decision. The lower envelope algorith of [29], [3], which generalizes the standard rent-or-buy algorith, puts a device into deeper sleep ode at a tie t such that the optial solution would use if the idle period finished right after t. This is like LCP following x U downward; in [29], [3] a device ust be fully on to serve work, and so there is no equivalent to x L forcing the syste to turn on in stages. An iportant difference between these siple odels and the current paper is that the cost of the rent action ay change arbitrarily over tie in the data center optiization proble. Probles with this sort of dynaics typically have copetitive ratios greater than 2. For exaple, when rental prices vary in tie, the copetitive ratio is unbounded in general [42]. Further, for etrical task systes [43], which generalize rent-or-buy probles and the data center optiization proble, there are any algoriths available, but they typically have copetitive ratios that are at best poly-logarithic in the nuber of servers. Perhaps the ost closely related prior work fro this area is the page-placeent proble (deciding on which server to store a file), which has copetitive ratio 3 [44]. The page replaceent-proble is nearly a discrete version of the data center optiization proble where the cost function is restricted to f(x) = x 1. VIII. EXTENSIONS The general odel captures any applications other than the right-sizing probles in data centers. Intuitively, this odel seeks to iniize the su of a sequence of convex functions while sooth solutions are preferred. Other applications ay be captured by this odel, such as video streaing [45], in which encoding quality should vary depending on network bandwidth, but large changes in encoding quality are visually annoying to users. A natural generalization of the fraework of (1) is to ake a vector, corresponding to anaging resources of ultiple different types. This extension has any iportant applications, including joint capacity provision in geographically distributed data centers [25], autoatically switched optical networks (ASONs) in which there is a cost for re-establishing a lightpath [46], and power generation with dynaic deand, since the cheapest types of generators typically have very high switching costs [47] Unfortunately, LCP(w) does not naturally generalize to the case that is a vector. Moreover, although the RHC(w) w= ean background load (% total) (a) Hotail w= ean background load (% total) (b) MSR Fig. 1. Ipact of background processes. The iproveent of LCP(w) over static provisioning as a function of the percentage of the workload that is background tasks. algorith can be directly applied, it can perfor poorly when is a vector [21]. An iproved algorith for the case of heterogeneous resources with lookahead has been proposed in [21], but an algorith without lookahead is still the subject of active research. IX. SUMMARY AND CONCLUDING REMARKS This paper has provided a new online algorith, LCP(w), for dynaic right-sizing in data centers. The algorith is otivated by the structure of the optial offline solution and guarantees cost no larger than 3 ties the optial cost, under very general settings arbitrary workloads, and general delay cost and general energy cost odels provided that they result in a convex operating cost. Further, in realistic settings the cost of LCP(w) is nearly optial. Additionally, LCP(w) is siple to ipleent in practice and does not require significant coputational overhead. Moreover, we contrast LCP(w) with the ore traditional approach of receding horizon control RHC(w) and show that LCP(w) provides uch ore stable cost saving with general settings. Additionally, the case studies used to evaluate LCP(w) highlight that the cost and energy savings achievable by dynaic right-sizing are significant. The case studies highlight that if a data center has PMR larger than 3, a cost of toggling a server of less than a few hours of server costs, and less than 4% background load then the cost savings fro dynaic right-sizing can be conservatively estiated at larger than 15%. Thus, even if a data center is currently perforing valley filling, it can still achieve significant cost savings by dynaic right-sizing. ACKNOWLEDGEMENTS This work was supported by NSF grants CCF 83511, and CNS 84625, Microsoft Research, the Lee Center for Advanced Networking, and ARC grant FT REFERENCES [1] J. Hailton, Cost of power in large-scale data centers, Nov. 29. [2] L. A. Barroso and U. Hölzle, The case for energy- proportional coputing, Coputer, vol. 4, no. 12, pp , 27. [3] C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Lipach, I. Pratt, and A. Warfield, Live igration of virtual achines, in Proc. USENIX NSDI, 25, pp

10 [4] G. Chen, W. He, J. Liu, S. Nath, L. Rigas, L. Xiao, and F. Zhao, Energyaware server provisioning and load dispatching for connection-intensive internet services, in Proc. USENIX NSDI, 28. [5] E. Thereska, A. Donnelly, and D. Narayanan, Sierra: a powerproportional, distributed storage syste, Microsoft Research, Tech. Rep. MSR-TR , 29. [6] H. Aur, J. Cipar, V. Gupta, G. R. Ganger, M. A. Kozuch, and K. Schwan, Robust and flexible power-proportional storage, in Proc. ACM SoCC, 21. [7] X. Wang and M. Chen, Cluster-level feedback power control for perforance optiization, in IEEE Int. Syp. High Perforance Coputer Architecture (HPCA), 28, pp [8] D. Kusic, J. O. Kephart, J. E. Hanson, N. Kandasay, and G. Jiang, Power and perforance anageent of virtualized coputing environents via lookahead control, Cluster coputing, vol. 12, no. 1, pp. 1 15, Mar. 29. [9] P. Bodik, M. P. Arbrust, K. Canini, A. Fox, M. Jordan, and D. A. Patterson, A case for adaptive datacenters to conserve energy and iprove reliability, University of California at Berkeley, Tech. Rep. UCB/EECS , 28. [1] SPEC power data on SPEC website at [11] L. Kleinrock, Queueing Systes Volue II: Coputer Applications. Wiley Interscience, [12] R. Urgaonkar, U. Kozat, K. Igarashi, and M. Neely, Dynaic resource allocation and power anageent in virtualized data centers, in Proc. IEEE/IFIP NOMS, Apr. 21. [13] R. Das, J. O. Kephart, C. Lefurgy, G. Tesauro, D. W. Levine, and H. Chan, Autonoic ulti-agent anageent of power and perforance in data centers, in Proc. Int. Joint Conf. Autonoous Agents and Multiagent Systes, 28. [14] J. S. Chase, D. C. Anderson, P. N. Thakar, A. M. Vahdat, and R. P. Doyle, Managing energy and server resources in hosting centers, in Proc. ACM SOSP, 21, pp [15] L. L. H. Andrew, M. Lin, and A. Wieran, ity, fairness and robustness in speed scaling designs, in Proc. ACM SIGMETRICS, 21. [16] L. Rao, X. Liu, L. Xie, and W. Liu, Miniizing electricity cost: Optiization of distributed internet data centers in a ulti-electricityarket environent, in Proc IEEE INFOCOM, Mar 21. [17] H. N. Van, F. Tran, and J.-M. Menaud, Sla-aware virtual resource anageent for cloud infrastructures, in IEEE Int. Conf. Coput. Infor. Technol. (CIT), OCT 29. [18] W. Kwon and A. Pearson, A odified quadratic cost proble and feedback stabilization of a linear syste, IEEE Trans. Autoatic Control, vol. AC-22, no. 5, pp , [19] W. H. Kwon, A. M. Bruckstein, and T. Kailath, Stabilizing state feedback design via the oving horizon ethod. Int. J. Contr., vol. 37, no. 3, pp , [2] D. Q. Mayne and H. Michalska, Receding horizon control of nonlinear systes, IEEE Trans. Autoatic Control, vol. 35, no. 7, pp , 199. [21] M. Lin, Z. Liu, A. Wieran, and L. L. H. Andrew, Online algoriths for geographical load balancing, in Proc. Int. Green Coputing Conf., San Jose, CA, June 212. [22] D. Gach, J. Rolia, L. Cherkasova, and A. Keper, Workload analysis and deand prediction of enterprise data center applications, in Proc. IEEE Syp. Workload Characterization, Boston, MA, Sept. 27. [23] E. Pinheiro, R. Bianchini, E. Carrera, and T. Heath, Load balancing and unbalancing for power and perforacne in cluster-based systes, in proc. Copilers and Operating Systes for Low Power, 21. [24] B. Khargharia, S. Hariri, and M. Yousif, Autonoic power and perforance anageent for coputing systes, Cluster coputing, vol. 11, no. 2, pp , Dec. 27. [25] Z. Liu, M. Lin, A. Wieran, S. H. Low, and L. L. H. Andrew, Greening geographical load balancing, in Proc. ACM SIGMETRICS, 211. [26] A. Kansal, J. Liu, A. Singh, R. Nathuji, and T. Abdelzaher, Seanticless coordination of power anageent and application perforance, in ACM SIGOPS, 21, pp [27] A. Beloglazov, R. Buyya, Y. C. Lee, and A. Zoaya, A taxonoy and survey of energy-efficient data centers and cloud coputing systes, Advances in Coputers, vol. 82, pp , 211. [28] C. G. Plaxton, Y. Sun, M. Tiwari,, and H. Vin, Reconfigurable resource scheduling, in ACM SPAA, 26. [29] S. Irani, R. Gupta, and S. Shukla, Copetitive analysis of dynaic power anageent strategies for systes with ultiple power savings states, in Proc. Design, Autoation, and Test in Europe, 22, p [3] J. Augustine, S. Irani, and C. Sway, power-down strategies, SIAM Journal on Coputing, vol. 37, no. 5, pp , 28. [31] A. Gandhi, M. Harchol-Balter, R. Das, and C. Lefurgy, power allocation in server fars, in Proc. of ACM Sigetrics, 29. [32] A. Gandhi, V. Gupta, M. Harchol-Balter, and M. A. Kozuch, ity analysis of energy-perforance trade-off for server far anageent, Perforance Evaluation, vol. 67, no. 11, pp , 21. [33] T. Horvath and K. Skadron, Multi-ode energy anageent for ultitier server clusters, in Proc. PACT, 28. [34] Y. Chen, A. Das, W. Qin, A. Sivasubraania, Q. Wang, and N. Gauta, Managing server energy and operational costs in hosting centers, in Proc. Sigetrics, 25. [35] R. Urgaonkar, U. C. Kozat, K. Igarashi, and M. J. Neely, Dynaic resource allocation and power anageent in virtualized data centers, in Proc. IEEE/IFIP NOMS, Apr. 21. [36] S. Albers, Energy-efficient algoriths, Co. of the ACM, vol. 53, no. 5, pp , 21. [37] A. Borodin and R. El-Yaniv, Online coputation and copetitive analysis. Cabridge University Press, [38] A. R. Karlin, M. S. Manasse, L. Rudolph, and D. D. Sleator, Copetitive snoopy caching, Algorithica, vol. 3, no. 1, pp , [39] A. R. Karlin, C. Kenyon, and D. Randall, Dynaic TCP acknowledgeent and other stories about e/(e 1), in Proc. ACM Syp. Theory of Coputing (STOC), 21. [4] T. Lu and M. Chen, Siple and effective dynaic provisioning for power-proportional data centers, in Inforation Sciences and Systes (CISS), th Annual Conference on. IEEE, 212, pp [41] T. Lu, M. Chen, and L. L. H. Andrew, Siple and effective dynaic provisioning for power-proportional data centers, IEEE Trans. Parallel and Distributed Systes, To appear. [Online]. Available: [42] M. Bienkowski, Price fluctuations: To buy or to rent, in Approxiation and Online Algoriths, 28, pp [43] A. Borodin, N. Linial, and M. E. Saks, An optial on-line algorith for etrical task syste, J. ACM, vol. 39, no. 4, pp , [44] D. L. Black and D. D. Sleator, Copetitive algoriths for replication and igration probles, Carnegie Mellon University, Tech. Rep. CMU- CS-89-21, [45] M. Zink, O. Künzel, J. Schitt, and R. Steinetz, Subjective ipression of variations in layer encoded videos, in Proc. Int. Conf. Quality of service. Springer-Verlag, 23, pp [46] Y. Zhang, M. Murata, H. Takagi, and Y. Ji, Traffic-based reconfiguration for logical topologies in large-scale WDM optical networks, IEEE J. Lightwave Technology, vol. 23, no. 1, p. 2854, 25. [47] S. Kaplan, Power plants: Characteristics and costs, Congressional Research Service, 28. APPENDIX A ANALYSIS OF THE OFFLINE OPTIMAL SOLUTION In this section we will prove Lea 1 and Theore 2. Before beginning the proofs, let us first rephrase the data center optiization (1) as follows. iniize g t ( ) + β y t (9) subject to y t 1,, y t. Next, we want to work with the dual of optiization (9). The Lagrangian (with respect to the first constraint) is L(x, y, ν) = g t ( ) + β y t + = (g t ( ) + (β ν t )y t ) T 1 ν t ( 1 y t ) + (ν t ν t+1 ) + ν T x T ν 1 x and the dual function is D(ν) = inf x,y L(x, y, ν).since we are interested in the axiu of D(ν) in the dual proble, 1

11 we need only to consider the case β ν t. Then the dual function becoes D(ν) ( T ) 1 = inf g t ( ) + (ν t ν t+1 ) + ν T x T ν 1 x x T 1 = gt (ν t+1 ν t ) gt ( ν T ) ν 1 x. where g t ( ) is the convex conjugate of function g t ( ) (enforcing g t ( ) = when < ). Therefore the corresponding dual proble of (9) is 1 ax gt (ν t+1 ν t ) gt ( ν T ) ν 1 x (1) subject to ν t β, where the copleentary slackness conditions are ν t ( 1 y t ) = (11) (β ν t )y t =, (12) and the feasibility condition is, y t, y t 1 and ν t β. Using the above, we now observe a relationship between the data center optiization in (9) and the upper and lower bounds, i.e., optiizations (6) and (5). Specifically, if ν +1 = in a solution of optiization (1), then ν 1,..., ν is a solution to: 1 ax gt (ν t+1 ν t ) g ( ν ) ν 1 x (13) subject to ν t β, which is identical to the dual proble of optiization (5). Thus, the corresponding x 1,..., x is a solution to optiization (5). On the other hand, if ν +1 = β in a solution of optiization (1), then ν 1,..., ν is a solution to: 1 ax gt (ν t+1 ν t ) g (β ν ) ν 1 x (14) subject to ν t β. Let ν t = β ν t, which akes (14) becoe 1 ax gt (ν t ν t+1) g (ν ) + ν 1x βx subject to ν t β. (15) It is easy to check that the corresponding x 1,..., x is a solution to optiization (6). We require soe notation and two technical leas before oving to the proofs of Lea 1 and Theore 2. Denote h i,j (x; x S ; x E ) = j t=i g t( )+β(x i x S ) + +β j t=i+1 ( 1 ) + + β(x E x j ) + for j i, and h i,j (x; x S; x E ) = j t=i g t( )+β(x S x i ) + +β j t=i+1 ( 1 ) + +β(x j x E ) + for j i. Then the objective of (5) is h 1, (x; x ; ). Siilarly, the objective of (6) is h 1, (x; x ; x M ) where x M = ax(ax,t The first lea is to connect h i,j and h i,j : x U,t, ax x t ). (16) t Lea 3. Given x S and x E, any solution iniizing h i,j (x; x S ; x E ) also iniizes h i,j (x; x S; x E ) and vice versa. Proof: h i,j (x; x S ; x E ) h i,j(x; x S ; x E ) j =β(x i x S ) + β ( 1 ) + β(x E x j ) =β(x E x S ), t=i+1 which is a constant. Thus any iniizer of h i,j (x; x S ; x E ) or h i,j (x; x S; x E ) is also a iniizer of the other. Lea 4. Given x S and x E ˆx E, let X = (x i,..., x j ) iniize h i,j (x; x S ; x E ), then there exists an ˆX = (ˆx i,..., ˆx j ) iniizing h i,j (x; x S ; ˆx E ) such that X ˆX. Proof: If there is ore than one solution iniizing h i,j (x; x S ; ˆx E ), let ˆX be a solution with greatest ˆxj. We will first argue that x j ˆx j. By definition, we have h i,j (X; x S ; x E ) h i,j ( ˆX; x S ; x E ) and h i,j ( ˆX; x S ; ˆx E ) h i,j (X; x S ; ˆx E ). Note that if the second inequality is an equality, then x j ˆx j. Otherwise, su the two inequalities, to obtain the strict inequality (x E x j ) + + (ˆx E ˆx j ) + < (ˆx E x j ) + + (x E ˆx j ) +. Since x E ˆx E, we can conclude that x j < ˆx j. Therefore, we always have x j ˆx j. Next, recursively consider the subproble h i,j 1 ( ) with x E = x j and ˆx E = ˆx j. The sae arguent as above yields that x j 1 ˆx j 1. This continues and we can conclude that (x i,..., x j ) (ˆx i,..., ˆx j ). Lea 4 shows that the optiizations in our paper have a unique axial solution, as follows. Corollary 1. If there is ore than one solution to optiization proble (1), (4), (5) or (6), there exists a axiu solution which is not less than others eleentwise. Proof: Let us consider proble (1) with ultiple solutions and prove the clai by induction. Assue that x is a solution with the first ( 1) entries not less than those in other solutions, but x +1 < ˆx +1 where ˆx is a solution with the greatest ( +1)th entry. Since (x 1,..., x ) iniizes h 1, (x; x ; x +1), Lea 4 iplies there exists a solution ( x 1,..., x ) iniizing h 1, (x; x ; ˆx +1) which is not less than (x 1,..., x ). Thus we can replace (ˆx 1,..., ˆx ) in ˆx by ( x 1,..., x ) to get a solution with the first + 1 entries not less than other solutions. The proof for optiization proble (4), (5) and (6) are siilar and thus oitted. We now coplete the proofs of Lea 1 and Theore 2. Proof of Lea 1: Let X L = (x L,1, x L,2,..., x L, ) be the solution of optiization (5) at tie and define X U syetrically for (6). Also, let X = (x 1,..., x ) be the 11

12 first entries of the offline solution to optiization (1). We know that X L iniizes h 1, (x; x ; ) and X iniizes h 1, (x; x ; x +1). Since x +1, Lea 4 iplies X X L, and thus, in particular, the last entry satisfies x x L,. Syetrically, X U iniizes h 1, (x; x ; x M ) where x M satisfies (16). By Lea 3, X U also iniizes h 1, (x; x ; x M ). Since x +1 x M, Lea 4 iplies that X X U, and thus x x U,. Proof of Theore 2: As a result of Lea 1, we know that x [x L, x U ] for all. Further, if x > x +1, by the copleentary slackness condition (11), we have that ν +1 =. Thus, in this case, x solves optiization (5) for the lower bound, i.e., x = x L. Syetrically, if x < x +1, we have that copleentary slackness condition (12) gives ν +1 = β and so x solves optiization (6) for the upper bound, i.e., x = x U. Thus, whenever x is increasing/decreasing it ust atch the upper/lower bound, respectively. APPENDIX B ANALYSIS OF LAZY CAPACITY PROVISIONING, LCP(w) In this section we will prove Lea 2 and Theore 3. Proof of Lea 2: First, we prove that x L,w x. By definition, x L,w = x L +w,, and so it belongs to a solution iniizing h 1,+w (x; x ; ). Further, we can view the optial x as an entry in a solution iniizing h 1,+w (x; x ; x +w+1). Fro these two representations, we can apply Lea 4, to conclude that x L,w x. Next, we prove that x L x L,w. To see this we notice that x L is the last entry in a solution iniizing h 1, (x; x ; ). And we can view x L,w as an entry in a solution iniizing h 1, (x; x ; x L +w,+1) where x L +w,+1. Based on Lea 4 we get x L x L,w. The proof that x x U,w x U is syetric. Fro the above lea, we iediately obtain an extension of the characterization of the offline optiu. Corollary 2. The optial solution of the data center optiization (1) satisfies the following backwards recurrence relation {, > T ; x = (17), T. (x +1) xu,w x L,w Moving to the proof of Theore 3, the first step is to use the above leas to characterize the relationship between x and x. Note that x = x = xt +1 = x T +1 =. Lea 5. Consider the tieslots = t < t 1 <... < t = T such that xt i = x t i. Then, during each segent (t i 1, t i ), either (i) xt > x t and both xt and x t are nonincreasing for all t (t i 1, t i ), or (ii) xt < x t and both xt and x t are nondecreasing for all t (t i 1, t i ). Proof: The result follows fro the characterization of the offline optial solution in Corollary 2 and the definition of LCP(w). Given that both the offline optial solution and LCP(w) are non-constant only for tieslots when they are equal to either x U,w t or x L,w t, we know that at any tie t i where xt i = x t i and xt i+1 x t i+1, we ust have that both xt i and x t i are equal to either x U,w t i or x L,w t i. Now we ust consider two cases. First, consider the case that xt i+1 > x t i+1. It is easy to see that xt i+1 doesn t atch the lower bound since x t i+1 is not less than the lower bound. Thus xt i xt i+1 since, by definition, LCP(w) will never choose to increase the nuber of servers it uses unless it atches the lower bound. Consequently, it ust be that x t i = xt i xt i+1 > x t i+1. Since x is decreasing, both xt i and x t i atch the lower bound. Further, the next tie, t i+1, when the optial solution and LCP(w) atch is the next tie either the nuber of servers in LCP(w) atches the lower bound x L,w t or the next tie the nuber of servers in the optial solution atches the upper bound x U,w t. Thus, until that point, LCP(w) cannot increase the nuber of servers (since this happens only when it atches the lower bound) and the optial solution cannot increase the nuber of servers (since this happens only when it atches the upper bound). This copletes the proof of part (i) of the Lea. The proof of part (ii) is syetric. Given Lea 5, we bound the switching cost of LCP(w). Lea 6. cost s (X ) = cost s (X ). Proof: Consider the sequence of ties = t < t 1 <... < t = T such that xt i = x t i identified in Lea 5. Then, each segent (t i 1, t i ) starts and ends with the sae nuber of servers being used under both LCP(w) and the optial solution. Additionally, the nuber of servers is onotone for both LCP(w) and the optial solution, thus the switching cost incurred by LCP(w) and the optial solution during each segent is the sae. Next, we bound the operating cost of LCP(w). Lea 7. cost o (X ) cost o (X )+β T x t x t 1. Proof: Consider the sequence of ties = t < t 1 <... < t = T such that xt i = x t i identified in Lea 5, and consider specifically one of these intervals (t i 1, t i ) such that xt i 1 = x t i 1, xt i = x t i. There are two cases in the proof: (i) xt > x for all (t i 1, t i ) and (ii) xt < x for all t (t i 1, t i ). We handle case (i) first. Our goal is to prove that t i t=t i 1 +1 g t(x t ) t i t=t i 1 +1 g t(x t ) + β x t i 1 x t i. (18) Define X = (x,1,..., x, 1, x ) where (x,1,..., x, 1 ) iniizes h 1, 1(x; x ; x ). Additionally, define X = (x,1,..., x, ) as the solution iniizing h 1, (x; x ; x ); by Lea 3 this also iniizes h 1, (x; x ; x ). We first argue that x, = x via a proof by contradiction. Note that if x, > x, based on siilar arguent in the proof of Theore 2, we have x, = x L, which contradicts the fact that x, > x x L. Second, 12

13 if x, < x, then we can follow a syetric arguent to arrive at a contradiction. Thus x, = x. Consequently, x,t = x,t for all t [1, ] and we get h 1, (X ; x ; x M ) = h 1, (X ; x ; x M ) (19) Next, let us consider X +1 = (x +1,1,..., x +1,, x+1 ) where (x +1,1,..., x +1, ) iniizes h 1, (x; x ; x+1 ). Recalling xt is non-increasing in case (i) by Lea 5, we have X +1 X by Lea 4. In particular, x +1, x. Thus h 1,+1(X +1 ; x ; x M ) (2) h 1, ((x +1,1,..., x +1, ); x ; x M ) + g +1 (x+1 ) =h 1, ((x +1,1,..., x +1, ); x ; x By definition of X, we get ) + g +1 (x +1 ) h 1, ((x +1,1,..., x +1, ); x ; x ) (21) h 1, (X ; x ; x ) h 1, (X ; x ; x M ) Cobining equations (19), (2) and (21), we obtain h 1,+1(X +1 ; x ; x M ) h 1, (X ; x ; x M )+g +1 (x+1 ). By suing this equality for [t i 1, t i ), we have t i t=t i 1+1 g t (xt ) h 1,t i (X ti ; x ; x M ) h 1,t i 1 (X ti 1 ; x ; x M ). Since xt i 1 = x t i 1, xt i = x t i, we know that both X ti 1 and X ti are prefixes 2 of the offline solution x, thus X ti 1 is the prefix of X ti. Expanding out h ( ) in the above inequality gives (18), which copletes case (i). In case (ii), i.e., segents where xt < x for all t (t i 1, t i ), a parallel arguent shows that (18) again holds. To coplete the proof we cobine the results fro case (i) and case (ii), suing equation (18) over all segents (and the additional ties when xt = x t ). We can now prove the copetitive ratio in Theore 3. Lea 8. cost(x ) cost(x ) + 2cost s (X ). Thus, LCP(w) is 3-copetitive for the data center optiization (1). Proof: Cobining Lea 7 and Lea 6 gives that cost(x LP C(w) ) cost(x ) + β x t x t 1. Note that, because both LCP(w) and the optial solution start and end with zero servers on, we have T x t x t 1 = 2 T (x t x t 1) +, which copletes the proof. All that reains for the proof of Theore 3 is to prove that the bound of 3 on the copetitive ratio is tight. Lea 9. The copetitive ratio of LCP(w) is at least 3. Proof: The following is a faily of instances paraeterized by n and 2, whose copetitive ratios approach the bound of 3. The operating cost for each server is defined as f(z) = z + f for z 1 and f(z) = otherwise, 2 That is, X ti 1 is the first t i 1 coponents of x and X ti is the first t i coponents of x. whence g t ( ) = f(λ t / ). The switching cost is β =.5. 1 Let δ (1, 1.5) be such that n = log δ δ 1. The arrival rate at tie i is λ i = δ i 1 for 1 i n, and λ i = for n < i T, where T > β/f + n with f = β(δ 1) n(δ n 1). For the offline optiization, denote the solution by vector x. First note that x i = x n for i [1, n] since x i is nondecreasing for i [1, n] and, for the above f, d dx [xf(λ i/x)] < for x [λ i, x n]. (22) Moreover, x i = for i [n + 1, T ]. Hence the iniu cost is n λ i i=1 (x + (nf + β)x n. Then by the first order n (stationarity) condition ) 1 we get (x n) = (nf + β)(δ 1) ( 1)(δ n 1), (23) which is saller than λ n because nf + β 1, 1 1 and δ > 1. Thus it is feasible (f(λ i /x n) is finite). The cost for the offline optial solution is then C = 1 (nf + β)x n. We consider with w = before considering the general case. Let C [i,j] denote the cost of LCP() on [i, j]. For the online algorith LCP(), denote the result by vector ˆx. We know ˆx i is actually atching x L i in [1, n] (x U i is not less than x i = x n), thus ˆx i is non-decreasing for i [1, n] and ˆx n = x n. By the sae arguent as for x n, we have =1 (ˆx ) = (f + β)(δ 1) ( 1)(δ 1) Thus the cost for LCP() in [1, n] is n ( ) λ C [1,n] = (ˆx ) 1 + f ˆx + βx n > > = Thus by (23) C [1,n] x n ) 1 n ( δ ( 1) (f + β)(δ 1) ( 1)(δ 1) =1 ( β(δ ) 1 1) n δ + βx n 1 =1 ( β(δ ) 1 1) δ n 1 1 (δ 1)δ 1 + βx n (24) + βx n δ n 1 > (δ n 1) β(δ 1) 1/ ( 1)(δ 1)δ 1 + β > δn 1 β(δ 1) δ n ( 1)(δ 1)δ 1 + β As n, both δ 1 and (δ n 1)/δ n 1, since n =. Since is independent of δ, L Hospital s Law gives log δ 1 δ 1 C [1,n] li n x li n δ 1 βδ 1 ( 1)(δ 1 ( 1)δ 2 ) + β = 1 β + β Now let us calculate the cost for LCP() in [n + 1, T ]. For > n, by LCP(), we know that x will stay constant 13

14 until it hits the upper bound. Let X = {x,t } be the solution of optiization (6) in [1, ] for any > n. We now prove that for such that ( n)f < β, we have x, x n, and thus ˆx = x n. Note that x,n x n since x n belongs to the lower bound. Given x,n, we know that x,n+1,..., x, is the solution to the following proble: iniize f X,t + β (X,t 1 X,t ) + t=n+1 subject to X,t t=n+1 Let x in = in{x,n,..., x, }. Then f x,t + β (x,t 1 x,t ) + t=n+1 t=n+1 t=n+1 f x in + β(x,n x in ) ( n)f x,n The last inequality is obtained by substituting β > ( n)f. We can see that x,i = x,n (i [n + 1, ]) is a solution, thus x, x n. (Recall that, if there are ultiple solutions, we take the axiu one). Therefore, we have C [n+1,] = ( n)f x n. Since f as δ 1, we can find an so that ( n)f β, and thus C [n+1,] βx n. (25) By cobining the cost for LCP() in [1, n] and [n + 1, T ], we have C [1,T ] /C 1 β + 2β 1 (nf + β) = 3 2/ 1 + nf /β. Using the relationship between n, f and δ, we can choose a large enough and n to ake this arbitrarily close to 3. This finishes the proof for LCP(). Now let us consider LCP(w) for w >. Denote the solution of LCP(w) by ˆx. At tie [1, n w], LCP(w) solves the sae optiization proble as LCP() does at + w. Thus ˆx = ˆx +w of LCP(). Thus C [1,n w] = = ( λ (ˆx ) 1 + f ˆx n ( 1 δ w n w =1 =1+w ) + βx n λ (ˆx ) 1 + f ˆx ) + βx n 1 > δ w C [1+w,n] By pushing n, and hence δ 1, we have C [1,n w] /C [1,n] 1. And for > n + 1, if f ( n) < β, then C [n+1, w] = ( w n)f x n. By pushing δ 1, we can find a such that C [n+1, w] βx n, the sae as (25). Therefore, as δ 1, we have C [1,T ] /C [1,T ] 1, thus the supreu over arbitrarily large 2 and arbitrarily sall δ > 1 of the copetitive ratio of LCP(w) on this faily is also 3. Finally, the following lea ensures that the optiizations solved by LCP(w) at each tieslot reain sall. Lea 1. If there exists an index [1, t 1] such that x U t,+1 < x U t, or x L t,+1 > x L t,, then (x U t,1,..., x U t, ) = (x L t,1,..., x L t, ). No atter what the future functions g i ( ) are, solving either (5) or (6) in [1, t ] for t > t is equivalent to solving two optiizations: one over [1, ] with initial condition x and final condition x U t, and the second over [ +1, t ] with initial condition x U t,. Proof: Consider the case x L t,+1 > x L t,. By the copleentary slackness conditions (12), the corresponding dual variable ν +1 = β. Based on the arguent in the proof of Theore 2, we know the dual proble up to tie is (14), which is identical to (15), the dual proble of (6). Since the optial x i depends only on ν i and ν i+1, the optial x i for i are copletely deterined by ν i for i [1, + 1], with ν +1 = β. Hence (x L t,1,..., x L t, ) = (x U t,1,..., x U t, ). The proof for the case x U t,+1 < x U t, is syetric. Notice that (x U t,1,..., x U t,t) iniizes h 1,t(x; x ; x M ) and thus h 1,t (x; x ; x M ) based on Lea 3, (x L t,1,..., x L t,t) iniizes h 1,t (x; x ; ). No atter what the future functions g i ( ) are, the first t entries of its solution ust iniize h 1,t (x; x ; +1 ) for soe +1 [, x M ] by the axiality of x M. Based on Lea 4, the first t entries are bounded by (x U t,1,..., x U t,t) and (x L t,1,..., x L t,t). However, we have seen that (x U t,1,..., x U t, ) = (x L t,1,..., x L t, ), thus the first entries of its solution are equal to (x U t,1,..., x U t, ) no atter what the future is. Minghong Lin received the B.Sc. degree in Coputer Science fro University of Science and Technology of China in 26, and the M.Phil degree in Coputer Science fro the Chinese University of Hong Kong in 28. Currently he is a Ph.D. student in Coputer Science at California Institute of Technology. His research interests include energy efficient coputing, online algoriths and nonlinear optiization. He has received best paper award at IEEE INFOCOM 11, IGCC 12 and best student paper award at ACM GREENMETRICS 11. Ada Wieran is an Assistant Professor in the Departent of Coputing and Matheatical Sciences at the California Institute of Technology, where he is a eber of the Rigorous Systes Research Group (RSRG). He received his Ph.D., M.Sc. and B.Sc. in Coputer Science fro Carnegie Mellon University in 27, 24, and 21, respectively. He received the ACM SIGMETRICS Rising Star award in 211, and has also received best paper awards at ACM SIGMETRICS, IFIP Perforance, IEEE INFOCOM, and ACM GREENMETRICS. He has also received ultiple teaching awards, including the Associated Students of the California Institute of Technology (ASCIT) Teaching Award. His research interests center around resource allocation and scheduling decisions in coputer systes and services. More specifically, his work focuses both on developing analytic techniques in stochastic odeling, queueing theory, scheduling theory, and gae theory, and applying these techniques to application doains such as energy-efficient coputing, data centers, social networks, and the electricity grid. 14

15 Lachlan Andrew (M 97-SM 5) received the B.Sc., B.E. and Ph.D. degrees in 1992, 1993, and 1997, fro the University of Melbourne, Australia. Since 28, he has been an associate professor at Swinburne University of Technology, Australia, and since 21 he has been an ARC Future Fellow. Fro 25 to 28, he was a senior research engineer in the Departent of Coputer Science at Caltech. Prior to that, he was a senior research fellow at the University of Melbourne and a lecturer at RMIT, Australia. His research interests include energy-efficient networking and perforance analysis of resource allocation algoriths. He was corecipient of the best paper award at IGCC212, IEEE INFOCOM 211 and IEEE MASS 27. He is a eber of the ACM. Eno Thereska received his PhD (ECE, 27), Masters (ECE, 23) and Bachelor (ECE + CS + Math inor, 22) degrees all fro Carnegie Mellon. He s been a Researcher with Microsoft Research since 27. He has broad interests in systes but also enjoys venturing into other areas occasionally, like achine learning and HCI. He was co-recipient of the best paper award at IEEE INFOCOM 211 and Usenix FAST 25, and the best student paper at Usenix FAST