Using Batteries to Reduce the Power Costs of Internet-scale Distributed Networks

Transcription

1 Using Baeries o Reduce he Power Coss of Inerne-scale Disribued Neworks Darshan S. Palasamudram, Ramesh K. Siaraman, Bhuvan Urgaonkar, Rahul Urgaonkar Penn. Sae Univ., UMass Amhers, Akamai Technologies, Rayheon BBN Technologies [email protected], [email protected], [email protected], [email protected] ABSTRACT Modern Inerne-scale disribued neworks have hundreds of housands of servers deployed in hundreds of locaions and neworks around he world. Canonical examples of such neworks are conen delivery neworks (called CDNs) ha we sudy in his paper. The operaing expenses of large disribued neworks are increasingly driven by he cos of supplying power o heir servers. Typically, CDNs procure power hrough long-erm conracs from co-locaion providers and pay on he basis of he power (KWs) provisioned for hem, raher han on he basis of he energy (KWHs) acually consumed. We propose he use of baeries o reduce boh he required power supply and he incurred power cos of a CDN. We provide a heoreical model and an algorihmic framework for provisioning baeries o minimize he oal power supply and he oal power coss of a CDN. We evaluae our baery provisioning algorihms using exensive load races derived from Akamai s CDN o empirically sudy he achievable benefis. We show ha baeries can provide up o 14% power savings, ha would increase o 22% for more power-proporional nex-generaion servers, and would increase even more o 35.3% for perfecly power-proporional servers. Likewise, he cos savings, inclusive of he addiional baery coss, range from 13.26% o 33.8% as servers become more power-proporional. Furher, much of hese savings can be achieved wih a small cycle rae of one full discharge/charge cycle every hree days ha is conducive o saisfacory baery lifeimes. In summary, we show ha a CDN can uilize baeries o significanly reduce boh he oal supplied power and he oal power coss, hereby esablishing baeries as a key elemen in fuure disribued nework archiecure. While we use he canonical example of a CDN, our resuls also apply o oher similar Inerne-scale disribued neworks. Caegories and Subjec Descripors C.4 [Performance of Sysems]: Modeling echniques; Design sudies General Terms Algorihms, Experimenaion, Economics Keywords Inerne Conen Delivery, Cloud Compuing, Energy E ciency, Energy Sorage, Nework Archiecure 1. INTRODUCTION Inerne-scale services ha deploy large disribued neworks of servers across he globe are fundamenally ransforming all aspecs of human aciviy by enabling a wide range of applicaions and funcionaliy. Canonical examples of such neworks are Conen Delivery Neworks 1 (CDNs, for shor) ha we sudy in his work. CDN s deliver Web conen, applicaions, 1 For convenience, we use he erm CDNs o denoe delivery neworks for no jus Web conen bu also media and applicaions alhough he sysem archiecure can be di eren in ways ha are no maerial o he curren work. [20] provides a deailed discussion of hese archiecural di erences. Permission o make digial or hard copies of all or par of his work for personal or classroom use is graned wihou fee provided ha copies are no made or disribued for profi or commercial advanage and ha copies bear his noice and he full ciaion on he firs page. To copy oherwise, o republish, o pos on servers or o redisribue o liss, requires prior specific permission and/or a fee. SOCC 12, Ocober 14-17, 2012, San Jose, CA USA Copyrigh 2012 ACM /12/10...$15.00.

2 and sreaming media over he Inerne wih high reliabiliy, performance, and scalabiliy. CDNs are an early example of a Plaform-as-a-Service (PaaS) cloud ha provides highly-disribued hosing and delivery services for conen and applicaion providers. For insance, a large CDN such as Akamai s consiss of over 100, 000 servers disribued in over 75 counries and over 1000 neworks and serves 15% o 30% of he global Web ra c [1, 14, 20]. Modern CDNs hos and deliver Web sies (boh saic and dynamic conen), Web-based and IP-based applicaions (such as e-commerce porals and SaaS applicaions), high-definiion (HD) sreaming media (boh live and on-demand), cloud sorage services, and even disribued compuing where Java applicaion componens are run on he CDN s edge plaform [20]. In he ques for greaer performance and scalabiliy, CDNs deploy clusers of servers in daa ceners (i.e., colocaion faciliies) around he world a he edges of he Inerne so as o be proximal in he nework sense o users around he world. Each cluser consiss of servers deployed in a specific colocaion faciliy a a specific locaion. A cluser can conain anywhere from ens of servers in a small Tier-3 ISP or a universiy o housands of servers in a large colocaion faciliy in a major mero area. A CDN s servers cooperaively deliver conen and applicaions o opimize he performance experienced by he cliens. Conen and applicaions can ypically be replicaed on demand o any server of he CDN. Each clien reques is roued by he CDN s load balancing sysem o an opimal server wihin a cluser ha can serve he conen wih high availabiliy and performance. In paricular, he load balancing sysem ensures ha a user is roued o a server ha is live, no overloaded, and is locaed in a cluser ha is proximal o he user in he sense of having a communicaion pah wih low laency, low loss, and high hroughpu. An imporan consideraion for he CDN is o minimize he cos incurred for conen and applicaion delivery. The cos of deploying clusers of servers around he globe includes he capial expendiure (CapEx) of procuring he servers hemselves and he expendiure of operaing hem (OpEx). Pure-play CDNs such as Akamai and Limeligh seldom own colocaion faciliies hemselves. Insead, hese CDNs rely upon colocaion providers (e.g., Swich and Daa) o house heir servers. Similarly, hese CDNs rely upon nework providers (e.g., Comcas) for heir nework conneciviy. The OpEx of a CDN ypically involves muliple componens 2 : a ren for he rack space for he CDN s servers, he power cos for powering he servers, and a bandwidh cos for he ra c served o users by he CDN s servers. Increasingly, an imporan componen of he OpEx of large disribued sysems is he cos of power. While bandwidh coss were he dominan facor in a CDN s OpEx in he pas, bandwidh prices have roughly halved each year during he pas decade. For insance, i cos $0.15 o deliver an MB in 1998 bu only coss abou $ per MB oday, a decrease of roughly 1.8X per year. In sark conras, he cos of power has been rising over he pas decade [11] and is now a key facor o reckon wih. More imporanly, power coss are expeced o rise even furher during he nex decade and are, herefore, likely o accoun for an even larger share of he OpEx of large disribued neworks. Re-archiecing nex-generaion disribued neworks wih energy 3 as a firs-order principle is an imporan long-erm goal of our work. To ha end, in his paper, we formulae and examine novel echniques for reducing he power expenses ha a disribued nework incurs by using energy sorage devices such as baeries. 1.1 How Baeries Can Reduce Power Coss We summarize our key idea ha we hen explore in deail hrough he res of he paper. Colocaion providers ypically charge heir cusomers for power using a ari model called he supplied power model. In his model, CDNs pay each colocaion faciliy for he amoun of power supplied o (i.e., provisioned for) heir server racks, no for he energy heir servers acually use. Figure 1 shows he power demand from he CDN s servers in a single cluser. The CDN pays for P supply, irrespecive of he peak power used (P peak ) or he acual energy used (area under he power demand curve). For insance, if a CDN is o ered a uni power price c p of $150 per KW per monh and wans he colocaion provider o provision 6.24 KW of power (208 V 30 Amps for abou 20 ypical servers), hen i would pay 6.24 $150 = $936 per monh o he colocaion provider. The maximum power drawn by he CDN s servers may no exceed he supplied power a any ime, and exceeding ha hreshold could rip circui breakers and resul in penalies. Therefore, i is cusomary o incorporae a small safey margin by procuring slighly more power supply han he expeced peak demand. Colocaion providers have always had cenralized baeries o provide an uninerruped power supply (UPS) in case he power uiliy fails. Recen echnology rends allow incorporaing baeries a he rack- or server-level o implemen disribued UPS [22], which improves energy conversion e ciency by requiring one less AC-DC conversion han cenralized UPS. This 2 We capure he cos caegories ha are ypical. However, i should be noed some individual variaions may exis, e.g., a small ier-3 nework may hos he CDN s servers for free if hose servers significanly decrease he upsream ra c ha his nework pays for. 3 Minimizing energy consumpion and energy coss are wo complimenary bu di eren objecives. We sudy coss raher han consumpion, while recen work such as [18] are oriened owards minimizing CDN energy consumpion.

3 Figure 1: The power demand from servers in a CDN s cluser. The CDN s power cos is proporional o he supplied power (P supply ), no he peak power draw (P peak ) or he oal energy used (area under he power demand curve). Using a baery, he power demand can be saisfied by a power supply of P ba where he baery is discharged when demand is high and recharged when he demand is low, enabling he CDN o reduce he supplied power by a leas P peak P ba. echnological developmen makes i possible for he firs ime for a CDN o deploy baeries for is own purposes. The key insigh of our work is ha he disribued UPS archiecure can also be e ecively used for reducing he power supply ha mus be provisioned for each of a CDN s clusers, resuling in a cos reducion in he supplied power ari model. To ake he example in Figure 1, wihou he use of baeries, a CDN would have o provision P supply o be a leas P peak. However, wih a baery of su cien capaciy, i migh su ce o provision P supply o be a leas P ba. Specifically, he baery could discharge and supply power when he power demand exceeds P ba and i could draw power o charge back again whenever he power demand is smaller han P ba. Thus, he CDN can decrease is provisioned power by a leas (P peak P ba )by using a baery, resuling in a decrease in he elecriciy bill of c p(p peak P ba ). Noe he key role ha he supplied power ari model plays in he above cos savings, since baeries a bes do no change he oal energy used, and a wors increase energy usage due o conversion losses. Cloud providers ha do no require CDNlike Inerne-scale nework deploymens in hundreds of locaions and ISPs around he world have oher deploymen opions ha may resul in di eren ari models for power pricing. For insance, a cloud provider who needs only fewer locaions for deploymen can own a few big daa ceners and can procure energy direcly from power uiliies where he cos could be based on peak and/or average energy usage. Such usage-based power pricing exiss bu is less common in CDNs oday, where pure-play CDNs ypically ren (raher han own) space for heir servers and procure power supply from he co-locaion faciliy raher han direcly from he uiliy. 1.2 Our Conribuions We sae and solve wo key opimizaion problems in power supply and baery provisioning for a CDN: minimizing he oal power supply (TP) and minimizing he oal cos (TC). The oal power supply TP (in KW) of a CDN equals P i SiPi, where P i is he supplied power (per server) 4 and S i is he number of servers a cluser i. The oal cos TC (in $s) of he CDN is he sum of is power cos and he amorized cos of procuring he baery. We heoreically characerize he benefis ha baeries provide and how hese benefis vary wih power demand, baery characerisics, power prices, baery prices, baery lifeimes, and server power proporionaliy. We also develop algorihms o solve he power supply minimizaion (TPM) and he power cos minimizaion (PCM) problems. Our algorihm and LP-formulaions provide a basis for provisioning ools ha a CDN can employ o derive how much power and baeries are needed a each cluser o achieve he larges cos reducions. Noe ha our opimizaions are o ine compuaions. We envision ha in a real deploymen, hese compuaions would be performed periodically (say, once a monh or once a quarer) on prediced power demands a a ime-scale where conracs and deploymens can be changed. In paricular, hey are no online compuaions ha aemp o respond o real-ime variaions in power demand. We use our heoreical formulaion and algorihms on exensive realisic load races from Akamai s CDN and model he power conracs prevalen in he CDN indusry o make an assessmen of he power and cos reducion achievable by using baeries. Our aim is no o provide a deailed sysem-level implemenaion of a CDN employing baeries in his fashion. Raher, our primary conribuion is o assess he viabiliy of baeries as a key elemen of nex-generaion CDN archiecure. Here, we make several specific conribuions ha we lis below. By convenion, we express baery capaciy in he uni of minues per server where a B-minue baery can power a server for B minues a is peak load. 4 For noaional ease, we express load, power demand, power supplied, and baery parameers on a per-server basis.

4 Even a small 5-minue baery (comparable o hose in UPS sysems oday) can provide a 7% power saving, while a large 40-minue baery provides 14%. As servers become more power-proporional by decreasing heir idle power, baeries provide even more power savings. For a nex-generaion server whose idle power is only 20% of is peak power, baeries can achieve up o 22% power savings. For a perfecly power-proporional server wih zero idle power, baeries can achieve 35.3% savings. Mos of he power savings are achievable wih a small cycle rae of one full discharge/charge cycle every hree days. This would allow baeries o las for abou 5 years, which is a he higher-end of server refresh duraions. Heerogenous baery deploymens and power-aware global load balancing does no provide su o warran he exra complexiy. cien exra power savings The ypical cos savings due o baeries is 13.26%, bu increases o 21.4% for more power-proporional nex-generaion servers, and increases even more o 33.8% for perfecly power-proporional servers. As an aside, noe ha in he supplied power ari model, some echniques for energy usage reducion, such as urning o idle servers in a CDN [18], do no necessarily resul in a significan reducion in he power cos. Baeries provide CDNs he complimenary opion of reducing he power cos, wihou a significan change in he energy usage. Furher, noe ha while we use he canonical example of a CDN hroughou our sudy, much of our echniques and resuls also apply o oher Inerne-scale disribued neworks ha deploy in colocaion faciliies, procuring energy using he commonly-used supplied power ari model. 2. BACKGROUND AND DATA SETS 2.1 CDN Archiecure A modern CDN is a large disribued sysem of servers where he servers are deployed in clusers hroughou he world. Conen providers such as Web porals, SaaS applicaion providers, e-commerce sies, news oules, media companies, social neworks, and movie disribuion services use CDNs o hos and deliver heir conen and applicaions. When a user accesses conen or an applicaion hosed by he CDN, he user is assigned o a specific server wihin a specific cluser by he CDN s load balancing sysem. The user hen proceeds o download he requesed conen from he chosen server wihin he chosen cluser. The user accesses induce load on he servers ha in urn induces a power demand ha mus be saisfied from he provisioned power supply a each cluser. Noe ha, unlike in cerain oher IT domains, he load wihin a CDN mus usually be saisfied immediaely and canno be deferred, since hese are real-ime requess from users. In fac, a primary goal of a CDN is o minimize he response ime o he user. The load balancing sysem of a CDN has wo componens: he global load balancing componen ha chooses he opimal cluser for each user, and he local load balancing componen ha chooses he opimal server wihin he chosen cluser. The load balancing sysem is opimized for maximizing availabiliy and performance, while also minimizing bandwidh coss. For much of our work, we assume ha he load balancing sysem is unalered and hence he load and power demand of each cluser are inpus o our problem and canno be changed by moving load beween clusers. However, in Secion 6, we sudy he impac of alering global load balancing iself o make i power-aware and we ask how much more savings are possible wih his addiional capabiliy. Furhermore, we make he simplifying assumpion 5 ha he CDN s servers are homogenous and ha he local load balancing sysem disribues he incoming load ino a cluser evenly among servers so ha each server receives he average cluser load. For a comprehensive reamen of CDN archiecure, please refer o [20]. 2.2 Power Power Prices. While in realiy he uni power price varies from one colocaion faciliy o anoher, he variaion wihin a geographical region (such as a counry) is smaller han across regions. For insance, while mos major US colocaion providers would currenly provide a cos in he range $125 o $175 per KW per monh, he cos in Europe is larger and ends o be in he range $275 o $325 per KW per monh. Noe ha he power price is a blended price ha also implicily pays for he rack space, cooling power, and oher overheads susained by he colocaion provider and so canno be direcly compared wih raw per KWH elecriciy pricing from a power uiliy. I is imporan also o conras pricing in he supplied power model ha is associaed wih long-erm conracs beween he CDN and he colocaion provider ha specify boh he supplied power and uni power price wih he real-ime power prices 5 Neiher of hese assumpions is sricly rue in a real-life CDN bu hese are reasonable simplificaions for our work.

5 in wholesale elecriciy markes considered in recen work [21, 23]. The wholesale elecriciy prices show fine-grained variaion across ime (e.g., on an hourly-basis) and space (from one locaion o anoher) caused by he real-ime supply and demand imbalances and ransmission ine ciencies. Exposure o such fine-grained variaions are, of course, absen in supplied power pricing conracs and is hence no exploiable by a CDN. In our experimens, we use a single blended uni power price for all clusers bu evaluae di eren values for c p saring wih he value of $150/KW/monh ha is ypical of curren US prices [2] - a low value ha will likely rise over he nex decade. We also consider a moderae value of $300/KW/monh ha is similar o prices Europe oday, and a high value of $500/KW/monh o model a siuaion where here is a sharp increase in power prices over he nex decade. Power Consumpion and Power Proporionaliy. We use he sandard linear model [10] where he power (in Was) consumed by a server serving load l is power( ) = P idle +(P peak P idle )l, (1) where he load 0 apple l apple 1 is he server s resource uilizaion 6, P idle is he power consumed by an idle server, and P peak is he power consumed by he server under peak load. Mos servers oday consume significan amouns of power even when idle. A holy grail for server design is o make hem power proporional by making P idle zero. We define a facor ha we call he power proporionaliy facor (PPF) ha equals P peak P idle P peak. PPF measures he degree o which a server is power proporional. Several power managemen echniques have been proposed o make servers power proporional [12]: some minimize energy a he circui level when idle, some deacivae componens no in use (dynamic componen deacivaion or DCD), ohers reduce energy consumpion by slowing server componens (dynamic volage/frequency scaling for CPU/Memory, or DRPM for disks), while sill ohers power o enire servers during non-peak hours, eiher wihin a daa cener [26] or a he level of a complee disribued service spanning several daa ceners [18]. While we are agnosic o wha echnique is used, we model hree di eren values for PPF. Our baseline is he curren generaion of servers engineered for energy e ciency ha is used in deployed CDNs wih a PPF of 0.6 (P peak =250W, P idle =100W ). In anicipaion of nex-generaion servers, we model a PPF of 0.8 (P peak =250W and P idle =50W ). Finally, we also consider he perfecly power-proporional server wih a PPF of 1.0 (P peak =250W and P idle =0W ). While wheher or no he ideal is reachable is debaable, here has been exciing recen research ha shows drasic decreases in idle power, e.g., he PowerNap mechanism where a reducion of he e ecive idle power of a blade server from 270W o 10.4W is repored [19]. Trends in power proporionaliy are paricularly imporan in CDNs as he ypical server uilizaions are kep low (generally under 40%, see Figure 3) o mainain high performance and adequae headroom for flash crowds, resuling in even smaller power e ciencies (in erms of used power vs. idle power) han wha PPF implies. 2.3 Baeries In curren colocaion faciliies, energy sorage capabiliy exiss in he form of cenralized UPS unis, ypically using leadacid baeries, ha supply power o he faciliy during a poenial ouage of he uiliy. Upon a uiliy failure, he charge sored in he UPS baeries is used o supply power for he duraion i akes for backup sources (e.g., diesel generaors) o sar up and ake over he role of supplying power. Whereas his ransiion is ypically only seconds long, for a variey of reasons (boh echnical and economic) he UPS unis end up being significanly over-provisioned and can acually keep he daa cener powered for up o a few minues [17]. Recen research has shown how colocaion faciliies can ap ino his energy wihin UPS baeries for opimizing coss [15, 23], while sill providing he reliabiliy guaranees ha a radiional way of using he UPS o ers [16]. In fac, a colocaion faciliy may find i cos-e ecive o procure even more energy sorage capaciy han o ered by ypically provisioned UPS unis [17]. Moivaed by his, we explore cos saving opporuniies for CDNs by using baeries. To realize such cos savings, he CDN would need o provide is clusers wih access o baeries ha hey can conrol/operae according o heir needs. In he curren hosing model, where he CDN rens rack space and power from colocaion faciliies for is clusers, a CDN does no have conrol over he faciliies cenralized UPS. However, he adven of disribued UPS [22] makes i feasible for a CDN o provision addiional rack-level or server-level baeries for power and cos savings ha are enirely under he conrol of he CDN. While our work does no assume where he baeries are locaed (rack- versus server-level), we do assume ha he baery is owned and operaed by he CDN. Furhermore, we assume ha even when i does no have per-server baeries (e.g., i may have per-rack UPS), i employs suiable sharing/virualizaion echniques ha allow each server o operae as if 6 In pracice, one may employ a more sophisicaed model ha considers separaely he uilizaion of various server resources such as CPU, memory, I/O devices, ec., and expresses he server s power consumpion in erms of hese. We choose o employ a simpler model which capures well servers where he CPU is he dominan power consuming resource, allowing us o ignore he power consumed by oher resources.

6 UPS Runime (Minues) (40% load, 76 minues) (%) Load conneced o he UPS (relaive o UPS capaciy 100 KW) (a) UPS runime Number of charge-discharge cycles (20% DoD, 2800 cycles) Deph of discharge - DoD (%) (b) UPS baery lifeime Figure 2: Lifeime char for a ypical lead-acid baery. igure 3: Runime char shows he amoun of ime he PS can supply power a a specified load level. Lifeime char shows he number of charge/discharge cycles usainable are relevan forodi our opimizaions. eren DoD levels. i possessed is own dedicaed baery. Correspondingly, mos of our subsequen discussion will be in erms of a single server employing is baery for savings in supplied power and coss. We model he following key characerisics of baeries ha Baery Capaciy B. Capaciy B represens he amoun of energy a baery can sore in KWH/server or Joules/server. We use he erm size in lieu of capaciy as well. Following convenion, capaciy is also expressed in he unis of ime by compuing he amoun of ime he baery can susain a server a peak load, i.e., 1 KWH is equivalen o 60/P peak minues of capaciy ariaions o oher baery echnologies. In he following disussion, wemaximum use examples Discharge and Charge whenever Raes. A baery necessary has limis onwih he maximum daarae from Pd max (where P peak is expressed in unis of KW). 100KWAPCUPS,hough can discharge (resp., charge). We assume heha mehodology a baery is able o discharge general a a raeand su load, i.e., P an be applied d max o even larger unis (which simply use more during a uiliy ouage. We se Pd aery cells). This UPS allows max = P peak and allow Pc baery packs max he range 5-10 for lead-acid baeries. ha can susin 9, 24, 41,..., minues of operaions a he highes load vel (100 KW). Examples below use a 24 minue baery ack, allowing up o 48 minues wih 1+1 redundancy. ischarge Behavior of a UPS. The UPS runime for a ad level (or power drawn from UPS) measures he ime ver which he UPS discharges saring from a fully charged ae, unil i does no have su cien charge o susain he quired power draw. In Figure 3(a), we presen he runime har of he 100KW UPS (char available on APC Web sie). e see ha he runime does no scale linearly wih he load ue o high energy losses a higher load levels. For insance, baery. e UPS shown in Figure 3(a) runs for 236 minues a 20% ad bu has a runime of only 76 minues (insead of 118 inues) when he load is doubled o 40%. PS Lifeime. This depends on several facors including: ) number of imes i undergoes charging/discharging, (ii) xen of discharge during is operaion, and (iii) issues such s grid corrosion, model, bu also analyze dry-ou, he exra ec. power [32]. and cos Each savings charge/discharge heerogeneiy can provide. ycle causes acive maerial (e.g. lead) o be shed from advanages. e elecrode plaes of he baeries, gradually reducing is moun and hence he lifeime [28]. For a given cycle, he eph of discharge (DoD) of a baery represens he fracon of oal baery capaciy discharged from he baery hen recharging begins. DoD is expressed as a percen- KW (resp., Pc max KW) a which i cien o power a server a maximum P peak. Noe ha his is also a requiremen for UPS baeries, as hey need o be able o power he server o be smaller. The discharge/charge raio r = P d max and is in Energy Loss Facor. The use of baeries resuls in energy loss in wo key ways: (i) loss due o AC-DC conversion (one conversion for disribued UPS and wo conversions for cenralized UPS), and (ii) leakage (whereby he baery loses energy merely wih he passage of ime). Whereas (ii) can be imporan for cerain echnologies (e.g., flywheels [25]), i is negligible for lead-acid baeries (over he charge/discharge duraions we deermine appropriae in Secion 4.2) ha is mos ap for our purposes. Therefore, we ignore (ii) and employ a loss facor o denoe he fracional energy los in he process of charging. We assume = 0.15 ha is ypical for lead-acid baeries. Baery Lifeime. I is crucial ha we model he lifeime of a baery as cos mus be incurred in replacing i. The lifeime L of a baery is dependen on complex facors ha include environmenal facors (e.g., emperaure), he echnology iself, and he manner in which i is used. A ypical lifeime desired from baeries in our CDN would be 5 years (equal o he ypical server refresh/upgrade cycle). Addiionally, we also consider a more conservaive lifeime of 3 years, and a wors-case lifeime of 1 year. A well-known relaionship beween a baery s usage and is expeced lifeime is given by he lifeime char ha presens he number of charge/discharge cycles a a cerain deph-of-discharge (DoD) ha he baery can undergo (on average) reliably [3]. Consequenly, our algorihms limi he cycle rae ha equals he number of full discharge/charge cycles performed per day, so ha a baery may las for is desired lifeime. Figure 2 depics he lifeime char for ypical lead-acid Baery Prices. The baery coss involve a uni price c b (in $ per KWH) ha is echnology dependen, e.g., c b is lower for lead-acid baeries han for lihium-ion or ulracapaciors. We consider muliple values for c b and lifeime L, saring wih he value of c b =$100/KWH and a lifeime L = 5 years ha is ypical for lead acid baeries ha are mos commonly used and are inernally similar o he omnipresen car baeries [5, 25]. We also consider a conservaive value of c b =$300/KWH and a lifeime L =3 years, and a wors case scenario of c b =$500/KWH and a lifeime L =1 year. Baery Deploymen Model. Our baery deploymen model considers wo opions: he homogenous model where he same baery capaciy B (per server) is deployed across every cluser of he CDN and he heerogenous model where each cluser i has a (poenially) di eren amoun of baery capaciy B i deployed per server. We primarily invesigae he homogenous I mus be noed ha homogenous deploymen has he advanage of a uniform baery archiecure across all clusers, which has poenial operaional and cos P max c

7 Average Load per Server (%) Time (days) Figure 3: Average normalized load per server measured every 5 minues from 22 Akamai clusers in he US over 25 days showing day/nigh variaions. Power Required (KW) New York Dallas Seale Time (Days) Figure 4: Power demands a a represenaive large-, medium-, and small-sized cluser. 2.4 Akamai Daa Ses For our empirical evaluaion, we use a reposiory of exensive races from Akamai s CDN colleced from 22 major clusers deployed across he US over a period of 25 days. Our races are a represenaive slice of he Web ra c ha Akamai s CDN served in he US during he measuremen period. In aggregae, our races represen daa from 15, 349 servers ha served 950 million requess during he course of our measuremens, including a peak ra c of 800, 000 requess per second. We use wo ypes of races colleced every five minues during he measuremen period. The firs ype of race has cluser load informaion ha consiss of each cluser reporing is average server load every five minues. The cluser load informaion is averaged sysem-wide across all he measured clusers and presened in Figure 3. In addiion, we also collec he geographic locaion of each cluser, including ciy, sae, laiude, and longiude, and he number of servers in ha cluser. We could no measure he power demands of each cluser direcly. However, we use he power funcion of Equaion (1) wih ypical values of P idle and P peak o map he cluser load sequences o cluser power demand sequences. Thus, we obain a power demand sequence ha provides he average power demand per server for each cluser for each 5-minue ime window over our measuremen period. Figure 4 shows he resuling power demand sequence a 3 represenaive clusers wih P idle =100W and P peak =250W. The power demand sequence hus derived serves as he inpu for all our opimizaion algorihms. Our mehodology of synheically deriving power demand from acual load races enables our sudy of servers wih di eren values for P idle, P peak, and PPF. The second ype of race is user load informaion ha includes, for each cluser and each block of user IPs, he number of requess made and he oal byes downloaded by users in ha IP block from ha cluser in he each 5-minue ime window. Furhermore, we colleced he geographic locaion of each block of user IPs. Combining he wo races above enables us o derive how much user load originaed a each block of IPs and which cluser served hem, for each 5-minue ime window. The cluser load informaion is used only in Secion 6 where we consider reassigning users o clusers wih an energy-aware global load balancing algorihm. 3. POWER SUPPLY OPTIMIZATION We formulae wo problems and formally derive algorihms and properies ha are criical o undersanding power supply and cos minimizaion for CDNs. The Power Supply Minimizaion (PSM) problem minimizes he oal power TP supplied o he clusers of a CDN by he colocaion providers, given a se of n clusers and a power demand sequence hp i,i, for i 2C

8 where C is he se of all clusers and 1 apple apple T, where P idle apple p i, apple P peak is he average power demand of a server (in KW) in cluser i a ime slo. The lengh of each ime slo in our daa ses is 5 minues and T is he lengh of our race of roughly 25 days. The oal power supply o be minimized is TP = P i SiPi, where Pi is he power supply per server and S i is he number of servers in cluser i. For each cluser i, P i mus be su cien o saisfy he power demand sequence hp i,i, for 1 apple apple T, wih he help of a provisioned baery wih capaciy of B i per server wih loss rae, maximum charge rae Pc max per server and he maximum discharge rae Pd max per server. Noe ha he maximum discharge rae seldom plays a role in our opimizaion as we assume ha Pd max = P peak. The Power Cos Minimizaion (PCM) problem minimizes he oal cos TC of supplying power o he CDN for a monh ha is inclusive of he cos of procuring he power supply from each colocaion provider plus he amorized cos of procuring baeries ha are deployed wih he servers. As wih he PSM problem, we need o ensure ha here is su cien power eiher from he power supply or he baeries o saisfy he power demand hp i,i a each server in cluser i, for i 2Cwhere C is he P se of all clusers, and 1 apple apple T. The oal cos o be minimized is TC = c p i SiPi + c b L Pi BiSi, where Pi is he power supply (in KWs) provisioned per server in cluser i, B i is he baery capaciy provisioned per server and S i is he number of servers in cluser i, c p is he uni cos of power (in $/ KW/monh), c b is he amorized uni cos of he baery capaciy ($/KWH) and L is he expeced lifeime of he baery (in monhs). Noe ha he PSM and PCM as defined above disallow load movemen beween clusers, i.e., he power demand sequence for each cluser are fixed inpus. Role in CDN Power Provisioning. A menioned earlier, boh PSM and PCM are formulaed as provisioning problems in he sense ha hey can guide he CDN operaor on how o provision he power supplies and baeries for each of he clusers of he CDN. PCM and PSM are solved in an o ine fashion using fuure predicions of power demand. We expec such power demand predicions will be derived from pas race daa, much like he race daa ha we use in our paper. Besides helping us answer fundamenal quesions on he e cacy of baeries in CDN deploymens, we expec ha our work will provide echniques for he CDN operaor o analyze poenial savings for di eren deploymen scenarios, server energy usage rends, fuure power cos regimes, and di eren baery echnologies. The CDN operaor can use our echniques periodically o deermine wha power resources need o be conraced from he colocaion faciliy and wha baery resources need o be provisioned along wih he servers. 3.1 A Characerizaion of he Opimum Power We sudy a simple case of he PSM problem and fully characerize he minimum power supply P op needed per server o saisfy he power demand of a single cluser, given a baery wih capaciy B per server. The characerizaion in his secion builds inuiion ha is criical for undersanding he properies and radeo s in opimizing he power supply. In addiion, i provides faser algorihms for he simpler cases of PSM, while he LP approach oulined in Secion 3.2, albei slower, is beer suied for he more complex forms of PSM. Theorem 3.1. Consider a power demand sequence hp ii, 1 apple i apple T, for a cluser wih S servers, baery capaciy B per server, loss facor and maximum charge rae Pc max per server. Suppose ha he baery is full a he sar of firs ime slo. The minimum required power supply o accomodae he power demand sequence is P op =max 1appleiapplejappleT P (i, j), where P P (i, j) =min{p : (1 ) min{(p p k ),Pc max } k2u(i,j,p ) +B P k2l(i,j,p ) (p k P )}, where U(i, j, P )={k : i apple k apple jandp p k } and L(i, j, P )={k : i apple k apple jandp<p k }. Proof. Firs, we show ha P op P (i, j), for any ime inerval (i, j), 1 apple i apple j apple T. Le (i, j) be any ime inerval and P be he provisioned power. The baery (poenially) ges charged a sep k 2 U(i, j, P ) and mus ge discharged a each sep k 2 L(i, j, P ) so as o no exceed he supplied power P (See Figure 5). A he beginning of ime i, he maximum energy available in he baery is clearly B. The maximum energy enering he baery during ime sep k 2 U(i, j, P )isamos (1 )min{(p p k ),Pc max }, since Pc max is he maximum rae a which he baery can be charged and is he loss facor. Likewise, for any k 2 L(i, j, P ), he rae a which he baery mus be discharged is min{(p k P ),Pd max } = p k P, since Pd max = P peak p k. Now, if a power supply of P and a baery capaciy of B saisfies he power demand sequence from ime i o j, he oal energy already presen or flowing ino he baery in ha ime inerval is a leas he oal energy flowing ou (see Figure 5). Tha is, X X (1 ) min{(p p k ),Pc max } + B (p k P ). (2) k2u(i,j,p ) k2l(i,j,p ) Since P (i, j) is smalles value of P ha saisfies Equaion 2, P op P (i, j). Now, since he above is rue for any i, j,

9 Figure 5: The opimum supplied power P op is he smalles value such ha for any ime inerval he sum of he baery capaciy B and he oal possible charging in ha inerval ( imes he shaded area beween he wo doed lines) is a leas he oal discharging required for ha inerval (area above he doed line), where Pc max is he maximum charge rae of he baery. 1 apple i<japple T, P op max P (i, j). (3) 1applei<jappleT Now, we show ha provisioning a power supply P equal o he RHS of Equaion 3 is su cien o saisfy he power demand of he cluser wih baery of B per server. This shows ha P op equals he RHS of Equaion 3, since he RHS is a lower bound on required power supply. Le P equal he RHS of Equaion 3 and le charging and discharging decisions be made in a simple manner, i.e., if P>p k hen he baery is charged unil capaciy is reached and when P<p k he baery is discharged a a rae of p k P. I su ces o show ha he power demand sequence hp ii, 1apple i apple T, can be saisfied wih he power supply P and baery capaciy B. For conradicion, suppose ha his were no rue. Then, here mus be a ime j where oo much power needs o drawn from he baery causing i o underflow. Now, le i be he larges ime i<jsuch ha he baery is full a he sar of ha ime sep. Such an i always exiss, since baery is full a he sar of he firs ime slo.we are guaraneed ha here is no baery overflow in he ime inerval (i, j) since he baery charge level is always less han B a any ime i<kapple j. Thus, we know ha any charging opporuniy is fully uilized during his ime inerval wihou a baery P overflow. Therefore, he oal charge enering he baery in ime inerval (i, j) is(1 ) min{(p p k ),Pc max }. If k2u(i,j,p ) here is a baery underflow a ime j, hen we can conclude ha amoun of charge in he baery a ime i plus he amoun of energy ha flowed ino he baery in ime inerval (i, j) is smaller han he amoun of energy ha flowed ou of he baery in same ime inerval, i.e., X (1 ) min{(p p k ),Pc max } + B< X (p k P ). k2u(i,j,p ) k2l(i,j,p ) Since we chose P o equal he RHS of Equaion 3, we know ha P obeys Equaion 2. Conradicion. The above characerizaion enables a fas algorihm o compue he opimum power supply. Corollary 3.2. Given he power demand sequence hp ii, for 1 apple i apple T and a baery wih capaciy B per server, he opimum power supply required P op can be compued in ime O(T 2 log(p peak )). Proof. We perform binary search o find he smalles value of P,0apple P apple P peak, ha saisfies Equaion 2 for all ime inervals (i, j), 1 apple i<japple T.ForagivenP, checking Equaion 2 for all ime inervals (i, j) can be performed in O(T 2 )ime using dynamic programming. We define DIFF P (i, j) o be he di erence beween he LHS and RHS of Equaion 2. I is easy o see ha DIFF P (i, j) can be compued from DIFF P (i, j 1) in consan ime by adding in he erms ha correspond o ime j. Thus, for a given P, we can compue DIFF P (i, j) for all values 1 apple i apple j apple T in O(n 2 ) ime. We can ascerain wheher or no P saisfies he power demand by examining wheher all values of DIFF P (i, j) are non-negaive in O(T 2 )ime. We perform binary search for he smalles value P using he above process o decide if a value of P is a feasible one. Since we ry O(log P peak ) values for P each aking O(T 2 ) ime, he oal ime is O(T 2 log(p peak )). 3.2 An LP approach o Power Opimizaion We provide a more general linear programming (LP) framework ha is capable of solving more complex varians of he PSM and PCM problem. While our prior characerizaion solves for he opimum power supply for each cluser individually,

10 one can equivalenly solve for all clusers simulaneously in he LP formulaion as follows. Once saed as an LP, we use he CPLEX package [4] o solve he LP. For our races, our formulaion involves roughly 1.7 million variables and akes beween 30 o 60 minues solve, which is accepable since we envision our opimizaions o be performed in an o ine fashion on hisorical or prediced race daa. The objecive funcion for he PSM problem is simply he oal supplied power ha needs be minimized, i.e., Minimize TC = X i2c S ip i, (4) where C is he se of all clusers, P i is he power supply required per server in cluser i, and S i is he number of servers in cluser i. The power demand p i, of a server in cluser i a ime is saisfied parly from he power supply and parly by discharging he baery: For all i 2C, 1 apple apple T, p i, = SupplyToServer i, + BaToServer i,. (5) The power provided from he power supply o he server is non-negaive: For all i 2C, 1 apple apple T, SupplyToServer i, 0. (6) Power from he baery o he server is non-negaive and a mos he maximum discharge rae: For all i 2C, 1 apple apple T, 0 apple BaToServer i, apple P max d. (7) The provisioned power supply P i a cluser i is parly used o power he server and parly used o charge he baery: For all i 2C, 1 apple apple T, SuppToServer i, + SuppToBa i, apple P i. (8) The power supplied o he baery for charging is non-negaive and a mos he maximum charge rae: For all i 2C, 1 apple apple T, 0 apple SuppToBa i, apple P max c. (9) Energy sored in he baery Energy i, a cluser i a he end of ime obeys he following: For all i 2C, 1 apple apple T, Energy i, = Energy i, 1 + SuppToBa i,(1 ) BaToServer i,, (10) where he second erm above represens he charging of he baery in ime sep, he hird erm represens he discharging of he baery, is he loss rae, and is he lengh of a ime slo (which is 5 minues in our experimens). Energy sored in he baery a cluser i is no more han he baery capaciy B i and no less han zero (no baery underflow): For all i 2C, 1 apple apple T, 0 apple Energy i, apple B i. (11) The iniial and final condiion of he baery can be specified in several ways. One could posi ha he baery is full a ime zero, i.e, Energy i,0 = B i, for all i 2C. Alernaely, we can posi ha energy in he sar of he ime period equals he energy a he end of he ime period so ha here is no ne charge or discharge, i.e., Energy i,0 = Energy i,t, for all i 2C. Given a power demand sequence hp i,i, i 2Cand 1 apple apple T,aseofS i servers in cluser i, i 2C, and baery deploymens B i, i 2C, he above consrains from Equaions 5 o 11 and he objecive funcion of Equaion 4 can be inpu ino an LP solver o produce he opimum value of he oal provisioned power TC. While he algorihm in Secion 3.1 can solve he basic TSM problem above, he LP framework is paricularly useful for he following varians ha we sudy. Bounding he Baery Cycle Rae. When he lifeime of he baery is a concern, we need o bound he number of imes he baery can discharge per uni ime. We can inroduce an addiional consrain ha bounds he cycle rae o be below a prescribed value of MaxCycleRae: For all i 2C, P 1appleappleT BaToServeri, apple MaxCycleRae. (12) B it Power Cos Minimizaion (PCM). We simply replace he objecive funcion of Equaion 4 by X Minimize TC = c p S ip i + c X b B is i, (13) L i where P i is he power supply (in KWs) provisioned per server in cluser i, B i is he baery capaciy provisioned per server and S i is he number of servers in cluser i, c p is he uni power price (in $/KW/monh), c b is he uni baery price i

11 ($/KWH/monh), and L is he expeced lifeime of he baery (in monhs). The inpus o PCM is he power demand sequence hp i,i, i 2Cand 1 apple apple T, a se of server deploymens S i servers, i 2C, and coss c p, c b, and L. The amoun of power P i and baery B i are variables ha are compued by he LP so as o minimize he new cos objecive funcion. Heerogenous versus Homogenous Baeries. The above formulaion models he more general case of heerogenous baeries. However, unless specified oherwise, we assume ha he baery per server is uniform hroughou he CDN, i.e., for all i 2 C, we replace B i wih a single variable B hroughou he LP. Load Balancing. To model global load balancing, we compue he oal load across all clusers ha ener he CDN denoed by hl i, 1apple apple T, where L is he incoming oal load a ime and can be derived from our races. The global load balancer porions he incoming load across clusers o such ha cluser i receives an average load of l i, per server. This can be capured by he following consrain: for all 1 apple apple T, P S il i, = L,. The corresponding power demand sequence i2c p i, = power(l i,), for all i 2C, 1 apple apple T. We hen solve he LP wih hese addiional consrains added o he exising consrains of Equaions 5 o POWER SAVINGS ANALYSIS The power savings of a CDN is defined o be he percenage reducion in he oal supplied power TP due o he use of baeries. Thus, power savings equal 100 Min TP wihou baeries Min TP wih baeries. (14) Min TP wihou baeries Noe ha in our definiion of power savings above, we compare he minimum oal power supply (TP) achievable wih and wihou baeries. In realiy, a CDN will likely be over-provisioned by he safey facor above he minimum power supply required o saisfy is expeced maximum power demand. The safey facor provides some headroom for unexpeced spikes in load or power demand; recall Figure 1. I is reasonable o assume ha he safey facor is he same wih or wihou baeries and hence does no a ec he power savings. We now ascerain he power savings achievable by a CDN as a funcion of he (i) baery capaciy, (ii) baery cycle rae, (iii) discharge/charge raio, (iv) power proporionaliy facor, and (v) baery deploymen model. 4.1 Baery Capaciy Firs, we prove ha he power savings is a concave funcion of he deployed baery capaciy for which law of diminishing reurns apply, i.e., an iniial increase in baery capaciy can significanly increase he power savings, while a furher marginal increase in baery size may lead only o a smaller marginal increase. Theorem 4.1. For any power demand sequence hp i, 1 apple apple T for a specific cluser, he opimum power supply P op for ha cluser is a non-increasing convex funcion of he baery capaciy B. Thus, oal power savings across he enire CDN is anon-decreasingconcavefuncionofcapaciyb. Proof. For wo baery capaciies B and B 0 provisioned in a given cluser, le P and P 0 be he opimum power supply values for ha baery capaciy respecively. I su ces o show ha a supplied power value of P 00 =(P + P 0 )/2 saisfies power demand sequence wih a baery of capaciy B 00 =(B + B 0 )/2. One can view a baery wih capaciy B 00 and a charge rae of Pc max as wo baeries wih capaciy B/2 and B 0 /2, each wih a charge rae of Pc max /2. Since P (resp. P 0 ) is feasible for serving he power sequence hp i wih a baery of capaciy B (resp. B 0 ), P/2 (resp. P 0 /2) is feasible for serving he power sequence hp /2i wih a baery of capaciy B/2 (resp., B 0 /2). Tha is, one can ake a feasible schedule for P (resp., P 0 )wih baery capaciy B (resp., B 0 ) and obain anoher feasible schedule by simply dividing he power supply, baery capaciy, baery charge rae, and power demand by a facor of wo. Puing ogeher he wo feasible schedules consruced in his fashion, we obain a feasible schedule wih power supply P 00 =(P + P 0 )/2, and baery capaciy B 00 =(B + B 0 )/2 for he original power demand sequence hp i, implying ha P 00 is a leas he opimum power supply value for baery capaciy B 00. Thus, P op for any cluser is a non-increasing convex funcion of baery capaciy, implying from Equaion 14 ha oal power savings is a non-decreasing concave funcion. Experimenal Resuls. We simulae a ypical deploymen of homogenous baeries wih discharge/charge raio r = 5, and servers wih PPF of 0.6. Figure 6 shows he opimal power savings for baeries of various capaciies for power demand derived from our Akamai s CDN races described in Secion 2.4 compued using he algorihms described in Secion 3. As suggesed by Theorem 4.1, he power savings vs. baery capaciy curve is concave wih diminishing gains as baery capaciy increases. We visually discern hree disinc regions in his curve: (i) I where relaively high improvemens resul from addiional baery capaciy - his spans ill he baery capaciy of abou 5 minues where he power savings are abou 7%, (ii)

12 Figure 6: Power savings increase rapidly ill abou a capaciy of 5 minues, grow more slowly ill abou 40 minues before reaching a poin of diminishing reurn. Figure 7: Power savings when he cycle rae is bounded. Mos savings can be realized wih a small cycle rae. II where he improvemen in power savings slows down - his spans ill he baery capaciy of abou 40 minues where he power savings are abou 14%, and (iii) III, which spans beyond baery capaciy of abou 40 minues and where addiional capaciy yields negligible or no addiional savings. Observaion 4.2. Even small baery capaciies of 1-5 minues can o er appreciable power savings of abou 4-7%. This bodes well for heir immediae use for power savings in CDNs since hese sizes are comparable o hose in currenly deployed sysems - no only in he form of radiionally popular cenralized UPS unis [15] bu also in rack/server-level baeries in emerging designs [22]. Observaion 4.3. The higher power savings of up o 14% are yielded by a baery capaciy of 40 minues. Volume consrains migh prohibi packing several ens of minues of baery capaciy a he server-level UPS unis [25]. This suggess ha, in order o realize he full exen of power savings possible via baeries, our CDN may wan o choose a design ha provides is clusers wih access o rack-level baeries ha can pack adequae capaciy. 4.2 Baery Lifeime Since a baery can only las a bounded number of cycles in is lifeime (Figure 2), a high cycle rae degrades is he lifeime. We ask how much of he power savings is obainable wih a bound on he cycle rae (see Equaion 12) in comparison wih he power saving achievable wih no bound on he cycle rae a all. Surprisingly, as shown in Figure 7, a small cycle rae of 0.33 is su cien o obain mos of he power savings. Observaion 4.4. Mos of he power savings obainable wih an unbounded cycle rae are achievable by charging he baery o full capaciy once every hree days on average (cycle rae of 0.33 per day), i.e., 86%, 88.3%, and 97% of he power savings are obainable for a 60-minue, 30-minue, and 5-minue baery, respecively wih he small cycle rae. This is compaible wih he baery lasing abou 5 years per he lifeime char.

13 '#!)$ "# $ '($.$.$, & $ - & $ * & $ + & $! "# $! %& $ Figure 8: Illusraion of he power demand sequence and hreshold policies for servers wih low and high PPF. 4.3 Discharge/Charge Raio While we use a value of discharge/charge raio (r) of 5 ha is ypical for lead-acid baeries, in his secion we explore if smaller values of r can impac savings. Noice ha, given ha Pd max = P peak, his amouns o exploring di eren values of Pc max allowed by he baery echnology being used. Cerain baery echnologies o er beer r han lead-acid. Two prominen examples of such echnologies include ulra-capaciors and flywheels, boh of which o er r close o 1 [25]. Theoreically, for a given baery size, a baery echnology ha allows faser charging (han anoher echnology) can allow more energy o be accumulaed during a given charging period, hus allowing a possibly higher reducion in supplied power laer. However, repeaing our experimens from Secion 4.1 wih Pc max higher han 50W, we do no find significan addiional benefi han wha we obained for r = 5. The reason is ha, as we saw in Secion 3.1, he power available o charge he baery a sep k is min{(p p k ),Pc max }, where P is supplied power a a cluser and p k is power demand a sep k. As noed earlier, he CDN s servers had load of a mos 40% mos of he ime resuling in a ypical power demand p k of a mos 160W, a ypical value of P p k was a leas P idle p k =60W. Thus, any Pc max larger han 60W made no di erence since firs erm of he minimizaion would be smaller. Observaion 4.5. The discharge/charge raio r o ered by lead-acid baeries is adequae for achieving he all of he realizable power savings in our CDN. 4.4 Power Proporionaliy We firs provide a heoreical basis for why we can expec larger savings due o baeries as he servers become more power proporional. Nex, we explore his issue empirically. Theorem 4.6. Given a server-level load sequence hl i, 1 apple apple T. Suppose ha he baery is full a he sar of he firs ime slo. Then he opimal power savings realized for a server wih peak power P peak,baerysizeb, andppfoff hi is greaer han or equal o ha realized for a server wih he same peak power and baery size bu a lower PPF f lo (i.e., f hi >f lo ). Proof. Le us denoe by hp hi i and hp lo i he power demand sequences induced on servers wih PPFs f hi and f lo, respecively, by he given load demand hl i. Since f hi >f lo, using (1) we have p lo p hi, 8, wih equaliy occurring when l = 1. Le us denoe by he di erence p lo p hi. We firs observe ha l i >l j ) i < j. Consequenly, he minimum value of he di erence beween power demands (call i ) occurs during a ime slo (call i m) where he load akes i maximum value. Tha is, =min = p lo m p hi m, where m = arg max l. Le Pop lo denoe he minimum required power supply for he sequence hp lo i for he server wih PPF f lo. Noe ha his is fully characerized by Theorem 3.1. Furhermore, as shown in Theorem 3.1, he hreshold based policy ha discharges he baery when Pop lo <p lo a a rae of p lo ha he peak power reducion achieved by his policy is equal o p lo m P lo op and charges unil capaciy is reached whenever P lo op P lo op. We now show ha a similar hreshold based policy ha uses he hreshold P 0 = P lo op p lo on he sequence hp hi is opimal. Noe i for he server wih PPF f hi is feasible. This means ha a peak power reducion of a leas p hi m P 0 =(p lo m ) (Pop lo )=p lo m Pop lo can be achieved for he load sequence hl i under he server wih higher PPF. Since he minimum oal power supply (TP) achievable wihou baeries under he server wih higher PPF (i.e., p hi m ) is smaller han ha under he server wih lower PPF (i.e., p lo m ), i follows from (14) ha he opimal power savings under he server wih higher PPF exceed hose under he server wih lower PPF. Le E hi denoe he oal energy sored in he baery a ime slo when he hreshold policy described above is used on he power demand sequence hp hi i for he server wih PPF f hi. Define E lo in a similar way for he server wih PPF f lo and power demand sequence hp lo i ha uses he hreshold Pop. lo Leu i denoe he sequence of ime slos a which he power

14 '!"!"#$"%&'(")!*+"#),'-.%(/) &#" &!" %#" $"+,-" %!" #"+,-" $!"+,-" $#" &!"+,-" $!" )!"+,-" #".!"+,-"!"!()"!(*" $"!*+"#)!#*0*#&.*%'1.&2)3'$&*#) Figure 9: Power savings improve as PPF increases. demand sequence hp hi i inersecs he hreshold P 0 such ha p hi is increasing a = u i.likewise,led i denoe he sequence of ime slos a which he sequence hp hi i inersecs he hreshold P 0 such ha p hi is decreasing a = d i. For simpliciy, we assume ha p lo 0 apple Pop lo so ha u i apple d i for all i. The opposie case where d i u i for all i can be reaed similarly. For every inerval [u i,d i] of he sequence hp hi i in which p hi P 0, here is a corresponding inerval [x i,y i] of he sequence hp lo i in which p lo Pop lo such ha x i apple u i and y i d i. This is shown in Lemma 4.7. We now show ha Eu hi i Ex lo i and Ed hi i Ey lo i for all ime slos u i,d i,x i,y i. This, combined wih he fac ha he oal energy required from he baery in he inerval [u i,d i] for he sequence hp hi i (equal o he area under he red curve and green doed line in he inerval [u i,d i] in Fig. 8) is smaller han he oal energy required from he baery in he inerval [x i,y i] for he sequence hp lo i (equal o he area under he blue curve and black doed line in he inerval [x i,y i] in Fig. 8) shows ha he hreshold policy is feasible. To show ha Eu hi i Ex lo i for all ime slos u i,x i, we use inducion. Noe ha his rivially holds for u 1,x 1. This is because Eu hi 1 = B Ex lo 1 since here has been no discharge from he baery ye. Nex, Ed hi 1 Ey lo 1 since he oal amoun discharged over he inerval [u 1,d 1] under he sequence hp hi i is smaller han he oal amoun discharged over he inerval [x 1,y 1] under he sequence hp lo i. Now suppose hese relaions hold for some i>1. We show ha hese holds for i + 1 as well. Consider he inerval [d i,u i+1] under he sequence hp hi i. Using Lemma 4.7, we have ha he lengh of his inerval exceeds ha of he inerval [y i,x i+1] under he sequence hp lo i. This means ha he oal amoun by which he baery ges recharged is a leas as large for he higher PPF case. This means ha E ui+1 E xi+1. Finally, E di+1 E yi+1 using he same argumen as before. Lemma 4.7. For every inerval [u i,d i] of he sequence hp hi i in which p hi he sequence hp lo i in which p lo Pop lo such ha x i apple u i and y i d i. Proof. This can be seen as follows. Suppose p hi u i = P 0 = P lo op we ge p lo u i p hi u i + = Pop lo and p lo canno exceed u i. Using similar argumens, i can shown ha y i d i. and p hi P 0,hereisacorrespondinginerval[x i,y i] of is increasing a = u i. Since p lo is increasing a = u i (by lineariy of (1)). Thus, he poin x i a which p lo p hi + 8, crossed Pop lo Figure 9 shows how power savings vary wih PPF for di eren baery sizes. One can see ha for any baery capaciy, power savings increase wih an increase in PPF. The inuiive reason is ha as PPF increases, he consan par of he power demand as represened by P idle shrinks, increasing he relaive variabiliy of he power demands. The increased variabiliy increases he opporuniies for a baery o decrease he peaks and hence decrease he supplied power. For a small baery of capaciy of 5 minues (in range I), he power savings across he enire CDN ranged from 7% for a PPF of 0.6, o 12.57% for a PPF of 1.0, wih a value of 10.5% for a PPF 0.8. Likewise, for a medium-sized baery of capaciy 30 minues (in range II), he power savings ranged from 12.4% o 26%, wih a value of 17.9% for PPF of 0.8. And, for a large baery of capaciy 60 minues, he power savings ranged from 13.9% o 35.3%, wih a value of 22.3% for PPF of 0.8. A perhaps less inuiive poin o noe is ha as server PPFs increase, here are greaer incenives o making large baeries even larger. As an example, consider how he power savings improvemen going from he baery capaciy of 30 minues o ha of 60 minues iself improves wih PPF: power savings increase by 1.5% for PPF=0.6, 4.4% for PPF=0.8, and 9.3% for PPF=1. Observaion 4.8. Improving he power proporionaliy of servers improves he power savings from 7-14% wih curren servers wih PPF=0.6, o a range of 10.5%-22% wih nex-generaion servers wih PPF=0.8, o he range % in he

15 Figure 10: Heerogenous baeries provide lile addiional power savings compared o homogenous ones. hypoheical bes-case of perfec power proporionaliy PPF=1. In addiion o increased power savings, a greaer PPF also increases he range of baery sizes ha can o er such savings. 4.5 Heerogeneous Baery Provisioning As Figure 10 shows, for he same amoun of oal baery capaciy, disribuing he baeries o clusers in a heerogenous manner wih some clusers geing (poenially) more baery capaciy han ohers produces lile addiional savings over providing each server in each cluser he same amoun of baery. The inuiive reason is ha power demands across all clusers (see Figure 4) had similar enough saisical characerisics in erms of day-nigh and weekly variaions ha he baery requiremens across clusers were similar. Observaion 4.9. Heerogenous baeries do no provide much addiional savings over homogenous ones. The addiional savings increase wih baery capaciy wih an addiional savings of 0.2% for a 5-minue baery, 0.2% for a 30-minue baery, and 0.75% for a 60-minue baery. Given he operaional advanages, a homogenous deploymen of baeries su ces. 5. COST SAVINGS ANALYSIS We analyze he cos savings (in $s) our CDN can obain by employing baeries. This involves characerizing he rade-o beween (i) he OpEx reducion o ered by an invesmen in baeries in he form of power savings, and (ii) he CapEx increase for procuring he baeries. The cos saving for a CDN equals he reducion in oal cos (TC) due o baeries, i.e., Min TC wihou baeries Min TC wih baeries 100, Min TC wihou baeries where he Min TC wih baeries is derived by minimizing he expression in Equaion 13. Likewise, Min TC wihou baeries is he leas power cos achievable wihou baeries, i.e., smalles cos wih baery capaciy of zero. Noe ha baery capaciy can eiher be fixed or be a variable in he minimizaion where he bes baery size ha minimizes TC is picked by our opimizaion. I is easy o see ha cos saving depends on he uni baery price (c b ), he baery lifeime (L), server power proporionaliy (PPF), and he uni power price (c p). As oulined in Secions 2.1 and 2.3, we now analyze cos savings for a discree se of values for each parameer above ha we believe characerizes he possible curren and fuure scenarios. 5.1 Baery Characerisics In Figure 11, we derive he cos savings assuming a low power price of $150 per KW per monh for hree di eren baery scenarios oulined in Secion 2.1: ypical (c b =$100/kWH, L=5 yr), conservaive (c b =$300/kWH, L=3 yr), and wors case (c b =$500/kWH, L=1 yr). Excep wih he overly pessimisic wors-case, our cos savings resuls are encouraging: wih he more realisic ypical and conservaive cases, we find ha baeries are capable of urning a posiive reurn-on-invesmen (ROI). We find cos savings of 10% wih a 40-minue baery in he conservaive case and 13.9% wih a 60-minue baery in he ypical case. Even for our wors-case, we find ha up o 5.5% cos savings can be achieved wih baery size of 5 minues. Beyond he size of 60 minues, baery invesmen acually becomes financially unsound for all cases, causing negaive ROIs in he wors-case.

16 15 10 Cos Savings (%) Bes Case 10 Average Case Wors Case Baery Size (Minues) Figure 11: The cos savings o ered by di eren baery sizes for our hree baery cos/lifeime models, namely ypical, conservaive, and wors-case. Observaion 5.1. Even for low power prices currenly seen in he US and a low PPF of 0.6, an invesmen in lead-acid baeries can allow cos savings for our CDN of abou 13.9% under ypical assumpions for baery cos and 10% under more conservaive assumpions. Furhermore, he opimal cos savings are achieved wih larger baery capaciies when he amorized baery cos (c b /L) issmaller. 5.2 Power Proporionaliy We now presen he opimal cos savings for di eren PPFs. The cos savings improve wih PPF for all hree baery Figure 12: The cos savings improve as servers become more power proporional. We show he resuls for a 30 minue baery for our hree di eren baery scenarios: ypical, conservaive and wors-case and ypical power price of c p =$150perKW per monh. scenarios: ypical, conservaive and wors-case. Noe ha he improvemen in cos savings is greaer in absolue erms for he ypical baery scenario as opposed o he wors-case baery scenario, since he amorized baery cos is cheaper in he firs scenario han in he second. Observaion 5.2. Baeries provide significanly higher cos benefis as servers become more power proporional. For a nex-generaion sysem wih PPF=0.8, he cos savings for a ypical baery are 21.4% and can rise even furher o 33% as he PPF nears Power Supply Prices In Figure 13, we plo he cos savings for a ypical baery as funcion of he supplied power coss (c p) in he range $100-$500/KWH/monh. The cos savings are 13.26% for he low power prices currenly in he US, which rises o 13.57% for moderae prices ypical of Europe, and becomes even larger a 13.69% when he power prices are in he high range. A reason for he increased cos savings ha you observe in he figure is ha he amorized cos of he baery becomes less significan in relaion o higher power coss, as c p increases. Furher, he amoun of addiional power savings needed o pay for more baery capaciy is lower for larger c p. Observaion 5.3. As he uni power price increases from $100/KW/monh o $500/KW/monh, he cos savings increased

17 Figure 13: Cos savings increase wih power price. Figure 14: Power-aware GLB wih baeries provides lile addiional benefi over using baeries alone. from 12.85% o 13.69%. While he increase in cos savings is modes and less han 1%, he cos savings is much greaer in absolue erms as he cos iself is abou five imes higher. 6. GLOBAL LOAD BALANCING We are ineresed in undersanding if i helps o make global load balancing (GLB) power aware, so ha i can move ra c beween clusers o shave o peaks and reduce he oal power supply TC. One can view GLB as a mechanism complemenary o baeries in ha one could sudy he benefi of eiher mechanism alone, or boh ogeher. In an acual CDN, GLB mus opimize wo key objecives oher han power. Firs, GLB mus ensure ha each user receives good performance by being served by a nearby cluser in he nework sense [20]. Second, GLB mus minimize bandwidh coss 7 paid by he CDN o each nework provider. The GLB of a modern CDN is already opimized for performance [20] and bandwidh coss [7]. Thus far, by assuming ha he load and power demands canno be moved across clusers we ensured ha he opimized performance and bandwidh coss refleced in he Akamai races remain unalered while we minimize power supply and power coss wih baeries. In order o undersand he poenial impac of GLB, we sudy he bes case scenario where we ignore performance and bandwidh coss and assume ha users and heir load can be moved o any cluser as long as he capaciy consrains of he cluser are me. As described in Secion 3.2, he bes case scenario for GLB can be incorporaed ino he power supply minimizaion problem by adding addiional variables and consrains. In Figure 14, we show he power savings obained wih and wihou GLB. Using GLB alone wihou any baeries produced only a small power savings of 1.78%. The addiional power savings due o GLB narrowed as we increased he baery capaciy: a 5-minue baery produced an addiional savings of 1.6% due o GLB, a 30-minue baery an addiional savings of 0.8%, and a 60-minue baery an addiional savings of only 0.3%. The small addiional savings due o GLB leads us o conclude ha a power aware GLB is no beneficial enough o implemen from a power savings sandpoin, especially in comparison wih baeries ha produce much larger gains. Incorporaing performance 7 The bandwidh cos of a cluser is compued by averaging he byes sen by he CDN s servers over 5-minue ime inervals for each monh. The cos is proporional o eiher he 95 h percenile or he average of hese 5-minue averages.

18 and bandwidh cos ha is a criical componen of any real-world global load balancer would resric he iner-cluser load movemens resuling in even less savings han he bes case evaluaed above. The inuiive reason for he low addiional power savings due o GLB is because he power demands across clusers of he CDN peak and ebb a around he same ime following Web access paerns of he users across he counry (see Figure 4). The power demands would be even more similar across clusers if GLB were o incorporae performance consrains since he available choices of clusers for a user would have o be nearby in he same geographical area wih even more similariy in usage paerns. Thus, GLB is unable o find significan gains by shaving peaks in he power demand of one cluser and moving i ino a valley in he power demand of anoher cluser. Noe also ha such opporuniies are made rare by he requiremen ha he CDN s load mus be served in real-ime, and canno be deferred o a laer ime when he power demands are low. Observaion 6.1. The maximum addiional benefi of a power-aware GLB is small and a mos 1.78%. Since he addiional power savings are likely even smaller if performance and bandwidh coss are incorporaed, we conclude ha GLB is no an imporan mechanism for power supply or power cos savings. 7. RELATED WORK Baeries have a long hisory of use in disaser recovery (i.e., for handling failures of he power uiliy) and in siuaions such as baery-powered sensor or mobile environmens [13] where power availabiliy is inermien. Recenly, baeries have been sudied for various forms of demand response in daa ceners, and we consider his body of work he closes o ours. These include using baeries o reduce he OpEx coss when he uiliy prices are ime-varying [23] or when he uiliy bill depends on he peak power usage [15]. These also include using baeries o reduce he CapEx coss for provisioning daa cener power infrasrucure [17, 25]. While benefiing direcly from he characerizaion of baery properies in hese papers, our work di ers significanly from hem. Firs, our work is in he conex of a CDN - a highly disribued sysem wih muliple deploymens across he globe - as opposed o a single daa cener. This di erence becomes especially sark when we consider GLB, which is unique o a CDN and does no apply o prior work on single daa ceners. Second, we work wih a very di eren ari model - he supplied power ari model wih long-erm conracs ha is sandard in he CDN indusry - which presens us wih a very di eren opimizaion crierion han hose arising in prior work. Third, a CDN s load canno be deferred and mus be served in real-ime. This resuls in di eren kinds of consrains in our opimizaion han for daa ceners where some of he workloads are delay-oleran. Finally, o he bes of our knowledge, our work is he firs o evaluae he e cacy of baeries in fuurisic seings where servers migh improve in heir power proporionaliy and elecriciy prices migh be higher han oday. In a seminal work [21], he auhors consider an Inerne-scale disribued sysem wih muliple deploymens like a CDN. They show ha if coss are based on elecriciy usage and if he power prices vary in real-ime, global load balancing decisions can be made such ha users are roued o locaions wih he cheapes power wihou significanly impacing user performance or bandwidh coss. Our work has a di eren focus as we use baeries, a power pricing model ha is based no on usage bu on how much power is provisioned, and power prices are fixed for he lengh of conrac ha is several monhs o a year. The disribued baery archiecure ha places baeries a he rack- or server-level ha we consider in our work is deployed and used by Facebook, Google [22] and Microsof [6]. While heir primary moivaion is o implemen a disribued UPS sysem ha coss less and wases less energy han a cenralized UPS, we use baeries for he di eren goal of power supply and power cos minimizaion. Finally, here is relevan algorihmic work on minimizing he peak draw from an uiliy using baeries [8]. In fac, Theorem 3.1 can be viewed as a generalizaion of heir work o a richer se of baery parameers. However, our work is disinguished from heirs in ha we focus on power and cos savings in a CDN conex, use a supplied power insead of he peak usage model, and focus on a muli-cluser disribued service. I is also worh noing ha recenly baeries have been considered for minimizing he peak power draw in residenial seings [9, 24]. 8. CONCLUSIONS Our work proposed and provided srong evidence ha a CDN could uilize baeries o significanly reduce boh he oal supplied power and he oal power coss. This work is imporan as CDNs are growing rapidly and are already he plaform for a significan fracion of Web, applicaion, and media ra c world-wide. We showed ha baeries can provide significan savings wih curren server and baery echnologies, and ha hese savings increased rapidly as servers became more power-proporional. Furhermore, when he savings were viewed in absolue erms, baeries yielded significanly larger cos savings as power prices increased. Since we are likely o see increases in boh power prices and more power-proporional

19 servers, and given he increasing focus on OpEx reducion in CDNs, we believe ha our work will have a significan impac in esablishing baeries as a key elemen in fuure CDN archiecure. Besides our experimenal work, our heoreical and algorihmic framework for formulaing and solving power supply minimizaion and power cos minimizaion is an imporan conribuion. In fac, we believe ha our algorihms will form he basis for ools ha CDNs could use o opimally provision power supplies and baeries, given a predicion of fuure load on he nework. I is imporan o noe ha we view our work as he firs sep in exploring he feasibiliy of baeries in a CDN by providing a high-level analysis of wha migh be possible. The favorable resuls of our analysis provide moivaion for a more deailed exploraion ha considers he hardware design, placemen, spaial requiremens, and any addiional operaional coss associaed wih deploying baeries. Finally, we observe ha our resuls exend beyond CDNs o oher Inerne-scale disribued neworks ha similarly deploy in co-locaion faciliies, procuring power using he ypical supplied power ari model. 9. ACKNOWLEDGEMENTS This work was suppored, in par, by NSF CAREER award REFERENCES [1] Akamai Technologies. hp:// [2] Analysis: Colocaion pricing rends. hp:// [3] Baery FAQ. hp:// [4] Cplex - ibm. hp://www-01.ibm.com/sofware/inegraion/opimizaion/cplex-opimizer/. [5] Lead-acid baery coss. hp://phoovolaics.sandia.gov/pubs_2010/pv\%20websie\%20publicaions\%20folder_09/hanley_pvsc09\%5b1\%5d.pdf. [6] Microsof reveals is specialy servers racks. hp:// [7] M. Adler, R. K. Siaraman, and H. Venkaaramani. Algorihms for opimizing he bandwidh cos of conen delivery. Compuer Neworks, 55(18): , [8] A. Bar-Noy, M. Johnson, and O. Liu. Peak shaving hrough resource bu ering. Approximaion and Online Algorihms, pages , [9] S. Barker, A. Mishra, D. Irwin, P. Shenoy, and J. Albrech. Smarcap: Flaening peak elecriciy demand in smar homes. In Pervasive Compuing and Communicaions (PerCom), 2012 IEEE Inernaional Conference on, pages IEEE, [10] L. Barroso and U. Holzle. The case for energy-proporional compuing. Compuer, 40(12):33 37, [11] C. Belady. In he daa cener, power and cooling coss more han he IT equipmen i suppors. Elecronics cooling, 13(1):24,2007. [12] A. Beloglazov, R. Buyya, Y. Lee, and A. Zomaya. A axonomy and survey of energy-e cien daa ceners and cloud compuing sysems. Arxiv preprin arxiv: , [13] Q. Cao, D. Fesehaye, N. Pham, Y. Sarwar, and T. Abdelzaher. Virual baery: An energy reserve absracion for embedded sensor neworks. In Real-Time Sysems Symposium, 2008, pages IEEE, [14] J. Dilley, B. M. Maggs, J. Parikh, H. Prokop, R. K. Siaraman, and W. E. Weihl. Globally disribued conen delivery. IEEE Inerne Compuing, 6(5):50 58,2002. [15] S. Govindan, A. Sivasubramaniam, and B. Urgaonkar. Benefis and limiaions of apping ino sored energy for daaceners. In ISCA, pages , Jun [16] S. Govindan, D. Wang, L. Chen, A. Sivasubramaniam, and B. Urgaonkar. Towards realizing a low cos and highly available daacener power infrasrucure. In Hopower, Oc [17] S. Govindan, D. Wang, A. Sivasubramaniam, and B. Urgaonkar. Leveraging sored energy for handling power emergencies in aggressively provisioned daaceners. In ACM ASPLOS, pages 75 86, Mar [18] V. Mahew, R. K. Siaraman, and P. J. Shenoy. Energy-aware load balancing in conen delivery neworks. In INFOCOM, pages , [19] D. Meisner, B. Gold, and T. Wenisch. Powernap: eliminaing server idle power. ACM Sigplan Noices, 44(3): , [20] E. Nygren, R. Siaraman, and J. Sun. The Akamai Nework: A plaform for high-performance Inerne applicaions. ACM SIGOPS Operaing Sysems Review, 44(3):2 19,2010. [21] A. Qureshi, R. Weber, H. Balakrishnan, J. Guag, and B. Maggs. Cuing he elecric bill for inerne-scale sysems. In Proceedings of he ACM SIGCOMM 2009 conference on Daa communicaion, pages ACM, [22] D. Schneider and Q. Hardy. Under he hood a google and facebook. Specrum, IEEE, 48(6):63 67, [23] R. Urgaonkar, B. Urgaonkar, M. J. Neely, and A. Sivasubramaniam. Opimal power cos managemen using sored energy in daa ceners. In ACM SIGMETRICS, pages , Jun [24] P. Ven, N. Hegde, L. Massoulie, and T. Salonidis. Opimal conrol of residenial energy sorage under price flucuaions. In Inernaional Conference on Smar Grids, Green Communicaions and IT Energy-aware Technologies, [25] D. Wang, C. Ren, A. Sivasubramaniam, B. Urgaonkar, and H. K. Fahy. Energy sorage in daaceners: Wha, where and how much? In ACM SIGMETRICS, Jun [26] A. Wierman, L. Andrew, and A. Tang. Power-aware speed scaling in processor sharing sysems. In INFOCOM 2009, IEEE, pages IEEE, 2009.