Pricing Data Center Demand Response

Transcription

1 Prcng Data Center Demand Response Zhenhua Lu, Irs Lu, Steven Low, Adam Werman Calforna Insttute of Technology Pasadena, CA, USA ABSTRACT Demand response s crucal for the ncorporaton of renewable energy nto the grd. In ths paper, we focus on a partcularly promsng ndustry for demand response: data centers. We use smulatons to show that, not only are data centers large loads, but they can provde as much or possbly more) flexblty as large-scale storage f gven the proper ncentves. However, due to the market power most data centers mantan, t s dffcult to desgn programs that are effcent for data center demand response. To that end, we propose that predcton-based prcng s an appealng market desgn, and show that t outperforms more tradtonal supply functon bddng mechansms n stuatons where market power s an ssue. However, predcton-based prcng may be neffcent when predctons are naccurate, and so we provde analytc, worst-case bounds on the mpact of predcton error on the effcency of predcton-based prcng. These bounds hold even when network constrants are consdered, and hghlght that predcton-based prcng s surprsngly robust to predcton error.. INTRODUCTION Demand response s wdely recognzed as a crucal tool for ncorporatng renewables nto the grd, e.g., see recent reports from the Natonal Insttute of Standards and Technology NIST) and the Department of Energy DoE) [3,4]. Demand response programs provde ncentves for customers to adapt ther electrcty demand to supply avalablty, for example, reducng ther consumpton n response to a peak load warnng sgnal or request from the utlty. Thus, demand response programs can help the grd transton from the paradgm of generaton follows demand to one where, at least partally, demand follows generaton. Such a transton s fundamental to the ntegraton of renewable energy because generaton s becomng more ntermttent and less controllable as renewable penetraton ncreases. In ths paper, we consder a promsng demand response resource: data centers. Data centers are partcularly wellsuted for demand response. Frst, data centers represent large loads for the grd. In 2, they consumed approxmately.5% of all electrcty worldwde and ndvdual data centers can be 5 MW, or more [,2,42]. Further, the energy consumpton of data centers s growng quckly, by approxmately -2% per year [, 2, 3]. Ths growth s crucal for keepng pace wth the growth of renewable adopton pre- Ths work was supported by NSF grants CCF 835, CNS 94, and CNS 84625, DoE grant DE-EE289, ARO MURI grant W9NF , Mcrosoft Research, Bell Labs, the Lee Center for Advanced Networkng, and ARC grant FT dcted for the comng years. Thrd, and most mportantly, data centers are extremely flexble loads. Data centers are hghly automated and montored, e.g., the power load and state of IT equpment and coolng facltes can be contnuously montored and panoramcally adjusted. For example, a recent emprcal study by LBNL has quantfed the flexblty n power usage of four data centers under dfferent management approaches [2]. They fnd that 5% of the load can typcally be shed n 5 mnutes and % of the load can be shed n 5 mnutes; and that these can be acheved wthout changes to how the IT workload s handled,.e., va temperature adjustment and other buldng management approaches. Further, f workload management approaches are consdered, the degree of flexblty can be larger, wthout addtonal tme needed to shed the load. Sgnfcant research has recently gone nto the desgn of such workload management, e.g., [, 8, 2, 34, 38, 48, 5, 52]. Data center demand response today. Despte wde recognton of the demand response potental of data centers, the current realty s that data centers perform lttle, f any, demand response [2, 42]. In partcular, the most common demand response program avalable for data centers s Concdent Peak Prcng CPP), whch s requred for medum and large ndustral consumers n many regons. These programs work by chargng a very hgh prce for usage durng the concdent peak hour, often over 2 tmes hgher than the base rate. It s common for the concdent peak charges to account for 23% or more of a customer s electrc bll accordng to Fort Collns Utltes [47]. Hence, a customer has a strong ncentve to reduce usage durng the peak hour. Although t s mpossble to accurately predct exactly when the peak hour wll occur, many utltes dentfy potental peak hours and send warnng sgnals to customers 5- per month), whch helps customers manage ther loads and make decsons about ther energy usage. For more detals about CPP see [47]. Unfortunately, CPP programs are poorly desgned from the perspectve of data center demand response. Not provdng response may ncur a very large charge and provdng a response may not actually result n any savngs f the concdent peak does not occur durng the warnng perod. As a result, even when they are forced to partcpate n such programs, data centers tend not to actvely respond to sgnals. Further, even f they do respond, such programs extract very lttle flexblty from data centers. At best they obtan curtalment of usage a few tmes per month. Ths wastes the The concdent peak hour s defned as the hour when the most electrcty s demanded from the load servng entty LSE).

2 potental responsveness of data centers. Demand response market desgn. Although researchers have begun to focus on new market desgns for data center demand response, e.g., [2, 28, 44, 45, 47], a clear vson remans elusve. Ths s also true outsde of the doman of data centers. Recently, the desgn of demand response programs has receved consderable attenton n a varety of settngs, e.g., electrc vehcles, pool pumps, and ar condtoner cyclng. Broadly speakng, the demand response programs that have emerged can be classfed nto two categores based on the nteracton wth users: ether ) users bd some degree of flexblty supply) nto the market, usually va a parameterzed supply functon, or ) users respond to a posted prce, whch was chosen usng predctons about the avalable flexblty e.g., supply functons). We term these approaches supply functon bddng and predcton-based prcng, respectvely. Examples of proposed desgns that use supply functon bddng nclude [29,49], and examples of predctonbased prcng desgns nclude [2, 32, 39]. Whle each of these desgn approaches has pluses and mnuses as we dscuss n Secton 3), our focus on data centers motvates us to focus on predcton-based prcng programs. In partcular, a key assumpton n the desgn and analyss of supply functon bddng demand response programs s that users are prce takers,.e., they do not antcpate ther mpact on the prce. Under ths assumpton, such desgns can mnmze the aggregate user cost whle achevng the desred curtalment of demand. However, f ths assumpton s volated, and users act strategcally, then neffcency emerges n the market. Data centers are a canoncal example of a user wth market power data centers can make up 5% of the load of the dstrbuton crcuts they are on, e.g., Facebook s data center n Crook County, Oregon. Thus, t s dangerous to treat them as prce takers. In contrast, predcton-based prcng s not nearly as mpacted by market power ssues. It s, however, hghly dependent on the accuracy of the predctons of the user response to prces. Thus, there are stll sgnfcant challenges n the desgn of such programs, and these ssues are the focus of ths paper. Contrbutons of ths paper. Ths paper makes two man contrbutons: ) t quantfes the potental of data center demand response through a comparson wth large-scale storage, and ) t presents and analyzes a novel desgn for predcton-based prcng of data center demand response. We dscuss each of these n more detal n the followng. The potental of data center demand response: To quantfy the potental of data center demand response we perform numercal case studes that compare the value of the flexblty provded by data centers wth that provded by large-scale storage. In partcular, n Secton 2, we ask: How much optmally placed) storage can a data center replace? Interestngly, our results hghlght that the flexblty provded by data centers s as valuable as, and often more valuable than, the flexblty provded by large-scale storage when t comes to ensurng that a dstrbuton network meets ts voltage constrants n the presence of a large-scale solar PV) nstallaton see Fgures 6). For example, the voltage volaton frequency that comes from usng a 3MW data center, whch can provde 2% flexblty, s roughly equvalent to that of MWh of optmally-placed storage n the 46 bus dstrbuton network from Southern Calforna Edson that we consder. Ths s a qute conservatve comparson because we assume storage wth nfnte chargng speed see Fgure 5 for the mpact of the chargng rate). Further, the beneft of data center flexblty s robust to the placement wthn the dstrbuton network there are very few locatons where the effectveness of the data center drops consderably see Fgure 7). Addtonally, we look at the mpact of a growng dchotomy n how IT companes address the sustanablty of ther data centers. Some companes, e.g., Apple [24], have nvested heavly n on-ste renewable generaton; whle others, e.g., Google [25], have tended to nvest n renewable generaton that s not co-located wth ther data centers. Both approaches have merts. Provdng renewable generaton onste ensures that t s avalable where a very large and flexble load s located, but f renewable generaton s not placed on ste t can be placed n locatons wth better generaton qualty and/or cheaper nstallaton costs. Interestngly, our case studes hghlght that co-locaton of data centers and large-scale PV nstallatons s very effcent. In partcular, the voltage volaton frequency when the data center s placed at the same bus as the PV n a dstrbuton network s wthn 4% of optmal. However, t s worth notng that a data center wth local PV s not nearly as effcent at helpng manage a large-scale PV nstallaton as a data center wthout local PV. In partcular, a 2MW data center wth 2% flexblty and a co-located 5MW solar nstallaton provdes the same voltage volaton frequency as.3mwh of optmally-placed storage,.e., 25% less than a 2MW data center wth no local PV. Thus, havng PV at the locaton of the data center s better than havng t elsewhere, due to the complementary durnal patterns of each, but a data center wthout local renewables s a more valuable resource for grd management than a data center wth local renewables. Predcton-based prcng: Gven the potental of data center demand response dentfed n the frst half of the paper, the second half of the paper focuses on desgnng a demand response program that can extract ths flexblty. As we have already dscussed, predcton-based prcng s an appealng canddate gven the market power data centers mantan. Thus, n Sectons 4 and 5 we present and analyze a desgn for predcton-based prcng. Secton 4 ntroduces the desgn n a context wthout the constrants mposed by the dstrbuton network, and then Secton 5 ncorporates the network constrants nto the desgn and analyss. The analyss n these sectons s focused on three ssues. Frst, we focus on the mpact of the accuracy of predctons on the effcency of the market desgn. Ths s, perhaps, the most crucal ssue for predcton-based prcng programs. Our results represent the frst analytc characterzaton of worst-case effcency bounds Theorem 2). In partcular, we derve tght bounds on the compettve rato of predctonbased prcng that hghlght the mpact of the varablty of the predcton error. The second ssue s the contrast between predcton-based prcng and supply functon bddng. As we have mentoned, predcton error hurts the former whle market power hurts the latter. Thus, the natural queston becomes: Under whch settngs s predcton-based prcng approprate? By contrastng our results wth those of [49] on the effcency of supply functon bddng, we gve an explct characterzaton n terms of market power and predcton error of when predcton-based prcng outperforms supply functon bddng Fgure ). Broadly speakng, the comparson hghlghts that predcton-based prcng s approprate for data center demand response even f predcton errors are large. Fnally, the thrd ssue our analyss focuses on s the m-

3 pact of network constrants on the desgn and effcency of predcton-based prcng. In our analyss, the network constrants manfest themselves as a chance constrant on the prce that ensures that voltage volatons n the network are rare. But, despte constrants on the prces, we prove that the effcency of predcton-based prcng s not mpacted by the network constrants,.e., the compettve rato remans unchanged Theorem 5). Ths represents the frst analytc bound on the effcency of predcton-based prcng n the presence of network constrants. 2. QUANTIFYING THE POTENTIAL OF DATA CENTER DEMAND RESPONSE Before lookng at the desgn of market programs to extract flexblty from data centers, t s crucal to quantfy the potental of such programs. In ths secton, we accomplsh ths by contrastng the flexblty provded by data centers wth that provded by large-scale storage. Often, when people thnk of the challenges for grd management that result from renewable energy, the thought s: f only we had large-scale storage... The problem s that large-scale storage s expensve, whch leads to the consderaton of demand response. But, besdes cost, demand response also has other benefts over storage. In partcular, storage needs to be pre-charged to be ready for use, whle demand response has no such requrement. However, storage has benefts as well. Frst, the placement of storage s more flexble than that of data centers. Second, apart from pre-chargng, storage does not brng wth t any electrcty demand, whereas data center demand response nherently requres the presence of a large load n the dstrbuton network. In the experments that follow, we study the mpact of these competng factors n order to understand how the potental of data center demand response compares to largescale storage. In partcular, we ask: How much optmally placed) storage can a data center replace? Snce we focus on boundng the potental of data center demand response n ths secton, we do not model market factors. Rather, we assume that the load servng entty LSE) can call on the data center and storage as needed. Market desgn s consdered n the second half of the paper. 2. Setup To quantfy the potental of data center demand response, we study a stuaton where a dstrbuton network has a large-scale solar nstallaton and ether large-scale storage or a data center to help manage the ntermttency of the solar nstallaton. The performance objectve we consder s that of mnmzng the frequency of volatons of voltage constrants n the dstrbuton network. To measure ths frequency we sum the number of buses wth voltage volatons at each tme slot and over tme,.e., the number of buses that result n voltages outsde the tolerance bounds gven by the network. We contrast the frequency of voltage volatons when a data center s present and when large-scale storage s present. Dstrbuton network. We consder two dstrbuton networks n our experments. Both are dstrbuton networks from the Southern Calforna Edson SCE) utlty company. The frst s a 47 bus network Fgure ) and the second s a 56 bus network Fgure 2). Both are descrbed n detal n [5]. There s no conventonal generaton on these dstrbuton Fgure : SCE 47 bus network. Fgure 2: SCE 56 bus network. networks. All power comes from the substaton bus, a.k.a., the zero bus, and the solar nstallaton whch we descrbe later). The demands are taken from SCE load profles [23], except for the data center, for whch the demand s descrbed later. Gven these settngs, a sgnfcant amount of the solar generaton can be transmtted out of the dstrbuton network through the substaton bus. However, because we consder a large-scale solar nstallaton, when the nstallaton has near peak generaton, the network constrants become bndng and voltage volatons are common. Note that the voltage constrant we consder s taken drectly from the network tolerance specfcatons, and s 3%. The number of volatons n our smulatons are consstent wth prevous work on these networks, e.g., [4, 5]. The presence of storage or the data center s used to help avod such volatons. For our smulatons, gven the network, the power flow s computed for a sequence of dscrete tme steps t =,..., T usng MatPower [53]. Then, we analyze the voltages for each tme step and determne the number of buses that have voltage volatons. Fnally, we sum the voltage volaton events from all buses over all tme steps, and use t to calculate the volaton frequency. The length of the tme steps that we consder s one mnute. Renewable energy. To model a solar nstallaton placed wthn a dstrbuton network, we use solar rradance data from Los Angeles, CA n February 22 [26] to alter the power load at the bus where the solar PV) generaton s located. Thus, rradance data acts lke an nstalled solar capacty. The trace s llustrated n Fgure 3a). For the experments reported, the PV s placed at bus 45 and szed at 3MW for the 47 bus network, and also placed at bus 45 but szed at 6MW for the 56 bus network. The results do not qualtatvely change when other locatons and szes are consdered. Data center model. To ncorporate a data center nto the experments, we need to model two aspects: the power usage of the data center over tme and the flexblty n the power usage of the data center.

4 Normalzed PV generaton hour a) PV generaton n February n Los Angeles, CA normalzed CPU demand hour normalzed CPU usage hour b) Interactve workload from a photo sharng web servce hour c) Facebook Hadoop workload d) Google data centers PUE Fgure 3: One week traces for a) PV generaton, b) nflexble workload, c) flexble workload, and d) coolng effcency. PUE To model the power usage of a data center, we adopt the model used n [3, 33, 34, 37], whch provdes a smple but representatve characterzaton. In partcular, we model the power demand of the data center as a functon of the workload, ncludng nteractve nflexble) and delay-tolerant flexble) workloads, and the coolng effcency, as measured by the Power Usage Effectveness PUE). To model the workload we use two traces. The nteractve workload trace s from a popular web servce applcaton wth more than 85 mllon regstered users n 22 countres see Fgure 3b)). The trace contans average CPU utlzaton and memory usage as recorded every 5 mnutes. The peakto-mean rato of the nteractve workload s about 4. The delay-tolerant workload nformaton comes from a Facebook Hadoop trace see Fgure 3c)). The total demand rato between the nteractve workload and batch jobs s :. Ths rato can vary wdely across data centers, but we choose ths rato as representatve based on dscussons n [35]. To model the data center power effcency ncludng coolng effcency, we use a trace of the PUE from Google data centers. As shown n the fgure, the PUE vares between.5 to.45, and has strong durnal pattern,.e., hgher around noon because outsde ar temperature s hgher. To combne the workload traces and the PUE to obtan a model of the total power demand of the data center, we use the followng relatonshp. vt) = P UEt)at) + bt)), where at) s the power demand from the nflexble workload and bt) s power demand from the flexble workload demand. Note that the data center power demand has the same average value as the PV generaton wth the same capacty n the dstrbuton network. The second aspect of the data center model that we must nclude s the flexblty of the power demand. For ths, our model s nformed by the recent emprcal study [2], whch we have dscussed n the ntroducton. To model the range of flexblty n our experments, we denote the demand flexblty of the data center by e and allow the data center to have demand wthn [ e)vt), mn{ + e)vt), C d }], where C d s the capacty of the data center and vt) s the data center power demand at tme t f no demand response s called upon. Thus, e =. could be acheved wthout workload management, and e =.2 can be acheved wth some workload management, e.g., qualty degradaton or load deferral. When demand response s requred from the data center, the load that mnmzes the voltage volaton rate s provded by the data center. Snce a downsde of data center demand response s that the LSE cannot control the placement of the data center, the placement of the data center s vared durng our experments n order to understand robustness to bad data center locatons. Storage model. To ncorporate large-scale storage nto our model, we adopt a standard model, e.g., from [9,27,3,46]. In order to provde a conservatve estmate of the potental of data center demand response we assume perfect storage,.e., no loss or leakage. Ths means that, at all tmes t, the storage level for the next tme step s Lt + ) = Lt) + ut), where ut) s the energy change n the level at tme t. Note that ut) s postve f we are chargng the storage and negatve f we are dschargng. Of course, Lt) [, C s] for all t, where C s s the storage capacty. So, ut) [ Lt), C s Lt)], where C s s the storage capacty. Ths range quantfes the amount of flexblty that can be called upon by the LSE. As n the case of the data center, the LSE wll call upon a feasble ut) that mnmzes the voltage volatons. Although more advanced energy storage management polcy could be used to further mprove the beneft, here we use ths smple greedy strategy for both data center and energy storage for comparsons. For most of the experments we assume that the storage can completely charge and dscharge n one tme step. Ths s, of course, unrealstc, but t allows us to gve a conservatve estmate of the benefts of data center demand response. We do evaluate the mpact of lmtatons on the chargng rate n Fgure 5 n order to hghlght the degree to whch ths assumpton leads to an underestmate of the value of data center demand response. As we have already mentoned, a beneft of storage s that t can be placed optmally wthn a network. The optmal placement of the storage s at bus 44 for the 47 bus network and bus 53 for the 56 bus network. Note that the optmal placement s robust as we adjust the capacty of the storage n our experments. 2.2 Case studes Usng the settng descrbed above, our focus s on two comparsons that each sheds lght on the potental of data center demand response: ) a comparson between data center demand response and large-scale storage, and ) a study of the mpact of on-ste renewable generaton on data center demand response. Data center demand response versus large-scale storage. To contrast large-scale storage wth data center demand response, we frst need to quantfy the benefts from large-scale storage. Ths s done n Fgures 4 and 5, whch show the mpacts of the storage capacty and the storage chargng rate on the voltage volaton rate n the two dstrbuton networks. Fgure 4 hghlghts that, as expected, the voltage volaton rate decreases as storage capacty grows. However, t also shows that ths relatonshp s nonlnear and depends strongly on the network structure. Smlarly, Fgure 5 hghlghts that, as expected, a smaller chargng rate ncreases the frequency of voltage volatons. However, the mpact of a smaller chargng rate s, perhaps, more sgnfcant than expected. Note that for our experments we conservatvely estmate the value of data center demand response by comparng t wth storage havng a chargng rate of,.e., we assume that the storage can completely charge and ds-

5 Volaton Frequency Storage Capacty MWh) a) SCE 47 bus network Volaton Frequency Storage Capacty MWh) b) SCE 56 bus network Fgure 4: Impact of energy storage capacty, C s, on the voltage volaton rates. Volaton Frequency capacty = MWh capacty = 3MWh capacty = 5MWh capacty = MWh capacty = 5MWh chargng rate fracton n one mnute) a) SCE 47 bus network Volaton Frequency capacty =.MWh capacty =.3MWh capacty =.5MWh capacty = MWh capacty =.5MWh chargng rate fracton n one mnute) b) SCE 56 bus network Fgure 5: Impact of energy storage chargng rate on the voltage volaton rates. charge n one mnute. Ths s unrealstc, but provdes a lower bound on the value of data center demand response. Gven the characterzaton of storage, we can now hghlght the value of data center demand response n terms of the equvalent storage capacty,.e., n terms of the capacty of optmally-placed large-scale storage necessary to provde the same voltage volaton frequency. The results of ths comparson are shown n Fgure 6. Naturally, the amount of storage equvalent to data center demand response grows wth the sze of the data center. However, the capacty plateaus after the data center sze grows beyond 35MW for the SCE 47 bus network and beyond 6MW for the SCE 56 bus network. Note that ths s a consequence of two dfferences between the networks the structure and the sze of the PV nstallaton 3MW vs. 6MW). But, n both networks, Fgure 6 hghlghts that data center demand response has a sgnfcant potental. In partcular, recall that the comparson n ths plot assumes storage wth nfnte chargng speed,.e., a chargng rate of, and s thus qute conservatve as llustrated n Fgure 5). Addtonally, the cost of storage s upwards of $5/kWh for lthum-on batteres whch have small chargng rates) and upwards of $5/kWh for technologes wth fast chargng rates, such as flywheels. Thus, the flexblty provded by one 3MW data center s worth upwards of $5, - $5,,. These numbers are conservatve estmates, and grow consderably f a slower chargng rate s used n the smulatons or f the flexblty of the data center, e, s ncreased. Fgures 7 and 8 delve nto the comparson of data center demand response and large-scale storage n more detal for each of the networks. In Fgure 7, we fx the capacty of the data center to 2MW, whch s a representatve sze for today s IT companes, and then nvestgate the mpact of the degree of data center flexblty, e, and the placement of the data center. For example, Fgures 7a)-7c) hghlght that the voltage volaton rates decrease as data center power demand becomes more flexble. In partcular, a 2MW data center wth 2% power demand flexblty placed at the PV locaton s equvalent to.67mwh of optmally-placed storage n the 47 bus dstrbuton network. Further, Fgure 7d) Storage Capacty MWh) DC Capacty MW) a) SCE 47 bus network Storage Capacty MWh) DC Capacty MW) b) SCE 56 bus network Fgure 6: Dagram of the capacty of storage necessary to acheve the same voltage volaton frequency as data centers of varyng szes. The data center has flexblty e =.2. shows that the beneft of data center flexblty s robust to the placement of the network n the dstrbuton network,.e., there are very few locatons where the effectveness of the data center drops consderably and many locatons that are near-optmal, e.g., placng the data center at the locaton of the PV Fgure 7b)). Fgure 7d) also llustrates that a 2MW data center s better than.33mwh of storage pretty much unformly. The results n a SCE 56 bus network are smlar, as shown n Fgure 8. Should data centers nvest n co-located renewables?. There s a dchotomy rght now n how IT companes address the sustanablty of ther data centers. Some companes, e.g., Apple [24], have nvested heavly n on-ste renewable generaton; whle others, e.g., Google [25], have tended to nvest n renewable generaton that s not co-located wth ther data centers. Both approaches have merts, as we have dscussed n the ntroducton. For the purpose of ths paper, the key dstncton s how on-ste renewable generaton mpacts data center demand response. Ths context hghlghts another beneft of on ste renewable generaton t ensures that the data center s placed close to the renewables, whch s very often a near-optmal placement for demand response purposes. Frst, Fgure 7d) hghlghts that co-locaton of data centers and large-scale PV nstallatons s very effcent. In partcular, the voltage volaton frequency when the data center s placed as the same bus as the PV n a dstrbuton network s wthn 4% of optmal. However, t s worth notng that a data center wth local PV s not nearly as effcent at helpng to manage a largescale PV nstallaton as a data center wthout local PV, by comparng Fgure 7c) wth 9a). In partcular, a 2MW data center wth 2% flexblty and a 5MW solar nstallaton provdes the same voltage volaton frequency as.3mwh of optmally-placed storage when helpng to manage 3MW of PV elsewhere on the dstrbuton network,.e., 25% less than a data center wth the same flexblty but no local PV. Thus, havng PV at the locaton of the data center s better than havng t elsewhere, due to the complementary durnal patterns of each, but a data center wthout local renewables s a more valuable resource for grd management than a data center wth local renewables. 3. MARKET CHALLENGES FOR DATA CENTER DEMAND RESPONSE The prevous secton hghlghts that data centers have the potental to be as useful as, f not more useful than, storage for demand response. However, realzng ths potental s challengng. Data centers today tend not to partcpate n

6 Volaton Frequency Data Center Optmal Storage.33 MWh.67 MWh MWh Data Center Demand Flexblty a) Data center placed at the optmal storage locaton Volaton Frequency Data Center Optmal Storage.33 MWh.67 MWh MWh Data Center Demand Flexblty b) Data center placed at the PV locaton Volaton Frequency Data Center Optmal Storage.33 MWh.67 MWh MWh Data Center Demand Flexblty c) Data center placed at bus 2 Volaton Frequency Data Center Storage Bus Locaton d) Data center vs. storage Fgure 7: Comparson of a 2MW data center to large-scale storage n a 47 bus SCE dstrbuton network. a)-c) show the volaton frequency as a functon of the amount of data center flexblty, e, and compare to optmally placed storage, for dfferent locatons of the data center. d) shows the volaton frequency resultng from a data center wth e =.2 versus.33mwh of storage, for each locaton. demand response programs and, f they do, they tend to partcpate passvely. For example, the most common program for data center demand response today s concdent peak prcng CPP) and, though many data centers are forced to partcpate, they typcally do not actvely respond to the warnngs ssued by the utlty. Further, even f they dd, ths would mean that the data center provded flexblty only 5- tmes a month, whch s far from the amount of avalable flexblty. Such lmted sgnalng from the LSE to the data center cannot possbly extract the potental flexblty llustrated n Secton 2. On the other hand, f the utlty company sends too many warnng sgnals, data centers smply wll not respond to them. Thus, realzng the potental of data center demand response requres new market programs. Whle the desgn of market programs for data centers s only begnnng to receve attenton, there has been consderable work on the desgn of demand response programs n other contexts n recent years, e.g., [2, 9, 6, 7, 22, 29, 36, 49]. Much of ths work focuses on the desgn of resdental programs for, e.g., electrc vehcles, pool pumps, and ar condtoner cyclng. Broadly speakng, the demand response programs that have emerged can be classfed nto two categores based on the nteracton wth users: ether ) users bd some degree of flexblty supply) nto the market, usually va a parameterzed supply functon, or ) users respond to a posted prce, whch was chosen usng predctons about the avalable flexblty e.g., supply functons). We dscuss each of these approaches below and hghlght the challenges of each when t comes to data center demand response. Supply functon bddng. In ths approach to market desgn each user announces a bd to the load servng entty LSE) that specfes the amount load wll be curtaled as a functon of the prce, a.k.a., a supply functon. The form of the supply functon s typcally fxed to have some parametrc form and the bd specfes the parameter. The LSE then chooses a market clearng prce that acheves the demand response target. Examples of market desgns of ths form nclude [29, 49] and the references theren. Typcally, a key assumpton n the desgn and analyss of such markets s that users are prce takers,.e., they do not antcpate ther mpact on the prce. Under ths assumpton, such desgns can mnmze the aggregate user cost whle achevng the desred curtalment of demand. However, f ths assumpton s volated, and users act strategcally, then neffcency emerges. Recent work has begun to characterze ths neffcency, and the basc concluson s that t can be Volaton Frequency Data Center Optmal Storage. MWh.25 MWh.28 MWh Data Center Demand Flexblty.5 Data Center Storage Bus Locaton a) Data center placed at bus b) Data center vs. storage 53 Fgure 8: Comparson of a 4MW data center to large-scale storage n a 56 bus SCE dstrbuton network. a) shows the volaton frequency as a functon of the amount of data center flexblty, e, and compare to optmally placed storage. b) shows the volaton frequency resultng from a data center wth e =.2 compared to.7mwh of storage at each locaton. extreme [49]. Whle the assumpton that users are prce takers s natural n many demand response settngs, e.g., resdental pool pump and ar condtoner programs; t s qute problematc n the case of data centers. A resdental user does not have the power to manpulate prces,.e., does not have market power, but a large data center can make up 5% percent of the load of the dstrbuton crcuts they are on, e.g., Facebook s data center n Crook County, Oregon. Thus, data centers are a canoncal example of an agent wth market power. Ths observaton motvates the consderaton of predcton-based prcng n the current paper. Predcton-based prcng. In ths approach to market desgn, the LSE presents the user a prce that they wll pay the user for curtalment, and then the user responds. Examples of desgns of ths type can be found n [2, 32, 39] and the references theren. The challenge n such a program s how the LSE should determne the prce. If the LSE knew the supply functon of the users, then t could easly set a prce to extract the desred curtalment. However, the LSE does not have ths nformaton, and snce t s not provded by the user as n the supply functon bddng approach), the LSE must predct the user supply functons. Then, usng the predcted supply functons, the LSE can determne an approprate prce to nduce the desred curtalment. Clearly, one should expect predcton-based prcng to only be approprate f supply functons can be predcted accurately. Ths s a challenge n the data center envronment Volaton Frequency

7 Volaton Frequency MWh.33 MWh.5 MWh. Data Center Optmal Storage Data Center Demand Flexblty a) Data center placed at bus 2. Volaton Frequency Data Center Storage Bus Locaton b) Data center vs. storage Fgure 9: Comparson of a 2MW data center wth a co-located 5MW PV nstallaton to large-scale storage n a 47 bus SCE dstrbuton network. a) depcts the data center located at bus 2. b) shows the volaton frequency resultng from a data center wth e =.2 compared to.33mwh of storage, for each locaton. snce the supply functons of the data center may depend on the workloads and weather among other thngs), each of whch s hghly non-statonary. The key task n the remander of the paper s to characterze how accurate predctons must be for the predctonbased prcng approaches to be useful. Interestngly, the contrast between the performance of predcton-based prcng and supply functon bddng depends on the balance between the market power of data centers and the accuracy of supply functon predcton. We dscuss ths n Secton 4.3 by contrastng our results wth those n [49]. 4. PREDICTION-BASED PRICING FOR DATA CENTER DEMAND RESPONSE In ths secton, we develop a market program for extractng flexblty from data centers. Gven the dscusson n Secton 3, our focus s on predcton-based prcng. In partcular, the goal of ths secton s ) to optmally desgn predctonbased prcng programs for data center demand response, ) to quantfy the effcency loss created by predcton error n such programs, and ) to contrast predcton-based prcng wth supply functon bddng. We do ths n the context of a classc supply functon model n ths secton, and then show how to ncorporate dstrbuton network constrants n Secton Model formulaton The settng we consder here s where an LSE wshes to procure a total amount D of load reducton from a set of users ndexed by, 2,..., n. We focus on one tme step and gnore the network constrants n ths secton. To procure ths load reducton, the LSE announces a prce p and pays user the amount ps when user reduces consumpton by s. The market desgn task s to desgn p so that the LSE acheves the desred amount of curtalment. To model the user reacton to the prce, we assume that each user ncurs a cost C d ) when she reduces her consumpton by an amount d. We assume some parameters) of the cost functon C ) are random so that for each d, C d ) s a random varable. Ths randomness captures the fact that, n practce, the LSE does not know the parameters) of C ) exactly. However, the LSE may be able to estmate the parameters from hstorcal consumpton data and the effect of estmaton error can be modeled through the dstrbuton of the random parameters) n C ). We assume that user strategcally reduces her consumpton when faced wth a prce p n a proft maxmzng manner. Let s p) denote the unque cost mnmzng curtalment. Specfcally, for each realzaton of C ), denoted by c ), user solves whch gves mn d c d ) pd, ) s p) = c p). 2) To ensure that a unque soluton s p) always exsts, we mpose that each realzaton c ) of the random cost functon C ) s non-negatve, ncreasng, strctly convex, twce contnuously dfferentable, and has c) =. Addtonally, note that we have mplctly assumed that the randomness n C d ) s ndependent of the prce p. These are standard assumptons n the electrcty market lterature, e.g., [4, 8, 4, 5]. Gven the model above, the total demand response the LSE acheves wth prce p s the random quantty sp). Gven the uncertanty about the user costs, ths curtalment lkely does not exactly match the demand response target D. We assume that the penalty for devaton from the target s captured through a penalty functon h ). In partcular, the penalty s h D sp)). We assume ths penalty functon h ) s convex, non-negatve, has a global mnmum h) =, and s contnuously dfferentable wth h ) =. These assumptons ensure that the optmal prce s welldefned, see Theorem. 4.2 The effcency of predcton-based prcng Gven the settng descrbed above, our task s to frst understand how to prce, and then to understand the effcency loss due to predcton error. We start wth the case where the LSE has perfect predctons of the data center supply functons,.e., wth perfect foresght. Then, we move to the case where the LSE has only predctons of the data center supply functons. Fnally, we quantfy the effcency loss that results from ths uncertanty. Throughout, to evaluate the effcency of the LSE s choce of p we use a noton of socal cost defned as the sum of the penalty of devaton from the demand response target D and the total user costs,.e., Gp) := h D sp)) + Csp)). 3) Note that the socal cost Gp) s random from the LSE s perspectve for two reasons: both C d ) and the user responses s p) are random. But, the randomness n both of these orgnates from the randomness of the user cost functons C ). Prcng wth perfect foresght. Before lookng at the desgn of predcton-based prcng, t s nformatve to consder how an LSE wth perfect foresght would prce. In partcular, consder an LSE that s clarvoyant,.e., has perfect knowledge about the cost functon, and can choose pω) to mnmze Gp) for the realzaton on nstance ω. We use ω here to hghlght ths prce s for each realzaton ω. In ths stuaton, the prce chosen by the LSE s summarzed n the followng theorem. Theorem. For each realzaton ω, there exsts a unque mnmzer p such that p ω) = h D ) s p ω)), 4) and p < p, where p satsfes sp) = D.

8 An nterestng aspect of ths theorem s that the optmal prce s strctly lower than any prce p that would exactly satsfy the demand response target. Of course, usng p n practce s nfeasble. However, t provdes an mportant benchmark for the performance of predcton-based prcng wthout perfect foresght. Note p s random from LSE s perspectve, snce the cost functon realzatons are random. Thus, the strategy yelds an expected cost whch we denote as follows [ ] E [Gp )] = E mn Gp). 5) p Predcton-based prcng. In practce, the LSE does not know the exact realzaton of the user cost functon, thus t can only use predctons of the cost functons n order to choose a prce ˆp. Here, we focus on the case where the LSE chooses ˆp n order to mnmze the expected cost that results,.e., Ths yelds the followng ˆp argmn E [Gp)]. 6) p E [Gˆp)] = mn E [Gp)]. 7) p Of course, other objectves that nclude some form of rsk management may also be nterestng to consder n future work. Note that we assume that users know ther own cost functon, and can therefore choose ther curtalment amount s p) based on the true cost functon c ) cf. 2)). Ths means the random events that determne the C ) are revealed only to ndvdual users, but not to the LSE or other users). The effcency of predcton-based prcng. Clearly the cost when prcng wth perfect foresght s no larger than the cost when usng predcton-based prcng. Here, our goal s to understand how much s lost because of uncertanty about the cost functon. To quantfy ths effcency loss, we study the worst-case rato between the cost of predcton-based prcng and the cost of prcng wth perfect foresght. Ths s a compettve rato. In partcular, let F be the jont dstrbuton of all random varables n the model, and F be a set of permssble dstrbutons. Then the compettve rato we consder s formally E[Gˆp)] E[Gp )]. defned as CR = max F F To evaluate the compettve rato, we need to restrct ourselves to the quadratc penalty functon and cost functons,.e., h D ) s p) := q D 2 s p)) and 8) 2 C d ) := d 2, 9) 2X where q > s known, but X > are random varables to the LSE. Note that ths may seem restrctve, but ths form s standard wthn the electrcty markets lterature, e.g., [4, 8, 4, 5]. Then, for each realzaton, we can explctly compute the curtalments of the users. Specfcally, from 2): s p) = X p and C s p)) = 2 Xp2 ) Now, we can state the man theorems of ths secton, whch bound the compettve rato of predcton-based prcng n terms of the varablty of predcton errors Theorem 2) and show that the bound s tght Theorem 3). Let X := X, denote the varance of X by V [X], and denote the squared coeffcent of varaton of X by C 2 [X] = V[X]/E[X]) 2. Theorem 2. Suppose the penalty functon and cost functons are gven by 8) and 9), respectvely. Then the compettve rato s upper bounded by E [Gˆp)] E [Gp ω))] + qe[x]) 2 C 2 [X] + qe[x])c 2 [X] + ). ) Moreover ˆp E [p ], wth equalty f and only f V[X] =. Theorem 3. Under the condtons of Theorem 2 the bound n ) s asymptotcally tght,.e., for all ɛ >, there exsts a probablty densty functon fx) such that E [Gˆp)] E [Gp ω))] + qe[x]) 2 C 2 [X] + qe[x])c 2 [X] + ) ɛ.2) Before movng on, t s worth makng a few remarks about these theorems. Frst, the results apply both when the predcton errors from users are ndependent and when they are correlated. Second, the compettve rato decreases as the varablty of X decreases. Ths means that a better predcton can provde better performance. In the extreme case, when there s no randomness n X,.e., perfect foresght, then Theorem 2 guarantees that the compettve rato s. Moreover ˆp = p ω) and Gˆp) = Gp ). In contrast, when there s predcton error, the LSE tends to have lower prce to prevent over provsonng. Ths s because the attaned curtalment sp) s an ncreasng functon of the prce p. Specfcally, we have ˆp = qe [X] D qe [X 2 ] + E [X] and p = qd qx +, 3) whch both ncrease wth q. Thrd, t s nterestng to note that the compettve rato does not depend on the partcular dstrbutonal form beyond the frst and second moments of an aggregated value. Ths s due to the quadratc nature of both the user cost functons C ) and the penalty functon h ). One should expect that f these functons were polynomals wth hgher order then hgher order moments would show up n the compettve rato. Fnally, t s mportant to consder the mpact of the number of users, n, on the compettve rato,.e., on the effcency of predcton-based prcng. Ths does not show up explctly n Theorem 2, but t s possble to extract the nformaton va a slghtly more detaled analyss. Consder a smple case where all X are..d. wth mean E [X ] = α and varance V [X ] = σ 2. Then, the mean and varance of the random varable Xn) := n = X are gven by: E [Xn)] = nα and V [Xn)] = nσ 2. 4) As n ncreases, the central lmt theorem guarantees that Xn) nα nσ tends to a Gaussan random varable wth zero mean and unt varance. Hence, nformally, Xn) tends to a Gaussan random varable wth ts mean and varance growng lnearly n n as n 4). Note, however, that 4) only mposes condtons on the frst two moments of Xn) and does not requre Xn) to be Gaussan nor ther dstrbutons to depend on just the

9 frst two moments. To hghlght the dependence on n, let G n, g n, p n), ˆpn), Xn), ˆXn) etc. denote the correspondng quanttes when there are n users. Then, we can prove the followng corollary of Theorem 2, whch shows that the compettve rato exceeds by an amount upper bounded by the normalzed varance qσ 2 /α. Corollary 4. Suppose the frst two moments of Xn) are gven by 4). Under the condtons of Theorem 2, the bound on the compettve rato s ncreasng n n. Moreover E [G nˆpn))] E [G n p n))] = + + qσ2 α qα 3 σ 2 + q 2 α 2 ) α 2 + qα /n σ 2 as n. Note that the compettve rato ncreases as the number of users ncreases. That s because the cost h ) s based on the sum, not mean, of the users elastctes. A system wth a small number of users s dentcal to a system wth a larger number of users n whch some are entrely nelastc, whch has lower uncertanty than the large system n whch all users have random elastcty. However, the analyss above should be taken wth a gran of salt because, n practce, users are correlated. For example, on a hot day, many users wll be more reluctant to turn ther coolng systems off. We can llustrate the mpact of such correlatons wth the followng smple model. X = ɛx + X, where X are..d. and ndependent of the common random varable X. In ths case, gven ɛ >, E[X] = Θn), V[X] = Θn 2 ), so C 2 [X] = Θ), and E [G nˆpn))] E [G n p n))] = Θn). Ths hghlghts that correlaton among users can magnfy the mpact of predcton errors compared to the uncorrelated case, whch has a negatve mpact on the performance of predcton-based prcng. Such effects are not too worryng n the case of data center demand response, snce t s unlkely for there to be a large number of data centers on any gven dstrbuton network. However, we have ncluded the dscusson n order to hghlght a danger of usng predcton-based prcng n other demand response contexts. 4.3 Predcton-based prcng versus supply functon bddng The prevous results hghlght that f predctons are accurate, then predcton-based prcng can be an effectve market desgn; however, f predctons are poor the market s hghly neffcent. We now contrast the effcency of predctonbased prcng wth the supply functon bddng approach dscussed n Secton 3. Recall that prevous work has concluded that supply functon bddng s an effcent market desgn when agents have lmted market power [29, 49]. Thus, whch desgn s approprate depends on the degree to whch partcpants have market power and the accuracy of the predctons of supply functons made by the LSE. To concretely llustrate the comparson between these two approaches, we contrast the compettve rato derved above wth the parallel results n [49]. The results of ths comparson are shown n Fgure. Effcency Loss predcton based prcng supply functon bddng σ a) σ.5 supply functon bddng best predcton based prcng best number of users Fgure : Comparson of predcton-based prcng and supply functon bddng demand response programs. a) shows the effcency loss as a functon of the predcton error wth n = 5. b) shows the predcton error at whch predcton-based prcng begns to have worse effcency than supply functon bddng for each n. Specfcally, Fgure a) shows the effcency loss of both predcton-based prcng and supply functon bddng. The mpact of predcton error n terms of the standard devaton σ of X when fxng E[X ] = ) can be seen n the fgure. In partcular, the fgure hghlghts that the effcency loss ncreases as the predcton error ncrease. When the number of users s small 5 n the fgure), and thus market power s an ssue, even wth large predcton error up to 6%), the predcton-based approach can stll provde better performance than supply functon bddng. Fgure b) shows how ths changes as the number of users grows, and thus market power becomes less of an ssue. In partcular, the fgure shows the standard devaton threshold where predcton-based prcng becomes worse than supply functon bdng. Naturally, ths threshold decreases as the number of users ncreases. However, even wth users, predcton-based prcng tolerates more than 3% predcton error before provdng worse effcency than supply functon bddng. Ths emphaszes that predcton-based prcng s an appealng approach for demand response snce t s unlkely to have more than a few data centers on a gven dstrbuton crcut. b) 5. INCORPORATING NETWORK CONSTRAINTS The prevous secton ntroduces predcton-based prcng n a context wthout a power network. In that context, the results hghlght that predcton-based prcng s an appealng approach for data center demand response, snce the effcency of the mechansm s robust to errors n predcton as long as there are not a large number of correlated agents. In ths secton, our goal s to add an addtonal degree of realsm to the model, power network constrants, and to nvestgate how these constrants mpact the performance of predcton-based prcng. 5. Modelng the network The settng we consder n ths secton s the same as n Secton 4, except for the addton of network constrants. Typcally, when electrcty market ssues lke demand response are consdered, the network constrants are ether gnored entrely or a lnear approxmaton, termed the DC model, s used. See [43] for an ntroducton. However, due to our focus on reducng voltage volatons wth data center demand response, the DC model s not approprate; t assumes the voltages at all buses are fxed at the reference value, whch s seldom true n dstrbuton networks. As a result, we adopt a dfferent model, called the branch flow model, whch s commonly used for modelng dstrbu-

10 ton systems, e.g., [5, ]. Ths model stll uses a lnear approxmaton of the power constrants, but now voltage varatons are allowed at all buses except the root bus. The branch flow model s defned as follows. The power network s represented by a drected, connected tree G = N, E), where each node n N := {,,..., n} represent a bus wth the root at bus, each edge n E represents a lne. Denote an edge by, j) or j f t ponts from bus to bus j. The orentaton of edges s fxed to be from the root to the leaves for G. For each edge, j) E, let z j := r j +x j be the complex mpedance on the lne, and let S j := P j + Q j be the sendng-end complex power from bus to bus j. Ths s the same as the recevng end power snce lnes are assumed to be lossless. Let s j = P j + Q j be the complex net load load mnus generaton) on bus j. Here P j s the real power consumpton, whch can be further wrtten as Pj s jp), where Pj s the real power consumpton wthout demand response and s jp) s the demand reducton gven prce p. Under our model, s p) = X p,. Q j s the reactve power consumpton on bus j and we assume Q j = β jp j, j. The branch flow model s defned by the followng set of power flow equatons. S j s j = k:j k S jk, j, 5) v v j = 2Rez js j),, j, 6) where Re ) s the real part of a gven complex number. Here 5) balances the power on each bus, and 6) characterzes the voltages across lne, j) accordng to Ohm s law. The constrant for the voltage on each bus s v v v,. 7) 5.2 Predcton-based prcng n networks The ncorporaton of the network has a sgnfcant consequence for the desgn of predcton-based prcng. Due to the randomness of the cost functons, t s mpossble for the voltage constrants to be always satsfed. Ths motvates a chance constrant where the goal of the LSE when settng prce ˆp s now E [Gˆp)] = mn p E [Gp)] 8) s.t. p P{voltage volaton p} ɛ. To determne more concretely what the set of feasble prces s for the chance constrant above, we frst need to transform the power network constrants nto constrants on feasble prces. To accomplsh ths, note that 5) gves that S j = k T j s k, where T j s the tree rooted at bus j ncludng bus j). Then, we can rewrte 6) as v v j = 2RezjS j) = 2Re r j x j) s k k Tj = 2 r j P k + x j Q k k T j k T j = 2 r j P k X k p ) + x j β k P k X k p ) k T j k T j = 2 k T j r j + x jβ k ) P k 2 k T j r j + x jβ k ) X k p := M j N jp. Note that M j s a constant here, whle N j s a random varable due to the uncertantes n X k. Next, assumng that E k s the set of edges from root to bus k, we have usng v = ) v k = M j N jp),j) E k = M j + N jp.,j) E k,j) E k Therefore v k v k v k becomes v k M j + N jp v k,,j) E k,j) E k whch further mples v k +,j) E k M j p v k +,j) E k M j.,j) E k N j,j) E k N j Ths condton should hold for all buses, and therefore the feasble set s max k v k +,j) E k M j v k +,j) E p mn k M j. 9),j) E k N j k,j) E k N j We can smplfy the feasble set further by assumng that the voltage constrants 7) are satsfed when there s no demand response,.e., v k M j v k, k. 2),j) E k Ths mples that the feasble range n 9) s nonempty. Addtonally, snce we only consder demand reducton wth p, 2 and v k +,j) E k M j, k, and we assume X k, we can further smplfy the feasble set to v k +,j) E p mn k M j. 2) k,j) E k N j Agan, recall that N j s random. Therefore, the constrant above s on realzatons. Importantly, for each realzaton, the constrants are lnear, and therefore we can translate the 2 Note that all the results here can be easly extended to the case where we allow p to be negatve.

11 constrants nto a bound on the fracton of volaton for each bus as follows. P N jp v k + M j ɛ, k. 22),j) E k,j) E k The above equaton can be vewed as a concrete specalzaton of the voltage volaton constrant n 8). Note that t has a number of nterestng propertes. In partcular, the volaton probablty s a strctly ncreasng { functon of p that } equals when p = and approaches P,j) Ek { Nj > } as p. Therefore, f P,j) Ek Nj > s smaller than ɛ, the chance constrant s satsfed for all p 3. Otherwse there s a threshold p ɛ at whch pont the volaton probablty exceeds ɛ. In ths case, the feasble prcng space s [, p ɛ], and the optmzng prce becomes the projecton of the unconstraned prce derved n Secton 4 onto ths nterval. 5.3 The effcency of predcton-based prcng n networks The prevous analyss hghlghts that the necessary adjustment n the prce used by the LSE due to network constrants can be acheved va a projecton onto a feasble space of prces, whch we have characterzed n 22). The goal of ths secton s to understand the mpact of network constrants,.e., the projecton nto the feasble space of prces, have on the effcency of the resultng prce. The man message of what follows s that network effects do not reduce the effcency of predcton-based prcng, when effcency s measured by the compettve rato. In partcular, let us compare our algorthm wth the clarvoyant algorthm that uses the same feasble set [, p ɛ] for each realzaton. Ths makes the offlne algorthm weaker than the one consdered n Secton 4,.e., the performance s strctly worse. Recall that we denote by Gˆp) and Gp ω)) the cost of our algorthm and the clarvoyant algorthm n Secton 4 where network constrants are not consdered. Let us now denote by Gˆp ɛ) and Gp ɛ ω)) the cost of our algorthm and the clarvoyant algorthm wth the same feasble set [, p ɛ], defned as a functon of the network constrants. Our goal s to compare the compettve rato wthout network constrants,.e., E[Gˆp)] E[Gp ω))], to the compettve rato E[G ˆp under network constrants,.e., ɛ)] E[Gp ω))]. ɛ The followng theorem hghlghts that constrants on the prcng space actually reduce the effcency loss from uncertanty, and so the compettve rato of predcton-based prcng remans unchanged when network constrants are consdered. In the statement, we consder the feasble prce set R := [p, p] and denote by gˆp R) and gp R) the cost of our algorthm and the clarvoyant algorthm wth the same feasble set for a convex functon g ), e.g., a realzaton of the random functon G ). Proof s gven n the appendx. Theorem 5. Consder any postve, convex functon g ) that s a realzaton of the random functon G ) and any non-empty feasble set R := [p, p]. Then, gˆp) gp ) gˆpr) gp R ), 23) 3 Ths does not happen n{ our case because we assume X s } are postve, therefore P,j) Ek Nj > = and thus E [Gˆp)] E [Gp ω))] E [G ˆpɛ)] E [Gp ɛ ω))]. 24) A key dstncton between ths theorem and Theorem 2 s that the feasble prce set of both the optmal and the algorthm are fxed to R := [p, p]. Ths mples that we are not comparng wth the true offlne optmal, whch may have dfferent feasble sets for the prce for dfferent realzatons. Instead, we are comparng wth the weaker offlne optmal that, because of uncertanty, optmzes over the same feasble prce set as our onlne algorthm, but then has the foresght necessary to choose optmally gven these prce constrants. Ths s a common choce for comparson when studyng the compettve rato of onlne algorthms n stuatons where clarvoyance yelds dfferent feasble acton spaces. 6. CONCLUDING REMARKS In ths paper we have hghlghted two man ponts. Frst, that data center demand response has sgnfcant potental and, second, that predcton-based prcng s an appealng mechansm wth whch to extract ths potental. More concretely, we have llustrated that, not only are data centers large loads to target wth demand response programs, they can provde nearly the same degree of flexblty for LSEs as large-scale storage f properly ncentvzed. However, ths last caveat s crucal t s much harder to extract flexblty from data centers than from storage. To that end, ths paper has argued that predcton-based prcng s a promsng market desgn for ths context. Whle, n general, predcton-based prcng may be less effcent than supply functon bddng due to predcton errors), because data centers typcally have sgnfcant market power on ther dstrbuton networks, supply functon bddng can be very neffcent whereas predcton-based prcng s less nfluenced. In partcular, the analytc results n Sectons 4 and 5 hghlght that the effcency of predcton-based prcng s favorable to that of supply functon bddng when market power s an ssue even when predctons are error prone. These analytc results are the frst, to our knowledge, that provde bounds on the compettve rato of predcton-based prcng programs, and also the frst to provde an analyss of predcton-based prcng programs n a context where network constrants are consdered. However, much work stll remans before predcton-based prcng can be used n practce. In partcular, t s mportant to do an emprcal study to understand how predctable the response of data centers wll be n such programs. Intal plot studes along these lnes are proceedng n some demand response markets, but these have yet to focus on data centers specfcally. Dependng on the result of such studes, t may be natural to consder hybrd mechansms that combne predctons and bddng n order to extract supply functon nformaton from data centers. Addtonally, many practcal aspects of predcton-based prcng programs stll requre careful thought. For example, what s the approprate tme-scale at whch prces should be adjusted? The tme-scale chosen allows for a balance between the responsveness desred by the LSE and the rskaverson of the data center. Further, n ths paper we have assumed a scalar prce. One could also nvestgate locaton dependent prces n dstrbuton networks, smlar to locatonal margnal prces LMPs) for transmsson networks. Whle these are not currently used, the extra geographcal flexblty they provde could be valuable. Fnally, there are nterestng exploraton-explotaton tradeoffs that come up

12 when settng prces n predcton-based prcng programs. We have not addressed ths ssue n ths paper due to the complextes of the power network, but work n the operatons research communty has begun to study ths n other contexts [6, 7], usng regret as the performance measure. It would be nterestng for future work to ncorporate ths ssue nto the demand response context. 7. REFERENCES [] Report to congress on server and data center energy effcency. 27. [2] D. J. Agner and J. G. Hrschberg. Commercal/ndustral customer response to tme-of-use electrcty prces: Some expermental results. The RAND Journal of Economcs, 63):34 355, 985. [3] B. Aksanl, J. Venkatesh, L. Zhang, and T. Rosng. Utlzng green energy predcton to schedule mxed batch and servce jobs n data centers. ACM SIGOPS Operatng Systems Revew, 453):53 57, 22. [4] B. Allaz and J.-L. Vla. Cournot competton, forward markets and effcency. Journal of Economc theory, 59): 6, 993. [5] M. Baran and F. F. Wu. Optmal szng of capactors placed on a radal dstrbuton system. IEEE Transactons on Power Delvery, 4): , 989. [6] O. Besbes and A. Zeev. Dynamc prcng wthout knowng the demand functon: Rsk bounds and near-optmal algorthms. Operatons Research, 576):47 42, 29. [7] O. Besbes and A. Zeev. On the mnmax complexty of prcng n a changng envronment. Operatons research, 59):66 79, 2. [8] D. W. Ca and A. Werman. Ineffcency n forward markets wth supply frcton. In Proc. of CDC, 23. [9] L. Chen, N. L, S. H. Low, and J. C. Doyle. Two market models for demand response n power networks. In IEEE SmartGrdComm, pages , 2. [] Y. Chen, D. Gmach, C. Hyser, Z. Wang, C. Bash, C. Hoover, and S. Snghal. Integrated management of applcaton performance, power and coolng n data centers. In Proc. of NOMS, 2. [] H.-D. Chang and M. E. Baran. On the exstence and unqueness of load flow soluton for radal dstrbuton power networks. IEEE Transactons on Crcuts and Systems, 373):4 46, 99. [2] A. J. Conejo, J. M. Morales, and L. Barngo. Real-tme demand response model. IEEE Transactons on Smart Grd, 3): , 2. [3] Department of Energy. The smart grd: An ntroducton. [4] M. Farvar, C. R. Clarke, S. H. Low, and K. M. Chandy. Inverter var control for dstrbuton systems wth renewables. In IEEE SmartGrdComm, pages , 2. [5] M. Farvar, R. Neal, C. Clarke, and S. Low. Optmal nverter var control n dstrbuton systems wth hgh pv penetraton. In IEEE Power and Energy Socety General Meetng, pages 7, 22. [6] L. Gan, U. Topcu, and S. H. Low. Optmal decentralzed protocol for electrc vehcle chargng. In IEEE CDC, pages , 2. [7] L. Gan, A. Werman, U. Topcu, N. Chen, and S. H. Low. Real-tme deferrable load control: handlng the uncertantes of renewable generaton. In Proc. of ACM eenergy, pages 3 24, 23. [8] A. Gandh, Y. Chen, D. Gmach, M. Arltt, and M. Marwah. Mnmzng data center SLA volatons and power consumpton va hybrd resource provsonng. In Proc. of IGCC, 2. [9] N. Gast, J.-Y. Le Boudec, A. Proutère, and D.-C. Tomoze. Impact of storage on the effcency and prces n real-tme electrcty markets. In Proceedngs of the fourth nternatonal conference on Future energy systems, pages ACM, 23. [2] G. Ghatkar, V. Gant, N. Matson, and M. Pette. Demand response opportuntes and enablng technologes for data centers: Fndngs from feld studes. 22. [2] J. Heo, P. Jayachandran, I. Shn, D. Wang, T. Abdelzaher, and X. Lu. Opttuner: On performance composton and server farm energy mnmzaton applcaton. Parallel and Dstrbuted Systems, IEEE Transactons on, 22):87 878, 2. [22] Y.-Y. Hsu and C.-C. Su. Dspatch of drect load control usng dynamc programmng. IEEE Transactons on Power Systems, 63):56 6, 99. [23] [24] [25] [26] [27] L. Huang, J. Walrand, and K. Ramchandran. Optmal demand response wth energy storage management. In IEEE SmartGrdComm, pages 6 66, 22. [28] D. Irwn, N. Sharma, and P. Shenoy. Towards contnuous polcy-drven demand response n data centers. Computer Communcaton Revew, 44), 2. [29] R. Johar and J. N. Tstskls. Parameterzed supply functon bddng: Equlbrum and effcency. Operatons research, 595):79 89, 2. [3] J. Koomey. Growth n data center electrcty use 25 to 2. Oakland, CA: Analytcs Press. August, :2, 2. [3] J.-Y. Le Boudec and D.-C. Tomoze. A demand-response calculus wth perfect batteres. In Measurement, Modellng, and Evaluaton of Computng Systems and Dependablty and Fault Tolerance, pages Sprnger Berln Hedelberg, 22. [32] N. L, L. Chen, and S. H. Low. Optmal demand response based on utlty maxmzaton n power networks. In IEEE Power and Energy Socety General Meetng, pages 8, 2. [33] M. Ln, Z. Lu, A. Werman, and L. Andrew. Onlne algorthms for geographcal load balancng. In Proc. of IGCC, 22. [34] M. Ln, A. Werman, L. L. H. Andrew, and E. Thereska. Dynamc rght-szng for power-proportonal data centers. In Proc. of INFOCOM, 2. [35] Z. Lu, Y. Chen, C. Bash, A. Werman, D. Gmach, Z. Wang, M. Marwah, and C. Hyser. Renewable and coolng aware workload management for sustanable data centers. In Proc. of ACM Sgmetrcs, 22. [36] Z. Ma, D. Callaway, and I. Hskens. Decentralzed chargng control for large populatons of plug-n electrc vehcles. In IEEE CDC, pages 26 22, 2. [37] A. H. Mahmud and S. Ren. Onlne capacty provsonng for carbon-neutral data center wth demand-responsve electrcty prces. ACM SIGMETRICS Performance Evaluaton Revew, 42):26 37, 23. [38] D. Mesner, C. Sadler, L. Barroso, W. Weber, and T. Wensch. Power management of onlne data-ntensve servces. In Proc. of ISCA, 2. [39] A.-H. Mohsenan-Rad and A. Leon-Garca. Optmal resdental load control wth prce predcton n real-tme electrcty prcng envronments. IEEE Transactons on Smart Grd, 2):2 33, 2. [4] F. Murphy and Y. Smeers. On the mpact of forward markets on nvestments n olgopolstc markets wth reference to electrcty. Operatons research, 583):55 528, 2. [4] Natonal Insttute of Standards and Technology. NIST framework and roadmap for smart grd nteroperablty standards. NIST Specal Publcaton 8, 2. [42] NY Tmes. Power, Polluton and the Internet. [43] N. S. Rau. Optmzaton prncples: practcal applcatons to the operaton and markets of the electrc power ndustry. John Wley & Sons, Inc., 23. [44] B. Urgaonkar, G. Kesds, U. V. Shanbhag, and C. Wang. Prcng of servce n clouds: Optmal response and strategc nteractons. 23. [45] R. Urgaonkar, B. Urgaonkar, M. Neely, and

13 A. Svasubramanam. Optmal power cost management usng stored energy n data centers. In Proc. of the ACM Sgmetrcs, 2. [46] P. Van de Ven, N. Hegde, L. Massoule, and T. Salonds. Optmal control of resdental energy storage under prce fluctuatons. In Proc. of ENERGY, pages 59 62, 2. [47] [48] H. Xu and B. L. Cost effcent datacenter selecton for cloud servces. In Proc. of ICCC, 22. [49] Y. Xu, N. L, and S. Low. On the effcency of parameterzed supply functon bddng wth capacty constrants. [5] J. Yao, S. S. Oren, and I. Adler. Two-settlement electrcty markets wth prce caps and cournot generaton frms. European Journal of Operatonal Research, 83): , 27. [5] Y. Yao, L. Huang, A. Sharma, L. Golubchk, and M. Neely. Data centers power reducton: A two tme scale approach for delay tolerant workloads. In Proc. of INFOCOM, pages , 22. [52] Q. Zhang, M. Zhan, Q. Zhu, S. Zhang, R. Boutaba, and J. Hellersten. Dynamc energy-aware capacty provsonng for cloud computng envronments. In ICAC, 22. [53] R. D. Zmmerman, C. E. Murllo-Sánchez, and D. Gan. A matlab power system smulaton package, 25. APPENDIX A. APPENDIX: PROOFS Proof of Theorem To begn, we compute as follows: g p) = h D ) s p) s p) + c s p)) h = s p) p h = D D s p) )) s p) s p), c s p))s p) )) where the last equalty follows from c s p)) = p for all. Our assumptons mply that s p) = c s p))) >, and hence g p) = f and only f vp) := p h D ) s p) =. Now, v) = h D), vp) = p >, and vp) s strctly ncreasng. Hence a unque p < p satsfes vp ) =. Moreover g p) < for p < p and g p) > for p > p mplyng that p s the unque mnmzer of gp). Proof of Theorem 2 We frst evaluate E [Gp )]. From Theorem p = h D ) s p ) = q D p ) X Hence = qd qxp. p = whch s a random optmal) prce. Next, from ) and 25) we have [ E [Gp )] = qd2 2 E q D, 25) + qx + qx ], 26) where the expectaton s taken over X. To evaluate 7) we have, usng ), [ q E [Gˆp)] = mn E p 2 D Xp)2 + ] 2 Xp2 [ = mn qe X 2 ] + E [X] ) p 2 2qDE [X] p + qd 2). p 2 Consequently, the unque mnmzer ˆp and the optmal value of 7) are ˆp = E [Gˆp)] = qd2 2 qe [X] D, 27) qe [X 2 ] + E [X] E [X] + qv [X] E [X] + qe [X 2 ]. 28) We can now quantfy the compettve rato usng 26) and 8). Jensen s nequalty mples E [Gp )] qd2 2 +qe[x].

14 Thus, E [Gˆp)] E [Gp )] = + E [X] + qv [X] + qe [X]) E [X] + qe [X 2 ] q 2 E [X] E [X] + qe [X 2 ] V [X]. Rewrtng the above n terms of the square coeffcent of varaton C 2 [X] gves: E [Gˆp)] E [Gp )] + qe[x]) 2 C 2 [X] + qe[x])c 2 [X] + ). Fnally, to compare ˆp n 27) wth p n 25) we can rewrte ˆp as ˆp = qd + qe[x]c 2 [X] + ). Hence E [p ] = [ ] qd E + qx qd + qe [X] qd + qe[x]c 2 [X] + ) = ˆp, where the frst nequalty follows from the Jensen s nequalty and the second nequalty follows from C 2 [X]. Both of these are equaltes f and only f X has zero varance. Proof of Theorem 3 To show tghtness we focus on the only nequalty used n the proof of Theorem 2, whch s E [Gp )] qd E [X]). We need to show that, for any ɛ >, there exsts a probablty dstrbuton fx) wth mean E [X] and varance V [X] such that E [Gp )] qd E [X]) + ɛ. We defne such a probablty dstrbuton as follows. For any < x <, let d := E [X] V [X] x)/x and d 2 := E [X] + V [X] x/ x). Then defne the followng probablty densty functon: where f xx) = xδe [X] d ) + x)δe [X] d 2), 29) δa) := { f a = otherwse E [Gˆp)] E [Gp )] + qe[x]) 2 C 2 [X] + qe[x])c 2 [X] + ) = + q 2 n 2 α 2 σ2 nα 2 ) σ + qnα + nα 2 = + Proof of Theorem 5 qα 3 σ qσ2 α q 2 α 2 ) α 2 + qα /n σ 2 as n. To prove that the compettve rato of predcton-based prcng does not become larger when there are constrants on the space of prces,.e., p [p, p], we consder two cases. The cases are dagramed n Fgure. Case 2 Case Fgure : Dagram of cases for proof of Theorem 5. Case : The prce p pcked by the clarvoyant algorthm s wthn the feasble set [p, p],.e., p [p, p]. We have p R = p and therefore gp R) = gp ). If the prce pcked by our algorthm ˆp [p, p], then we have ˆp R = ˆp and therefore gˆp R) = gˆp). Hence gˆp) = gˆp R) gp ) gp. R ) Otherwse ˆp / [p, p]. We have ˆp R = p f ˆp < p and ˆp R = p f ˆp > p. In ether case gˆp R) gˆp), and therefore gˆp) gp ) gˆp R ) gp R ). Case 2: The prce p pcked by th clarvoyant algorthm s outsde the feasble set [p, p]. Wthout loss of generalty, we assume p < p, as shown n the fgure. We have p R = p and gp R) gp ). If the prce pcked by our algorthm ˆp [p, p], then we have ˆp R = ˆp and therefore gˆp R) = gˆp). Hence gˆp) gˆp R) gp ) gp. R ) Otherwse ˆp / [p, p]. We have ˆp R = p f ˆp < p and ˆp R = p f ˆp > p. In the frst case we have ˆp R = p R = p and therefore gˆp R) = gp R), hence gˆp) gˆp R) gp ) gp =. In the second case ) R we have gˆp R) gˆp), and therefore gˆp) gˆp R) gp ) gp. R ) and δa)da =. Note that for any < x <, the probablty dstrbuton defned n 29) has mean E [X] and varance V [X] and lm E qd 2 x [Gp )] = 2 + E [X]). Thus, the bound s tght. Proof of Corollary 4 Gven E [Xn)] = nα and V [Xn)] = nσ 2, we have C 2 [X] = σ 2 nα 2. Thus, we can compute as follows.