Joint Request Mapping and Response Routing for Geo-distributed Cloud Services

Transcription

1 Jont Request Mappng and Response Routng for Geo-dstrbuted Cloud Servces Hong Xu, Baochun L henryxu, bl@eecg.toronto.edu Department of Electrcal and Computer Engneerng Unversty of Toronto Abstract Many cloud servces are runnng on geographcally dstrbuted datacenters for better relablty and performance. We consder the emergng problem of ont request mappng and response routng wth dstrbuted datacenters n ths paper. We formulate the problem as a general workload management optmzaton. A utlty functon s used to capture varous performance goals, and the locaton dversty of electrcty and bandwdth costs are realstcally modeled. To solve the large-scale optmzaton, we develop a dstrbuted algorthm based on the alternatng drecton method of multplers ADMM). Followng a decomposton-coordnaton approach, our algorthm allows for a parallel mplementaton n a datacenter where each server solves a small sub-problem. The solutons are coordnated to fnd an optmal soluton to the global problem. Our algorthm converges to near optmum wthn tens of teratons, and s nsenstve to step szes. We emprcally evaluate our algorthm based on real-world workload traces and latency measurements, and demonstrate ts effectveness compared to conventonal methods. I. INTRODUCTION Cloud servces have already become an essental part of our lfe. Notable examples nclude onlne search Google), vdeo streamng Netflx), socal networkng Facebook), etc. Many cloud servces are deployed on a geographcally dstrbuted nfrastructure,.e. datacenters located n dfferent regons as shown n Fg., for better performance and relablty. Two problems are of partcular mportance to the effcent operaton of cloud servces runnng on geographcally dstrbuted datacenters. Frst, clent requests across the wde area must be drected to an approprate datacenter, whch consttutes the request mappng problem. Second, a datacenter s usually connected through multple ISP lnks to the Internet, a practce known as mult-homng [3]. When a request s processed, the response packets must be sent back to the clent through one of the lnks avalable, whch corresponds to the response routng problem. Today, request mappng and response routng are managed ndependently, leadng to poor performance and hgh costs n many cases [7], [2]. For example, too many requests may be drected to a datacenter whose upstream lnks then become congested, resultng n long queueng delays and poor performance. The obectves of the two decsons can also be msalgned and lead to sub-optmal equlbra. In lght of the problems, we study the ont request mappng and response routng problem that has started to gan attenton recently [2] wth dstrbuted datacenters. Specfcally, we formulate the problem as a general workload management optmzaton, where key performance and cost ssues are realstcally modeled. We use a utlty functon of the average latency [33] to capture varous performance goals provders wsh to acheve for ther servces. We consder both the electrcty and bandwdth costs, whch exhbt sgnfcant locaton and provder dversty [26], [28] and together account for the maorty of the datacenter operatonal expense OPEX) [4]. Fg.. Clents Mappng nodes Requests Datacenters A cloud servce runnng on a geographcally dstrbuted nfrastructure. The workload management problem s a convex optmzaton, and can be solved n a centralzed way. However, t s nherently a very large-scale problem that makes a centralzed algorthm neffcent. In a producton system, the problem typcally has mllons of varables and hundreds of thousands of constrants as we shall llustrate n Sec. II-D. A centralzed algorthm cannot take advantage of the abundant server resources n a datacenter to parallelze the computaton for such largescale problems. Though solvng the optmzaton at a central server perodcally s possble, such a desgn also makes the system less responsve to handle sudden changes n request rates.e. flash crowds) or network condtons.e. lnk falures). In these stuatons, a soluton wth fast computaton and modest accuracy s more desrable. Thus, for reasons of performance, scalablty, and robustness, we are motvated to develop a dstrbuted soluton for the workload management problem. Our algorthm s based on the alternatng drecton method of multplers ADMM), a smple yet powerful algorthm that recently has found practcal use n many large-scale dstrbuted convex optmzaton problems [6]. ADMM works by frst separatng the obectve and varables nto two parts, and then alternatvely optmzng one set of varables that accounts for one part of the obectve to teratvely reach the optmum. Merts of ADMM, compared to conven-

2 tonal methods such as subgradent methods [5], are ts fast convergence to modest accuracy, nsenstvty to step szes, and robustness wthout strong assumptons such as strct convexty of the obectve functon [4], [6]. Our contrbutons are three-fold. Frst, we develop a general formulaton of the ont request mappng and response routng problem for cloud servces n Sec. II. We use utlty functons to capture varous performance obectves, and consder the locaton dversty of the assocated electrcty and bandwdth costs. Our second contrbuton s a novel dstrbuted algorthm based on ADMM to solve the large-scale optmzaton problem effcently Sec. III). We demonstrate that after a transformaton, the problem can be decomposed nto many small sub-problems, the solutons of whch are coordnated to fnd the global optmal soluton, and can be effcently solved n the general case. We further provde solutons n analytcal form for the case when the utlty functon s affne, and dscuss ssues pertanng to a parallel mplementaton of the algorthm n the cloud. Our thrd contrbuton s an emprcal evaluaton of the algorthm usng the Wkpeda workload traces [27], as well as realworld latency measurements [9] n Sec. IV. It s demonstrated that our algorthm offers near-optmal performance wthn 20 teratons. Fnally, we stress that the technques developed n the paper to transform the problem and apply ADMM are farly general, and may be applcable to problems n datacenters and other domans, where an effcent parallel algorthm s requred to solve large-scale convex optmzaton problems. II. A FRAMEWORK FOR JOINT MAPPING AND ROUTING Let us start by presentng our model and optmzaton framework. A. Infrastructure We consder a provder that runs her cloud servce over a set of datacenters N n dstnct geographcal regons. Each datacenter n s mult-homed to a set of ISP lnks M n, each wth a fxed capacty. Let I denote the set of clents, where n ths work a clent s smply a unque IP prefx smlar to [24]. The provder deploys a number of mappng nodes as shown n Fg. to map clent requests to an approprate datacenter based on certan crtera. Ths s the request mappng decson. In practce, these mappng nodes can be authortatve DNS servers as used by Akama and most CDNs, or HTTP ngress proxes as used by Google and Yahoo [24], [30]. We allow a mappng node to arbtrarly splt a clent s request traffc among the set of datacenters. DNS servers and HTTP proxes can acheve such flexblty n commercal products [7], [30]. When a datacenter fnshes servng a request, t sends the response packets back to the clent through one of the avalable ISP lnks. Ths corresponds to the response routng decson. Today s BGP routng pcks a sngle egress ISP lnk for each IP prefx. We relax ths constrant and allow the provder to arbtrarly splt the response traffc among all ISP lnks, whch s commonly accepted n the lterature [2], [23]. Such fractonal routng can be acheved by hash-based traffc splttng n practce [8]. Wthout loss of generalty, we vew every possble combnaton of datacenter and ISP lnk as a vrtual stub datacenter, a concept we use to facltate our analyses n the sequel. We let J, J := N {M n } denote a stub datacenter,.e. the tuple n, m, n N, m M n. Each stub datacenter then has a fnte capacty C determned by ts correspondng ISP lnk s capacty. Here we mplctly assume that the lnk capacty s the bottleneck of the servce compared to the datacenter s computatonal capablty, whch s generally the case n realty. The request mappng and response routng decsons can then be treated ontly as a sngle workload management optmzaton between the clents and the stub datacenters. The provder perodcally, e.g. hourly or daly, computes the workload management decsons to better cope wth dynamc request traffc under normal operatons [2], [3], [26]. We use α [0, ] to denote the proporton of requests dstrbuted to stub datacenter from clent. α s our decson varable. We assume the provder employs statstcal machne learnng technques [23], [25] to predct the traffc demand of each clent D before each optmzaton nterval. Such an assumpton s commonly made n the lterature [2], [30], [3]. B. Performance Latency s arguably the most mportant performance metrc for most cloud servces. A small ncrease n the user-perceved latency can cause substantal revenue loss for the provder [6]. In ths paper we focus on the end-to-end propagaton latency between users and datacenters, whch largely accounts for the user-perceved latency compared to other factors such as request processng tmes at datacenters [], [2]. The provder obtans the propagaton latency L between clent and stub datacenter through actve measurements [9] or other means. A clent s performance depends on the average propagaton latency ts requests receve α L through a generc utlty functon U. U can take varous forms dependng on the performance goals the provder pursues. We only requre that U s a decreasng, dfferentable, and concave functon. Ths utlty noton allows us a consderable amount of expressveness. For example, t can ncorporate farness among clents by usng the canoncal alpha-far utlty functons [20]. C. Costs Two knds of operatng costs electrcty and bandwdth are nvolved n servng clent requests, both of whch scale wth the total volume of the workload. The electrcty prce exhbts sgnfcant locaton dversty whch has been exploted to save costs for datacenters [8], [26], [3]. We use P E to denote the power prce of the stub datacenter, whch s determned by the locaton of the correspondng datacenter. The bandwdth prce vares across ISPs and also exhbts locaton dversty n practce [28], and s denoted by P B dependng on the correspondng ISP and locaton of the stub datacenter. In realty many ISPs adopt the 95-percentle chargng scheme. However we assume the bandwdth cost s lnear wth the traffc volume. Optmzng a lnear cost n each nterval can reduce the monthly 95-percentle bll [32].

3 D. Problem Formulaton We are now n a poston to formally formulate the workload management problem as an optmzaton that maxmzes the total utlty of servng the requests, mnus the electrcty and bandwdth costs ncurred. mn α D α P E I J + P B ) ) D U α L I J ) α =, I, 2) J α D C, J, 3) I α 0, I, J. 4) ) s the obectve functon that poses the maxmzaton problem n an equvalent mnmzaton form. Note that by addng a scalar weght factor n front of the utlty functon, any desred trade-off pont between performance and cost can be acheved. For smplcty we assume the weght s. 2) s the workload conservaton constrant that dctates each clent s demand has to be satsfed. 3) s the capacty constrant that prevents the ISP lnk of a stub datacenter from overflow. 4) s smply the non-negatvty constrant for the varables. Our formulaton focuses on the performance-cost trade-off. In practce a provder may also need to consder varous polces when desgnng the request mappng and response routng strateges. Though we do not consder polces n ths paper, they can be modeled as addtonal constrants to the problem and do not fundamentally change the formulaton. The optmzaton ) s a very large-scale problem. To have a rough understandng, the number of clents represented by the number of unque IP addresses s O0 5 ), and the number of datacenters and ISP lnks s around O0 2 ) n some producton clouds [7], [2]. Ths mples that the problem can have O0 7 ) varables, and O0 5 ) constrants for a producton system. E. Exstng Approaches As argued n Sec. I, the lack of effcency and robustness n centralzed algorthms motvates our desgn of a dstrbuted soluton amenable to parallel mplementatons n the cloud. The common approach to develop dstrbuted algorthms s to relax the constrants and employ dual decomposton to decompose the problem nto many ndependent sub-problems [9]. Subgradent methods can then be used to update the dual varables towards the optmalty of the dual problem [5]. Yet, these approaches are not applcable here. Frst of all, dual decomposton requres the utlty functon to be strctly convex, for an affne functon wll make the Lagrangan unbounded below n α. However, for workload management n cloud computng, an affne utlty functon s n fact one of the most popular and commonly studed utlty functons [2], [3]. More mportantly, subgradent methods suffer from the curse of step sze. For the output to be close to the optmum, we need to strategcally pck the step sze at each teraton, leadng to the well-known problems of slow convergence and performance oscllaton when the problem scale s large. Even f U were ndeed a strctly convex functon, subgradent methods are not well suted n our problem. Summarzng the dscussons, we need a scalable and practcal dstrbuted algorthm that converges fast to modest accuracy, and s not senstve to step szes. In the followng, we present such an algorthm based on the alternatng drecton method of multplers ADMM) [6]. III. ALGORITHM DESIGN We frst provde a bref prmer on ADMM whch s the corner stone of our algorthm desgn. A. A Prmer on ADMM ADMM, developed n the 970s [4], has recently receved renewed nterest n solvng large-scale dstrbuted convex optmzaton n statstcs, machne learnng, and related areas [6]. The algorthm solves problems n the form mn fx) + gz) 5) Ax + Bz = c, x C, z C 2, wth varables x R n and z R m, where A R p n, B R p m, and c R p. f and g are convex functons, and C, C 2 are non-empty polyhedral sets. Thus, the obectve functon s separable over two sets of varables, whch are coupled through an equalty constrant. We can form the augmented Lagrangan [5] by ntroducng an extra L-2 norm term Ax + Bz c 2 2 to the obectve: L x, z, λ) = fx) + gz) + λ T Ax + Bz c) + /2) Ax + Bz c ) > 0 s the penalty parameter L 0 s the standard Lagrangan for the problem). The augmented Lagrangan can be vewed as the unaugmented Lagrangan assocated wth the problem mn fx) + gz) + /2) Ax + Bz c 2 2 Ax + Bz = c, x C, z C 2, Clearly ths problem s equvalent to the orgnal problem 5), snce for any feasble x and z the penalty term added to the obectve s zero. The beneft of ntroducng the penalty term s that L s strctly convex even when f and g are affne, and we can work on the dual problem wthout strong assumptons on f and g. The penalty term s also called a regularzaton term and helps substantally mprove the convergence of the algorthm. ADMM solves the dual problem wth the teratons: x t+ := argmn L x, z t, λ t ) 7) x C z t+ := argmn L x t+, z, λ t ) 8) z C 2 λ t+ := λ t + Ax t+ + Bz t+ c). 9) It conssts of an x-mnmzaton step 7), a z-mnmzaton step 8), and a dual varable update 9). Note the step sze

4 s smply the penalty parameter. Thus, x and z are updated n an alternatng or sequental fashon, whch accounts for the term alternatng drecton. Separatng the mnmzaton over x and z s precsely what allows for decomposton when f or g are separable, whch wll be useful n our algorthm desgn. The optmalty and convergence of ADMM can be guaranteed under very mld techncal assumptons. Theorem : [4] Assume that the optmal soluton set of problem 5) s non-empty, and ether C s bounded or else the matrx A T A s nvertble. Then a sequence {x t, z t, λ t } generated by 7) 9) s bounded, and every lmt pont of {x t, z t } s an optmal soluton of the problem 5). In practce, t s often the case that ADMM converges to modest accuracy wthn a few tens of teratons [6]. B. Our Algorthm Our problem ) cannot be readly solved usng ADMM. The constrants 2) and 3) couple all varables together, whereas n ADMM problems the constrants are separable for each set of varables. The couplng s especally dffcult, because t happens on two orthogonal dmensons smultaneously: The per-clent workload conservaton constrant 2) couples α across stub datacenters, and the per-stub datacenter capacty constrant 3) couples α across clents. To address ths challenge, we ntroduce a new set of auxlary varables β = α, and re-formulate the optmzaton: mn D α P E α,β I J J U J α L )) + J α =, I, β D C, J, I D β P B I α = β 0, I, J. 0) Ths problem 0) s clearly equvalent to the orgnal problem ). We observe that the new formulaton s n the ADMM form 5). The obectve functon s now separable over two sets of varables α and β. α controls the net utlty gan of processng the requests,.e. utlty mnus electrcty cost, whle β determnes the bandwdth cost of transmttng the response packets. α and β are connected through an equalty constrant. Overall, they control the provder s total utlty gan of runnng the cloud servce. The use of auxlary varables also enables the separaton of per-clent and per-stub datacenter constrant sets, whch s the key step towards decomposng the problem as we demonstrate now. The augmented Lagrangan of 0) s L α, β, λ) = ) ) D α P E U α L ) + D β P B + λ α β ) + /2α β ) 2). The dual problem s solved by updatng α and β sequentally. At the t + )-th teraton, the α-mnmzaton step nvolves solvng the followng problem accordng to 7): mn α α D P E + λ t + α 2β t ) ) D Uα ) 2 α =, Uα ) = U α L ), α 0,, 2) where α s the vector of α for clent, and Uα ) s a shorthand for s utlty functon. Ths problem s decomposable over clents, snce the obectve functon and constrants are separable over. Effectvely, each clent needs to ndependently solve the followng sub-problem: mn α α D P E + λ t + 2 α 2β t ) ) D Uα ) α =, Uα ) = U α L ), α 0. 3) The per-clent sub-problem s of a much smaller scale, wth J varables and J + constrants, and can be effcently solved by a standard optmzaton solver. As dscussed n Sec. I, n realty the number of stub datacenters J = O0 2 ) and s much smaller than the number of clents I. Dependng on the exact shape of the utlty functon, n some cases we can even provde analytcal soluton as we shall see n Sec. III-D. We have solved the α-mnmzaton step dstrbutvely across all clents by decomposng the problem 2) nto I per-clent sub-problems 3). After obtanng α t+, the β-mnmzaton step can also be smlarly attacked as we show now. Accordng to 8), the β-mnmzaton step conssts of solvng the followng: mn β β D P B λ t + 2 β 2α t+ ) ) β D C,, β 0,,. 4) Ths problem s also decomposable over the set of stub datacenters J nto J sub-problems. Specfcally, each stub datacenter needs to solve mn β,β 2,... β D P B λ t + 2 β 2α t+ ) ) β D C, β 0,. 5) The per-stub datacenter problem s a quadratc program, whose solutons can be provded n analytcal form as follows. Lemma : At the t+)-th teraton, for all I such that λ t D P B 0, = 0. Denote the remanng set { I λ t D P B > 0} as I t+. Then for I t+ s: If I t+ λ t D P B )D C, = λt D P B, )

5 otherwse, where ν t+ = max { } λ t D P B + ν t+ ), 0, 0 s determned by the followng D = C. I t+ The proof can be found n Appendx A. Havng obtaned the optmal α t+ and, the fnal step s to perform the dual varable update: λ t+ = λ t ). 6) The entre procedure s summarzed n Algorthm. Snce the constrant set C for α s clearly bounded n our problem 0), accordng to Theorem the algorthm converges to the optmal soluton. Lemma 2: Our algorthm based on ADMM converges to the optmal soluton α and β of 0) and equvalently ). Algorthm Optmal Dstrbuted Soluton for ). Each stub datacenter ntalzes β 0 = 0, λ0 = 0, and broadcasts ts electrcty prce P E to each clent. 2. Gven β t = [β t, βt 2,...] and λt = [λ t, λt 2,...], each clent solves the per-clent sub-problem 3), and sends the optmal soluton α t+ to the correspondng stub datacenter. 3. Gven α t+ = [α t+, αt+ 2,...], each stub datacenter solves the sub-problem 5) as n Lemma wth local nformaton P B and λ t = [λt, λt 2,...]. 4. Each stub datacenter updates the dual varables λ t = [λ t, λt 2,...] as n 6). It then sends the optmal soluton and updated dual varable λ t+ to the correspondng clent. 5. Return to step 2 untl convergence. Intutvely, the workng of our algorthm follows a dvdeand-conquer paradgm. Recall that α controls the net utlty gan of processng the requests, whle β determnes the bandwdth cost of transmttng the response packets. Our algorthm frst optmzes α for the mappng aspect of the problem gven the response routng soluton β t. It then optmzes β for the response routng aspect of the problem gven the prevously computed mappng soluton α t+. The dual update ensures the two sets of solutons converge to the same workload management soluton, whch s also optmal. C. A Parallel Implementaton n the Cloud The dstrbuted nature of Algorthm allows for an effcent parallel mplementaton n the cloud that has abundant server resources. Here we dscuss several ssues pertanng to such an mplementaton n realty. Frst, at each teraton, each clent solves the per-clent subproblem n step 2. Ths can be readly mplemented n a parallel fashon on each server of one of the datacenters the provder owns, whch we call the desgnated datacenter. A producton datacenter typcally has O0 4 ) O0 5 ) servers [4]. Thus each server only needs to solve O0) O) per-clent subproblems at each teraton. Snce the per-clent sub-problem 3) s a small-scale convex optmzaton, the computatonal complexty s low. A mult-threaded mplementaton can further speed up the algorthm on mult-core hardware. The penalty parameter and utlty functon U can be confgured across all servers before the algorthm starts off. Smlarly, step 3 of Algorthm, whch solves the per-stub datacenter sub-problem, also has a parallel mplementaton n the desgnated datacenter. Only J servers are requred, each responsble for solvng one nstance of 5) accordng to the soluton n Lemma. It can even be mplemented on the same servers that mplement step 2 for the per-clent sub-problems. The parallel mplementaton of our algorthm thus makes t well suted n the cloud envronment. Second, our algorthm can be termnated before convergence s reached. ADMM s not senstve to step sze, and usually fnds a soluton wth modest accuracy wthn tens of teratons [6]. As argued n Sec. I, a soluton wth modest accuracy s suffcent n stuatons of flash crowds of requests and falure recovery. A provder can apply an early-brakng mechansm n these scenaros to termnate the algorthm after several tens of teratons wthout worryng about performance ssues. We fnally comment that the message passng overhead of our algorthm s also low. As a prerequste, the electrcty and bandwdth prces of each datacenter and ISP needs to be gathered at the desgnated datacenter. The fnal output of the algorthm α needs to be dssemnated to the mappng nodes and datacenters recall Fg. ). All the other message passng, for exchangng α, β, and λ amongst servers, happens n the nternal network of the desgnated datacenter, whch n many cases s specfcally desgned to handle the broadcast and shuffle transmsson patterns of HPC applcatons such as MapReduce [3]. Note that the amount of ntermedate data our algorthm produces s much smaller than the bulky data of HPC applcatons [29]. Thus the message passng overhead ncurred n the datacenter network s low. D. Case Study: Affne Utlty Functons Before concludng ths secton, we provde a case study of the workload management problem wth an affne utlty functon. Affne utlty functons are the de facto utlty functon wdely used n the lterature [2], though some studes have argued for more complcated utlty functons wth farness consderatons [3]. An affne utlty functon has the followng form: ) U α L = a α L, 7) where a > 0 s a converson factor that translates userperceved latency nto utlty e.g., revenue). Wth an affne utlty functon, the per-clent sub-problem 3) becomes a

6 quadratc program n the followng form: mn α D al + P E ) + λ t + α 2β α 2 ) ) t α =, α 0. 8) Optmal solutons can then be derved n an analytcal form through the KKT condtons. Lemma 3: At the t+)-th teraton, the optmal soluton of the per-clent sub-problem 8) wth an affne utlty functon for a gven clent s as follows. { } α t+ where µ t+ = max β t D al + P E) + λt + µt+, 0 0 s determned by the followng α t+ =. J The proof can be found n Appendx B. Essentally, ths s a system of J + equatons wth J + varables, whose soluton can be effcently computed. Thus, n the case of an affne utlty functon, the per-clent sub-problem reduces to a quadratc program and s partcularly easy to solve. IV. EVALUATION To realstcally evaluate the performance of our algorthm, we conduct trace-drven smulatons n ths secton. A. Setup Request traffc 0 7 ) Hour Fg. 2. Total request traffc of the Wkpeda traces [27]. We use the Wkpeda request traces [27] to represent the request traffc of a cloud servce. The dataset we use contans, among other thngs, 0% of all user requests ssued to Wkpeda from 3:56PM, January, 2008 GMT to 4:57PM, January 2, 2008 GMT. The predcton of workload can be done accurately as demonstrated by prevous work [22], [23], and n the smulaton we smply adopt the measured request traffc as the total demand. We assume the optmzaton s done hourly, and Fg. 2 plots the hourly request traffc of the traces for 24 hours of the measurement perod., Fg. 3. The U.S. electrcty market and our datacenter map. Source: [0]. We smulate a cloud that deploys ten datacenters across the contnental U.S. Accordng to the Federal Energy Regulatory Commsson FERC), the U.S. electrcty market s conssted of multple regonal markets as shown n Fg. 3 [0]. Each regonal market has several hubs wth ther own prcng. Thus for the ease of exploraton, we assume that one datacenter s deployed n a randomly chosen hub n each of the ten regonal markets as shown n Fg. 3. We use the 20 annual average day-ahead on peak prce as the electrcty prce for each datacenter,.e. P E, as summarzed n Table I. In the smulatons we calculate the cost by assumng that one request consumes 0W of energy on average, ncludng the server, network, and coolng energy consumpton. TABLE I 20 ANNUAL AVERAGE DAY AHEAD ON PEAK PRICE $/MWH) IN DIFFERENT REGIONAL MARKETS. SOURCE: [0]. Regon Hub Prce Calforna NP5 $35.83 Mdwest Mchgan Hub $42.73 New England Mass Hub $52.64 New York NY Zone J $62.7 Northwest Calforna-Oregon Border COB) $32.57 PJM PJM West $5.99 Southeast VACAR $44.44 Southwest Four Corners $36.36 SPP SPP North $36.4 Texas ERCOT North $6.55 TABLE II TIERED BANDWIDTH PRICES. SOURCE: AMAZON EC2 Lnk capacty requests/hour) Prcng $/request) < > Each datacenter has 3 ISP lnks. Thus the number of stub datacenters J = 30. The prces of the ISP lnks are estmated n two steps. Frst, the capacty of each ISP lnk s randomly set such that the total capacty across the 30 lnks s requests per hour. Then, the prce of an ISP lnk s determned from a tered structure based on the lnk capacty, where a lnk wth larger capacty has a lower cost. We assume a request s response packets contan MB of data on average, and use Amazon EC2 bandwdth prces n the U.S. east regon to determne the exact prce per request presented n Table II.

7 Utlty gan $0.000) CDF ADMM Hour Fg. 4. Optmal average utlty gan Latency ms) Latency ms) CDF ADMM Hour Fg. 5. Optmal average latency performance Costs $0.000/request) Costs $0.000) CDF ADMM Hour Fg. 6. Optmal average costs per request Number of stub datacenters Fg. 7. CDF of per request latency. Fg. 8. CDF of per request costs. Fg. 9. CDF of number of stub datacenters per clent. Ths setup resembles the volume dscount strategy commonly used n the ndustry. We rely on Plane [9], a system that collects wde-area network statstcs from Planetlab vantage ponts, to obtan the latency nformaton. We set the number of clents I = 0 5, and choose 0 5 IP prefxes from a RouteVews [] dump. We then extract the correspondng round trp latency nformaton from the Plane logs, whch contan traceroutes made to a large number of IP addresses from Planetlab nodes. We only use latency measurements from Planetlab nodes that are close to our datacenter locatons. Therefore the propagaton latency depends on the datacenter locaton but not on the specfc ISP lnk used. We beleve ths s a reasonable approxmaton when the geographcal dstance nstead of lnk condton domnates the propagaton delay. Now snce the Wkpeda traces do not contan any clent nformaton, to emulate the geographcal dstrbuton of requests, we splt the total request traffc among the clents followng a normal dstrbuton. The utlty functon s the smple affne functon as n 7) wth a = 0 4. That s, a request wth 00 ms latency translates to $0.0 revenue for the provder. Fnally, the penalty parameter s set to n all our smulatons. B. Performance We evaluate two varants of our algorthm n the smulatons. The frst varant, referred to as ADMM n the fgures, runs Algorthm untl convergence s reached. The second varant, referred to as ADMM-20, apples an early-brakng method and runs Algorthm for only 20 teratons. Fg. 4 plots the average utlty gan per request for the two varants. Throughout the day, we observe that ADMM-20 wth 20 teratons can acheve utlty gans close to optmum wthn $ dfference, whle the regular ADMM converges wthn 56 teratons n all the cases more on convergence n Sec. IV-C). The average value of α β after 20 teratons s merely Therefore, our algorthm converges quckly to near optmum. Fg. 5 6 further plot the average latency and servng costs per request. Observe that the average clent latency stands below 80 ms most of the tme, and never exceeds 20 ms. The average servng costs s approxmately $0.005 per request throughout the day. Both metrcs fluctuate closely wth the total traffc as shown n Fg. 2. To understand the performance of our algorthm on a mcroscopc level, we plot the CDF of the request latency and servng costs across all clents and all hours for the ADMM-20 varant n Fg. 7 and 8. Most of the requests, more than 90%, are served wth latency less than 00 ms. The CDF of costs s more skewed, mplyng that the per-request costs vary sgnfcantly across clents. Ths s because the bandwdth) cost dfference amongst the ISP lnks of the same datacenter s clearly larger than the latency dfference, whch s assumed to be zero. One may wonder at ths pont that, our algorthm may drect requests only to the best stub datacenter for a clent, whch s not preferable for dversty and reslence purposes. However, Fg. 9 shows that ths s not the case. We plot the CDF of the number of stub datacenters a clent s requests are drected to for hour 0 data and the ADMM-20 varant). The fgure shows that for more than 80% of the clents, the requests are drected to 2-5 stub datacenters. On average, each clent has 3.6 stub datacenters to serve ts requests. Ths leads us to beleve that our algorthm dstrbutes the workload n a balanced way.

8 C. Convergence CDF ADMM Subgradent Number of teratons Fg. 0. CDF of the number of teratons to acheve convergence for our ADMM algorthm and the subgradent method. We now nvestgate the convergence and runnng tme of our algorthm. For comparson, we use the subgradent method [5] to solve the dual problem of the transformed optmzaton 0) wth the augmented Lagrangan ). Specfcally, the prmal varables α and β are ontly optmzed nstead of sequentally updated as n our ADMM algorthm to speed up the convergence, and the dual varables λ are updated by the subgradent method. The step sze has to be carefully chosen, snce a too large value wll make the fnal output far away from the real optmum, and a too small value wll slow down the convergence. We choose the step szes accordng to the dmnshng step sze rule [5]. Fg. 0 plots the CDF of the number of teratons the two algorthms take to acheve convergence for the 24 runs on the traces. We can clearly see that our ADMM algorthm converges much faster than the subgradent method. Our algorthm takes at most 56 teratons to converge, whle the subgradent method takes at least 72 teratons. For 80% of the tme our algorthm converges wthn 40 teratons, whle the subgradent method takes 0 teratons. Ths demonstrates the fast convergence of our algorthm compared to conventonal methods. We fnally study the runnng tme of our algorthm. Note that snce we do not have enough hardware resources to experment wth a parallel mplementaton, our algorthm s mplemented on a sngle server machne where each per-clent and per-stub datacenter sub-problem s sequentally solved. We observe that one teraton takes on average seconds on a Dual Dual-Core Intel Xeon 3.0 Ghz 64-bt) server. Snce I = 0 5 and J = 30, solvng each sub-problem takes around 0.05 second. Thus, a parallel mplementaton on 000 servers wll take less than a second to run one teraton, whch demonstrates the effcency of our algorthm for large-scale problems. V. RELATED WORK The topcs of request mappng and response routng for a geo-dstrbuted nfrastructure are usually treated separately n the lterature. On the former, [26] ntroduced the dea of utlzng the locaton dversty of electrcty prce to ntellgently drect requests to datacenters wth lower prces. [30] developed a decentralzed request mappng algorthm wth confgurable polces. [2], [8] consdered the effect of request mappng on provdng envronmental gans by usng green energy. [3] the performance farness ssue n request mappng. On the latter, [3] developed routng algorthms to optmze performance and cost for a mult-homng ISP. [32] proposed to optmze traffc engneerng across all upstream ISPs, assumng requests are smply mapped to the closest ngress pont. The ont study of mappng and routng has started to gan attenton recently. [2] studed the data placement problem n a geo-dstrbuted cloud, consderng the data localty, bandwdth costs, and storage capacty. We assume the content s replcated on all datacenters. [2] consdered the ont problem wth bandwdth costs, and the resultng lnear program was solved by standard methods. We consder both bandwdth and electrcty costs, and develop a new dstrbuted algorthm to solve the convex optmzaton problem. VI. CONCLUDING REMARKS We studed the ont request mappng and response routng problem for geographcally dstrbuted datacenters. We formulated the problem as a general convex optmzaton, where the locaton dversty of performance and costs are modeled. We developed an effcent dstrbuted algorthm based on ADMM to decompose the large-scale global problem nto many subproblems, each of whch can be quckly solved. We dscussed a parallel mplementaton of the algorthm that s well suted n a cloud envronment wth abundant server resources. Tracedrven smulatons are conducted to evaluate the algorthm s performance. As future work, we plan to more thoroughly study ts mpact on exstng wde-area traffc engneerng schemes. REFERENCES [] [2] S. Agarwal, J. Dunagan, N. Jan, S. Sarou, A. Wolman, and H. Bhogan. Volley: Automated data placement for geo-dstrbuted cloud servces. In Proc. USENIX NSDI, 200. [3] M. Al-Fares, A. Loukssas, and A. Vahdat. A scalable, commodty data center network archtecture. In Proc. ACM SIGCOMM, [4] D. P. Bertsekas and J. N. Tstskls. Parallel and Dstrbuted Computaton: Numercal Methods. Athena Scentfc, 997. [5] S. Boyd and A. Mutapcc. Subgradent methods. Lecture notes of EE364b, Stanford Unversty, Wnter Quarter stanford.edu/class/ee364b/notes/subgrad method notes.pdf. [6] S. Boyd, N. Parkh, E. Chu, B. Peleato, and J. Ecksten. Dstrbuted optmzaton and statstcal learnng va the alternatng drecton method of multplers. Foundatons and Trends n Machne Learnng, 3): 22, 200. [7] S. Boyd and L. Vandenberghe. Convex Optmzaton. Cambrdge Unversty Press, [8] Z. Cao, Z. Wang, and E. Zegura. Performance of hashng-based schemes for Internet load balancng. In Proc. IEEE INFOCOM, [9] M. Chang, S. H. Low, A. R. Calderbank, and J. C. Doyle. Layerng as optmzaton decomposton: A mathematcal theory of network archtectures. Proc. IEEE, 95):255 32, January [0] Federal Energy Regulatory Commsson. U.S. electrc power markets [] C. Fralegh, S. Moon, B. Lyles, C. Cotton, M. Khan, D. Moll, R. Rockell, T. Seely, and S. Dot. Packet-level traffc measurements from the Sprnt IP backbone. IEEE Netw., 76):6 6, November [2] P. X. Gao, A. R. Curts, B. Wong, and S. Keshav. It s not easy beng green. In Proc. ACM SIGCOMM, 202.

9 [3] D. K. Goldenberg, L. Qu, H. Xe, Y. R. Yang, and Y. Zhang. Optmzng cost and performance for multhomng. In Proc. ACM SIGCOMM, [4] A. Greenberg, J. Hamlton, D. A. Maltz, and P. Patel. The Cost of a Cloud: Research Problems n Data Center Networks. SIGCOMM Comput. Commun. Rev., 39):68 73, [5] M. R. Hestenes. Multpler and gradent methods. Journal of Optmzaton Theory and Applcatons, 45): , 969. [6] R. Kohav, R. M. Henne, and D. Sommerfeld. Practcal gude to controlled experments on the web: Lsten to your customers not to the hppo. In Proc. ACM SIGKDD, [7] R. Krshnan, H. V. Madhyastha, S. Srnvasan, S. Jan, A. Krshnamurthy, T. Anderson, and J. Gao. Movng beyond end-to-end path nformaton to optmze CDN performance. In Proc. ACM IMC, [8] Z. Lu, M. Ln, A. Werman, S. H. Low, and L. L. Andrew. Greenng geographcal load balancng. In Proc. ACM Sgmetrcs, 20. [9] H. V. Madhyastha, T. Isdal, M. Patek, C. Dxon, T. Anderson, A. Krshnamurthy, and A. Venkataraman. Plane: An nformaton plane for dstrbuted servces. In Proc. USENIX OSDI, [20] J. Mo and J. Walrand. Far end-to-end wndow-based congeston control. IEEE/ACM Trans. Netw., 85): , October [2] S. Narayana, J. W. Jang, J. Rexford, and M. Chang. To coordnate or not to coordnate? Wde-Area traffc management for data centers. Techncal report, Prnceton Unversty, 202. [22] D. Nu, H. Xu, B. L, and S. Zhao. Rsk management for vdeo-ondemand servers leveragng demand forecast. In Proc. ACM Multmeda, 20. [23] D. Nu, H. Xu, B. L, and S. Zhao. Qualty-assured cloud bandwdth autoscalng for vdeo-on-demand applcatons. In Proc. IEEE INFOCOM, 202. [24] E. Nygren, R. K. Staraman, and J. Sun. The Akama network: A platform for hgh-performance Internet applcatons. SIGOPS Oper. Syst. Rev., 443):2 9, August 200. [25] K. Papagannak, N. Taft, Z.-L. Zhang, and C. Dot. Long-term forecastng of Internet backbone traffc: Observatons and ntal models. In Proc. IEEE INFOCOM, [26] A. Quresh, R. Weber, H. Balakrshnan, J. Guttag, and B. Maggs. Cuttng the electrcty bll for Internet-scale systems. In Proc. ACM SIGCOMM, [27] G. Urdaneta, G. Perre, and M. van Steen. Wkpeda workload analyss for decentralzed hostng. Elsever Computer Networks, 53): , July [28] V. Valancus, C. Lumezanu, N. Feamster, R. Johar, and V. V. Vazran. How many ters? Prcng n the Internet transt market. In Proc. ACM SIGCOMM, 20. [29] V. Vasudevan, A. Phanshayee, H. Shah, E. Krevat, D. G. Andersen, G. R. Ganger, G. A. Gbson, and B. Mueller. Safe and effectve fne-graned TCP retransmssons for datacenter communcaton. In Proc. ACM SIGCOMM, [30] P. Wendell, J. W. Jang, M. J. Freedman, and J. Rexford. DONAR: Decentralzed server selecton for cloud servces. In Proc. ACM SIGCOMM, 200. [3] H. Xu and B. L. A general and practcal datacenter selecton framework for cloud servces. In Proc. IEEE CLOUD, 202. [32] Z. Zhang, M. Zhang, A. Greenberg, Y. C. Hu, R. Mahaan, and B. Chrstan. Optmzng cost and performance n onlne servce provder networks. In Proc. USENIX NSDI, 200. [33] Y. Zhu, B. Helsley, J. Rexford, A. Sganpora, and S. Srnvasan. LatLong: Dagnosng wde-area latency changes for CDNs. IEEE Trans. Netw. Servce Manag., to appear, 202. APPENDIX A PROOF OF LEMMA The KKT condtons [7] of the per-stub datacenter problem 5) consttute the followng system of equatons. α t+ ) + D P B + ν t+ ) λ t τ t+ = 0,, 9) ν t+ C ) D = 0, τ t+ = 0, 20) s the optmal soluton, and ν t+ s the KKT multpler. 9) s the frst-order optmalty condtons, 20) s the complementary slackness condton, and 2) are the prmal and dual feasblty condtons. For all I that satsfy λ t D P B 0, assume > 0. Then accordng to the complementary slackness condton 20) τ t+ = 0. The left hand sde LHS) of 9) s always postve, whch contradcts the optmalty condton. Thus = 0. As n Lemma, denote the rest of stub datacenters as the set I t+. λ t D P B > 0 holds for all I t+. If I t+ λ t D P B )D C, then accordng to 20) ν t+ = 0. Ths s so because for those I t+ such 0 = 0. Then, that > 0, λ t D P B snce ν t+ n 9). Thus I t+ D C, and ν t+ τ t+ = 0 must hold for all I t+, for otherwse and the LHS of 20) s always negatve. Substtutng ν t+ and τ t+ = 0 nto 9) yelds If I t+ λ t D P B = 0 = 0 = λt DP B. )D > C, note that the +νt+ ) obectve of 5) s mnmzed at λt DP B > 0 when the capacty constrant s absent, we must have λ t DP B +νt+ ) < to conform to the capacty constrant. Snce the obectve functon ] of 5) s convex n β, for β [0, λt DP B +νt+ ) +α t+ t s ncreasng. Thus the optmal must satsfy the capacty constrant at equalty, and equal to max { λ t DP B +νt+ ), 0 }. APPENDIX B PROOF OF LEMMA 3 The KKT condtons for the per-clent sub-problem wth an affne utlty functon 8) are α t+ β) t + D al + P E ) + λ t +µ t+ σ t+ = 0,, 22) α t+ = 0, 23) µ t+ 0, σ t+ α t+ = 0, α t+ 0, σ t+ 0,, 24) where α t+ s the optmal soluton as n 7), and µ t+ and σ t+ are the KKT multpler for the equalty and nequalty constrants of 8), respectvely. 22) corresponds to the frst-order optmalty condton, 23) s one of the prmal feasblty condtons, and 24) captures the other prmal feasblty condton, the dual feasblty, and the complementary slackness condtons. Essentally, snce α t+ σ t+ never appear at the same tme n 22), α t+ max { β t D al + P E satsfy 23). Thus the proof. ) + λ t + µ t+ and ) /, 0 }, and must = C D 0, ν 0, 0, τ t+ 0,. 2)