212 IEEE Ffth Internatonal Conference on Clou Computng A General an Practcal Datacenter Selecton Framework for Clou Servces Hong Xu, Baochun L henryxu, bl@eecg.toronto.eu Department of Electrcal an Computer Engneerng Unversty of Toronto Abstract Many clou servces nowaays are runnng on top of geographcally strbute nfrastructures for better relablty an performance. They nee an effectve way to rect the user requests to a sutable atacenter, epenng on factors nclung performance, cost, etc. Prevous work focuse on effcency an nvarably consere the smple objectve of maxmzng aggregate utlty. These approaches favor users closer to the nfrastructure. In ths paper, we argue that farness shoul be consere to ensure users at savantageous locatons also enjoy reasonable performance, an performance s balance across the entre system. We aopt a general farness crteron base on Nash barganng solutons, an present a general optmzaton framework that moels the realstc envronment an practcal constrants that a clou faces. We evelop an effcent strbute algorthm base on ual ecomposton an the subgraent metho, an evaluate ts effectveness an practcalty usng realworl traffc traces an electrcty prces. I. INTRODUCTION Internet-scale onlne servces are becomng ncreasngly essental to our everyay lfe, wth mportant applcatons nclung web search, veo streamng, an onlne gamng. The burgeonng of clou computng platforms, such as Amazon AWS, further enables small enterprses to raply eploy clou servces at scale. Almost all of these clou servces are bult atop geographcally strbute nfrastructures,.e. atacenters locate n fferent regons as shown n Fg. 1, to prove relablty an performance. They nee an effectve way to rect clents across the we area to an approprate atacenter. Usually, clou servces hanle atacenter selecton by eployng mappng noes, whch are typcally DNS servers, to customze the IP aress(es) returne to fferent clents. Alternatvely, they can also outsource atacenter selecton to thr-partes [1], [2] or the clou prover [3]. Clents Mappng noes (DNS) Mappng Requests Datacenters Fg. 1. An example of a clou servce runnng atop a geographcally strbute clou nfrastructure. An effcent atacenter selecton algorthm s mperatve to the operaton of clou servces. Many prevous works exst n ths area. The problem can be cast as an optmzaton that maxmzes the total utlty or mnmze the total cost, both of whch can be efne n fferent ways epenng on how the clou servces efne performance. The formulate problems an ther solutons are focuse on the effcency ssue. These approaches thus ten to favor clents closer to the nfrastructure, because prortzng them n assgnng atacenter capacty mproves system utlty. Ths results n poor performance for savantage users far away from the nfrastructure, an can potentally lea to substantal revenue losses. For nstance, Amazon reports that every 1 ms elay n page loa tme ecreases sales by 1 percent [4]. Many farness crtera have been consere n the tratonal context of traffc engneerng n wre networks, an resource allocaton n cellular networks. Max-mn an proportonal farness moels are arguably the most wely use n the lterature [5] [7]. However, n the context of clou computng, clou servces usually have stnct Servce Level Agreements (SLAs) that nee to be satsfe. Ths requrement cannot be accommoate by nether the max-mn nor the proportonal farness moel. Moreover, the max-mn approach eals wth the worse-case scenaro an penalzes clents wth better contons, thus unnecessarly reucng the system effcency. In ths paper, our man contrbuton s a general optmzaton framework for atacenter selecton base on the farness crteron stemmng from the concept of Nash barganng games n game theory [8]. Essentally, we can vew the problem as a cooperatve barganng game amongst the mappng noes. Each noe, servng requests aggregate from a specfc area, has a mnmum utlty requrement for clents t serves base on the SLA, an they compete ( bargan ) cooperatvely for the clou resources. The soluton of ths game, calle the Nash barganng solutons (NBS), s a unque Nash equlbrum pont wth NBS farness among mappng noes, whch s a generalze proportonal farness noton. It strves to satsfy the mnmum utlty requrements frst, an allocates the remanng atacenter capacty proportonally among mappng noes accorng to ther contons. Therefore t s able to acheve hgh system effcency wth goo farness. Our framework s general n the sense that t moels practcal envronments that clou operators face. The utlty 978--7695-4755-8/12 $26. 212 IEEE DOI 1.119/CLOUD.212.16 9
abstracton encompasses many possble performance conseratons, nclung throughput an latency, as well as polcy preferences, such as atacenter loa, user localty, etc. It s a functon of user-atacenter tuples n orer to moel the locaton versty of performance. We also conser the cost of servng requests an moel the prce versty of atacenters, snce recent works have recognze the beneft of locatonepenent electrcty prce n terms of mnmzng the energy bll of atacenters [9]. By usng ual ecomposton, our optmzaton formulaton can be ecentralze to the mappng noe level. Specfcally, the problem can be ecompose nto many subproblems, each solvable by an nvual mappng noe tself. Ths enables us to evelop effcent strbute mplementatons of our atacenter selecton algorthm to fn the optmal noeatacenter assgnment, base on the subgraent metho. We also pont out that our algorthms reman relevant n an are applcable to other request recton scenaros, such as a commercal CDN. We evaluate the effectveness an practcalty of our ecentralze mplementaton usng the real-worl traffc traces collecte from UUSee [1], a commercal Veo-on-Deman prover n Chna, as well as real-worl electrcty prces collecte from the U.S. Feeral Energy Regulatory Commsson [11]. Results show that our algorthm acheves better farness wth satsfactory performance n terms of total utlty an cost, an s amenable to practcal mplementatons, snce t converges wthn 2 teratons. The rest of the paper s structure as follows. In Sec. II we ntrouce the concept of NBS an present our formulaton of the atacenter selecton problem. In Sec. III we evelop strbute algorthms to solve the optmzaton problem base on ual ecomposton. Numercal results are prove n Sec. IV. In Sec. V we summarze relate work. Fnally, we conclue the paper n Sec. VI. II. AN OPTIMIZATION FRAMEWORK BASED ON NBS In ths secton, we present our optmzaton framework base on NBS. A. System Moel We start by ntroucng the system moel. We conser a clou nfrastructure wth M geographcally strbute atacenters. The clou eploys N mappng noes (e.g. DNS servers) at fferent locatons to rect clent requests to the approprate atacenters as etermne by the atacenter selecton algorthm. Snce the request traffc fluctuates ynamcally, the atacenter selecton algorthm has to be run perocally to optmze performance. We assume that the clou operator employs learnng technques [12], [13] to prect the traffc eman of each noe D n each epoch wth satsfactory accuracy. We also assume that the electrcty prce at each atacenter W s avalable at the begnnng of an epoch, an remans statc throughout the entre epoch. Ths s a practcal assumpton n toay s electrcty market. If the local electrcty market of atacenter s a regulate utlty regon, the electrcty prce s fxe. If on the other han the atacenter s n a eregulate market regon, such as Calforna an Texas, there s a forwar market wth settlements of varous kns, such as ay-ahea an hourahea, for customers to lock n the prce [11]. W =[W ] s calle the cost matrx. We use an abstract utlty noton U to capture the performance of the clou servce, when a request from noe s recte to atacenter. Ths noton allows us a conserable amount of expressveness. For example, f the clou servce s an nteractve applcaton an seeks mnmal latency, U can be a ecreasng functon of the roun trp tme (RTT), rectly measure or estmate by varous means. If the clou servce s a bulk transfer applcaton an seeks goo throughput, U can be a ecreasng functon of the network congeston level or the lnk utlzaton. Ths utlty functon can be any shape e.g. a convex functon of the latency or throughput. For more scusson of the generalty of the utlty noton one can refer to [2]. Fnally, U =[U ] s calle the performance matrx n the followng. B. Bascs of Nash barganng solutons We present the salent concepts an results from Nash barganng solutons here, whch are use n the sequel. For etals we refer reaers to [8]. The basc settng s as follows: Let N be the set of mappng noes. Let S be a close an convex subset of R N to represent the set of feasble utlty allocatons. Let U mn be the mnmal utlty that the -th noe requre. In our atacenter selecton problem, ths s obtane from clents SLAs. Suppose {U S U U mn, N}s a nonempty boune set. Defne U mn =(U1 mn,...,un mn), then the par (S, U mn ) s calle a N-noe barganng problem. Wthn the feasble set S, we frst efne the noton of Pareto optmalty as a selecton crteron n a typcal game. Defnton 1: The pont (U 1,...,U N ) s sa to be Pareto optmal f an only f there s no other allocaton U such that U U, N, an U >U, N. That s, there exsts no other allocaton that leas to superor performance for some noe wthout nferor performance for some other noe. Our next selecton crteron s the farness of resource sharng. In ths paper, we use the NBS farness axoms from game theory. Defnton 2: r s a NBS,.e. r = φ(s, R mn ), f the followng axoms are satsfe: Feasblty, Pareto Optmalty, Inepenence of Irrelevant Alternatves, Inepenence of Lnear Transformatons, an Symmetry [8]. Theorem 1: There s a unque soluton functon φ(s, U mn ) that satsfes all axoms n Defnton 2 such that [8] φ(s, U mn ( ) argmax U U mn ). (1) U S,U U mn N NBS farness naturally fts to the atacenter selecton problem. U mn correspon to the specfc SLA of the mappng 1
noe to guarantee performance of clents. After the mnmal requrements are met for all noes, the rest of the resources are allocate proportonally to noes accorng to ther contons, so that every noe obtans a far share. When U mn =for all, NBS farness reuces to proportonal farness [7]. C. An Optmzaton Framework base on NBS Now we formally ntrouce our optmzaton framework. For our atacenter selecton problem, we wsh to conser long-term NBS farness whch epens on the average utlty. Long-term farness not only fathfully reflects clents performance, but also gves more flexblty to explot locaton versty of the clou nfrastructure. As scusse, the atacenter selecton problem can be vewe as a barganng game. Each mappng noe has the average utlty Ū as ts objectve. The goal s to maxmze all Ū cooperatvely. Each noe also has U mn that represents the mnmal utlty requrement base on the SLAs wth the clou. In the general case, fferent noes can have fferent SLAs epenng on ther locatons an thus fferent U mn. Note that the SLAs have to be satsfe n each epoch, nstea of statstcally over tme. The problem at epoch t, then, s to fn the NBS,.e. to solve for the optmzaton problem max U(t) U mn N (Ū (t) U mn ). (2) Mathematcally t s equvalent to the followng objectve: N ln ( Ū (t) U mn ). (3) Note that Ū(t) s a functon of the nstantaneous utlty U (t), typcally obtane by usng an exponentally weghte lowpass tme wnow: Ū (t) =(1 ρ w )Ū(t 1) + ρ w U (t). ρ w =(T s /T w ) where T s s the slot length an T w s the length of the tme wnow to calculate the average. Ths s ffcult to solve because the logarthm functon s not lnear. It has been shown n the semnal work [14] that maxmzng the total margnal gan of V ( X (t 1))X (t) at each t acheves long-term maxmzaton of V ( X (t)) asymptotcally. Thus, we can greatly reuce the computatonal complexty by transformng (3) to the followng lnear objectve: N U (t) U mn Ū (t 1) U mn. (4) Note that wthout conserng long-term performance, the optmzaton must guarantee farness n each epoch. However, when a tme wnow s use, the farness requrement s relaxe to the tme wnow length. Ths proves more flexblty to mprove the system effcency, by makng the current atacenter selecton relate to prevous ones. The term Ū (t 1) U mn n the enomnator of (4) serves as a weght factor to ajust the prorty of noe. If the noe has an unfarly large utlty gan from prevous epochs, ts prorty s lower n obtanng a goo atacenter n the current epoch. Therefore the long-term farness moel encourages noes to share the clou resources cooperatvely, an n turn gan more when t nees more help. In general t helps to acheve better system performance whle enforcng the farness noton over long run. Wthout loss of generalty, we use Ū to enote Ū(t 1), an rop the tme nex t n all notatons an the constant term U mn n the numerator of the objectve functon (4). The optmzaton problem at each epoch then can be formulate as DC-FAIR: N U max U Ū (5) U mn M s.t. U = p U,, (6) M p U U mn,, (7) M p =1,, p,,, (8) N p P,, (9) N p D C,, (1) M N W p D B. (11) The ecson varables are p,.e. the proporton of requests recte to atacenter from noe. The equalty (6) calculates the total utlty U gven by the atacenter selecton p an the performance matrx U. Constrant (7) s the basc SLA requrement for each noe. Constrant (8) correspons to the smple facts that all the requests at noe shoul be serve an that the ecson varable p s non-negatve. Constrant (9) moels the possble loa balancng requrement of the clou servce such that atacenter shoul at least hanle P [, 1] out of the total requests. (1) s the capacty constrant at, an (11) captures the cost constrant that the total cost of servng all the requests shoul not excee the buget B. Some may argue that some QoS parameters efne n a SLA, such as fee, responsblty, an securty level, cannot be capture by utlty. We comment that though ths s the case, ts effect on our framework s mnmal. Typcal clou servces o not have QoS guarantees on securty an prvacy, whch cannot be feasbly realze currently an stll remans an actve research topc. Even when these QoS parameters o nee to be consere, they can be easly ncorporate as atonal constrants on the request recton matrx, because the requests of a mappng noe can only be recte to atacenters that satsfy the responsblty an securty requrements. 11
III. A DECENTRALIZED IMPLEMENTATION The optmzaton problem DC-FAIR s essentally a LP, an can be solve n polynomal tme. However, ths requres a central coornator whch ntrouces a sngle pont of falure an s vulnerable to attacks. Further, the complexty of solvng the LP also ncreases sgnfcantly when the problem sze scales up, snce the number of varables s NM an the number of constrants s 2N +3M. A centralze soluton also makes t less aaptve to suen changes n traffc eman n a flash crow scenaro. Thus, for reasons of relablty, securty, scalablty, an performance, we are motvate to evelop strbute solutons n whch the mappng noes teratvely solve the DC-FAIR problem. A. Dual Decomposton Substtutng (6) nto the objectve functon (5), an relax the constrants (9) (11), we can obtan the Lagrangan of DC- FAIR: L(p, λ, μ,ν)= p U Ū + ( ) λ U mn p P + ( μ C ) ( p D + ν B ) W p D, where λ, μ, ν are the Lagrange multplers assocate wth the loa balancng, capacty, an cost constrants, respectvely. The ual functon s then { max g(λ, μ,ν)= p s.t. L(p, λ, μ,ν) p, U U mn (12) To solve g(λ, μ,ν), t s equvalent to solvng the problem wth the followng objectve ( ) U p Ū + λ U mn μ D νw D where the constant terms n L(p, λ, μ,ν) can be safely remove. The key observaton here s that t can be ecompose nto N per-noe maxmzaton sub-problems ( ) U p + λ μ D νw D max p Ū U mn s.t. p =1, p U U mn, (13) The per-noe sub-problem naturally emboes an economc nterpretaton. Each mappng noe strves to maxmze the total utlty of servng the requests, scounte by the costs of volatng the loa balancng, capacty, an buget constrants, as prce by the Lagrange multplers λ, μ,ν. The SLA constrant prevents the noe from rectng all ts requests to the most economcal atacenter, an forces t to strbute the requests among atacenters wth goo performance. Ths balances the clou operator s nterest n mnmzng ts costs an the clent s nterest n maxmzng ts performance. Note that the ecomposton greatly reuces the complexty of the optmzaton. The per-user sub-problem has only M varables an 2 constrants. In a typcal proucton clou, the number of atacenters M s much smaller than the number of mappng noes N, whch can be hunres. It s essentally a small-scale lnear program that can be solve effcently by stanar optmzaton solvers. B. A Dstrbute Algorthm We have shown that the ual functon of DC-FAIR can be ecompose nto N per-noe maxmzaton problem, whch s a smple lnear program. Now we nee to solve the ual problem mn λ,μ,ν g(λ, μ,ν) s.t. λ, μ,ν. (14) Subgraent metho [15] can be use to solve the ual problem. The upatng rules for the ual varables are as follows: [ ( λ (l+1) = μ (l+1) = [ λ (l) μ (l) + δ(l) + ɛ(l) P )] + p,, (15) ( )] + p D C,, (16) [ ( + ν (l+1) = ν (l) + σ (l) W p D B)], (17) where [x] + represents max{,x}, an δ, ɛ,σ are the step szes. Accorng to [15], the above proceure s guarantee to converge as long as the followng conton s satsfe. Proposton 1: The subgraent upates as n (15) (17) converge to the optmal ual varables f a mnshng step sze rule s followe for choosng δ, ɛ, σ [15]. Observe that, because of the ual ecomposton, ual optmzaton by subgraent metho can be one n a strbute fashon. Frst, n each teraton, the per-noe problems (13) can be solve smultaneously by the mappng noes, wth λ, μ, ν gven by atacenters. Secon, subgraent upates can also be strbutvely performe by each mappng noe. The algorthm can be perceve as an teratve barganng process. The ual varables λ, μ are transmtte from atacenters to all noes. They serve as prce sgnals to coornate the resource consumpton an loa balancng. For example, when the total traffc route to atacenter excees ts capacty,.e. p D >C, ncreases ts prce μ for the next roun of barganng to suppress the excessve eman. Smlarly, f has not reache ts mnmum loa P,.e. p <P, t ncreases the rewar prce λ to attract more traffc an therefore balance the loa 1. The process contnues untl t converges to the optmal resource allocaton. The upate metho of the other ual varable ν,.e. the buget prce, s also worth mentonng. Whle λ an μ can be nepenently upate by each atacenter wth only local nformaton, ν nees to be upate wth global nformaton 1 Note that λ s a postve term n (13). 12
from all atacenters. Ths can be one n a strbute way as follows. Intally, the prevous ν (l) s mae common knowlege among the atacenters. Frst, a atacenter s ranomly chosen an gven a token wth the total buget B. It calculates ts own monetary cost of servng the requests W p D, an euct ths amount from B. It puts a mark n the token, an pass t on to the next atacenter, who also upates the remanng buget, marks the token, an passes t further own. A atacenter etermnes t s the last one n the loop by examnng that except tself, everyone else has marke the token. It thus upates the remanng buget, calculates the upate buget prce ν (l+1), an broacasts to every mappng noe an atacenter. Algorthm 1 Optmal Dstrbute Datacenter Selecton 1. Each atacenter ntalzes λ (),μ(). ν() s ntalze to. 2. Each mappng noe collects λ (l),μ (l),ν (l), an nepenently solves the per-noe problem (13) usng stanar optmzaton solvers an obtan p. 3. Each atacenter collects ts loa p from all noes, an perform a subgraent upate for the loa an capacty prce λ (l),μ(l) as n (15) an (16). The upate λ (l+1),μ (l+1) are broacast to every mappng noe. 4. A atacenter s ranomly chosen an gven a token wth the buget B. 5. The atacenter eucts ts cost W p D from the remanng buget n the token, marks t, an passes t own. 6. Repeat step 5 untl the last atacenter calculates the fnal remanng buget, upates ν (l) as n (17), an broacasts to every atacenter an mappng noe. 7. Return to step 2 untl convergence. The complete barganng algorthm s shown n Algorthm 1. Snce t optmally solves the ual problem (14), t optmally solves the prmal problem DC-FAIR because the ualty gap for lnear programs s zero. Theorem 2: The strbute barganng algorthm as shown n Algorthm 1 always converges, an when t converges ts soluton optmally solves the atacenter selecton problem DC- FAIR. C. Dscussons We scuss some mplementaton ssues relate to our strbute atacenter selecton algorthm. Frst, step (2) an (3) of Algorthm 1 nee to be performe n a synchronze fashon across the mappng noes, whch may be of concern to some reaers for practcal mplementaton. We comment that ths can actually be one effcently. Note that the synchronzaton requrement s loose n the sense that we only requre each noe to have the complete ual varables before solvng (13) n step (2), an that step (3) to be performe after step (2) s complete for each noe. Ths can be realy acheve wthout any nee of explct or mplct tme synchronzaton, by havng each noe to wat for all the upate ual varables λ (l),μ (l),ν (l) at roun l to be receve completely before attemptng to solve the per-noe optmzaton, an by havng each atacenter to wat for all the upate solutons p (l) to be receve completely before upatng ts own ual varables. Therefore the entre proceure s naturally a self-synchronzng soluton. The concept of roun oes not nee any synchronzaton or central coornaton ether, snce each noe an atacenter knows for how many tmes t has solve the per-noe problem or upate the ual varables. Secon, we examne the message exchangng overhea of Algorthm 1. Each atacenter nees to share ts loa an capacty prce to all noes, whch mples O(MN) messages n total, each of sze 2. Each noe nees to share ts selecton ecson of sze M wth all atacenters, whch mples O(MN) messages, each of sze O(M). Thus n each roun, O(MN) messages are exchange each of sze O(M). The overhea scales lnearly wth the number of noes an atacenters, whch s practcal. IV. EVALUATION We present our smulaton stues n ths secton. A. Setup 1) Deman matrx: To represent the request traffc for a clou servce, we use the real-worl traces collecte from UUSee Inc. [1], a major commercal Veo-on-Deman prover wth servers strbute geographcally n Chna. The ataset contans, among other thngs, the banwth eman for UUSee veo programs sample every 1 mnutes, n a 12-ay pero urng the 28 Bejng Olympcs, from 14:51:58, Fray, August 8, 28 (GMT+8) to 23:43:56, Tuesay, August 19, 28 (GMT+8). Although the scale of the UUSee nfrastructure s not as large as that of a clou prover, we beleve the traces fathfully reflect the traffc eman strbuton for a clou servce, an t s approprate to use them for the purpose of benchmarkng the performance of our atacenter selecton algorthm. The precton of traffc eman can be one accurately as emonstrate by prevous work [12], [13], an n the smulaton we smply aopt the measure traffc eman as the precte eman matrx D. We use the traffc eman of stnct veo channels to represent eman of stnct mappng noes. We smulate a clou wth 1 mappng noes. Fg. 2 shows a sample of traffc eman for 3 mappng noes. Snce the ata s collecte every 1 mnutes, the optmzaton epoch s also set to 1 mnutes. 2) Datacenter placement an cost matrx: To capture the locaton versty of the clou nfrastructure an electrcty market, we assume the atacenters are eploye across the contnental U.S. Accorng to the Feeral Energy Regulatory Commsson (FERC), the U.S. electrcty market s consste of multple regonal markets as shown n Fg. 3 [11]. Each regonal market has several hubs wth ther own prcng. Therefore for the ease of exploraton, we assume that there s one atacenter eploye n a ranomly chosen hub n each 13
Deman (Mbps) 2 15 1 5 Fg. 2. Noe 1 Noe 2 Noe 3 Tme (epochs) A 1-epoch sample of traffc eman for 3 mappng noes. regonal market. We use the 211 annual average ay-ahea on peak prce ($/MWh) publshe onlne by FERC as the electrcty prce for each atacenter,.e. W, as summarze n Table I [11]. The capacty of each atacenter s ranomly set such that ther total capacty s 15 Mbps. Fg. 3. The U.S. electrcty market an our clou atacenter map. Source: FERC [11]. TABLE I 211 ANNUAL AVERAGE DAY AHEAD ON PEAK PRICE ($/MWH) IN DIFFERENT REGIONAL MARKETS. SOURCE: FERC [11]. Regon Hub Prce Calforna NP15 $35.83 Mwest Mchgan Hub $42.73 New Englan Mass Hub $52.64 New York NY Zone J $62.71 Northwest Calforna-Oregon Borer (COB) $32.57 PJM PJM West $51.99 Southeast VACAR $44.44 Southwest Four Corners $36.36 SPP SPP North $36.41 Texas ERCOT North $61.55 3) Performance matrx: Fnally, we conser a latencycrtcal clou servce, whose utlty can be efne by the negatve Euclean stance between the mappng noes an the atacenters. To calculate the performance matrx, we frst obtan the longtue an lattue of ten ranomly chosen countes n the area covere by each of the ten hubs as the exact locatons of our atacenters n the U.S. We then ranomly choose another 1 countes as the locatons of the 1 mappng noes. All the locaton nformaton s obtane from [16]. The Euclean stance between any gven par of mappng noe an atacenter then can be realy calculate, whch consttutes the performance matrx U. Wthout loss of generalty, we assume that servng 1 Gbps per epoch,.e. 1 mnutes, consumes 1 kwh electrcty. The SLA constrant for each noe U mn s set to be 3% lower than the best utlty may obtan among all atacenters. B. Effectveness We frst evaluate the effectveness of our strbute atacenter selecton algorthm. The buget B s set to $5 per epoch. Fg. 4 shows the per-noe average utlty wth total eman for a 1-epoch pero of tme. We observe that when the average utlty stans at 52 most of the tme,.e. on average the request s recte to a atacenter 52 km away, unless the total eman shoots beyon aroun 12 Mbps. Ths emonstrates that when eman s low, our algorthm matches mappng noes to ther closet atacenters, thus rectng requests to the best avalable atacenters. The consstent performance can be better explane by Fg. 5 that shows the total cost of servng requests versus total eman uner the same settng. Clearly the total cost fgure closely follows the total eman, an s below the $5 buget lmt all the tme. Thus, the buget constrant s nactve, whch correspons to a scenaro where the clou operator has ample fnancal resources to optmze performance wthout havng to conser the extra cost nvolve n ong so. Ths results n a constant average utlty curve am fluctuatng eman. When the eman becomes sgnfcant, t becomes necessary to rect some requests to stant atacenters to conform the buget constrant, whch explans the performance egraaton n epochs 74 84. To see the effectveness of our algorthm on guaranteeng SLAs, we plot U U mn for each mappng noe at three sample epochs, 1, 75, an 77 n Fg. 6. Epoch 1 correspons to a low eman epoch, an epoch 75 an 77 correspon to extremely hgh eman peros. We observe that no matter the eman, the SLA s always satsfe for each noe because the curves stay above. It can also be seen that each noe enjoys a better utlty when eman s low n epoch 1, whch s ntutve to unerstan. Now to see the effectveness of our algorthm on guaranteeng the buget, we evaluate the effect of buget on the performance of atacenter selecton. We run our algorthm wth a reuce buget of $4, whle keepng the other settngs unchange. Compare to Fg. 4, Fg. 7 shows that now the performance swngs wely along wth the eman curve, an egraes to less than 6 when eman pkes over 1 Mbps. Fg. 8 also shows that the total cost s reuce compare to Fg. 5. It s thus event that a tghter buget negatvely affects the performance, but helps reucng the operatng costs of the atacenters. Reaers may notce that urng the epochs from 74 to 84 wth a hgh volume of eman, total cost s actually beyon the $4 buget. The strngent buget constrant causes the problem 14
1 Mbps 1.4 1.2 1.8 5 54 58 62.6 Total eman 66 Average utlty.4 1 25 5 75 7 1 Tme (epoch) km 1 Mbps 1.4 1.2 1.8 5.2 4.4 3.6.6 Total eman 2.8 Total cost.4 1 25 5 75 2 1 Tme (epoch) 6 $ per 1 mnutes (km) U U mn 6 4 2 Epoch 1 Epoch 75 Epoch 78 Noe Fg. 4. Average utlty an total eman, B =5. Fg. 5. Total cost an total eman, B =5. Fg. 6. U U mn at sample epochs, B =5. 1 Mbps 1.4 1.2 1.8 5 54 58 62.6 Total eman 66 Average utlty.4 1 25 5 75 7 1 Tme (epoch) km 1 Mbps 1.4 1.2 1.8 5.2 4.4 3.6.6 Total eman 2.8 Total cost.4 1 25 5 75 2 1 Tme (epoch) 6 $ per 1 mnutes (km) U U mn 6 4 2 Epoch 1 Epoch 75 Epoch 78 Noe Fg. 7. Average utlty an total eman, B =4. Fg. 8. Total cost an total eman, B =4. Fg. 9. U U mn at sample epochs, B =4. to be nfeasble for those epochs wth extremely hgh eman. In our mplementaton, we set a stoppng crteron to be relate to the absolute change of the objectve values, so that our algorthm satsfes the buget constrant whenever t s feasble to o so, an stll prouces sensble results wth an nfeasble buget constrant. The SLA constrant s always satsfe as shown n Fg. 9, however, because t s not relaxe n the per-noe problem (13). C. Effcency an farness To emonstrate the effcency an farness acheve wth our algorthm, we use a smple LP, calle DC-OPT, that shares all the same constrants wth our DC-FAIR problem, but uses the total utlty U as the objectve functon nstea of (5) as the benchmark. Therefore t oes not conser farness. DC- OPT can also be solve by the same metho of strbute subgraent upates. Fg. 1 frst shows the farness comparson of DC-FAIR an the smple DC-OPT. We use the stanar evaton of Ū (t) U mn,.e. the tme average surplus utlty, across noes at each epoch t as the farness measure. A far selecton algorthm yels a proportonal surplus utlty allocaton among noes wth a smaller stanar evaton, whereas a poor algorthm wthout conserng farness prouces an allocaton wth a larger stanar evaton. We can observe from Fg. 1 that n most of the tme, DC-FAIR acheves a much smaller stanar evaton for the surplus utlty, whch translates to much better farness among noes. Farness s generally mprove over tme except for the hgh eman epochs, whch emonstrates the effectveness of our long-term NBS base farness moel. The performance an cost comparsons between our DC- FAIR an the DC-OPT algorthms are shown n Fg. 11 an Fg. 12. DC-OPT performs better than our DC-FAIR n terms of per-noe average utlty along the tme lne, whch s expecte snce the sole objectve of DC-OPT s to maxmze the aggregate utlty wthout conserng farness. In terms of cost, our DC-FAIR algorthm acheves slghtly better results. Overall, DC-FAIR attans a fferent performance-farness trae-off wth much better farness among noes, at the cost of system-we performance. As we scusse, farness s a crtcal requrement of the atacenter selecton problem, snce clents serve by a partcular mappng noe wsh to obtan a far share of the avalable clou resources after the mnmum SLAs are satsfe. Our algorthm thus acheves the farness requrement wth satsfactory performance. D. Spee of convergence 4 3 2 1 5 1 15 2 25 Number of teratons 5 1 15 2 25 Number of teratons Fg. 13. Hstogram of the convergence spee, B =5. gence spee, B Fg. 14. Hstogram of the conver- =4. 4 3 2 1 Fnally, we evaluate the convergence spee of the DC-FAIR algorthm. Fg. 13 an Fg. 14 show the hstograms of the number of teratons the algorthm takes to converge to the optmal soluton for fferent bugets. Clearly, we observe that t usually takes 19 2 teratons, an never takes more than 2 teratons to fnsh. For a problem wth 1 atacenters an 1 noes, the spee of convergence shoul be consere 15
Stanar evaton 95 9 85 8 FAIR OPT 75 Tme (epoch) km 5 55 6 FAIR OPT 65 Tme (epoch) $ 5 4 3 FAIR OPT Tme (epoch) Fg. 1. Farness comparson, B =4. Fg. 11. Average utlty comparson, B =4. Fg. 12. Total cost comparson, B =4. very satsfactory. Thus we beleve that our algorthm s also practcal for real-worl problems wth larger scales. V. RELATED WORK The topc of atacenter selecton an loa recton for a geo-strbute clou has starte to gan attenton n the research communty. Quresh et al. [9] ntrouce the ea of utlzng the locaton versty of electrcty spot prce to ntellgently rect requests to atacenters wth lower prces. Wenell et al. [2] evelope a ecentralze atacenter selecton algorthm for clou servces, an evaluate ts performance usng a prototype an realstc traffc traces. Rao et al. [17] consere a jont loa balancng an power control problem for Internet atacenters to explot the tme an locaton versty of electrcty prce. [18] specfcally consere the effect of geographcal loa balancng on provng envronmental gans by encouragng the use of green energy. [19] stue a complementary problem of ata placement n a geo-strbute clou, conserng the ata localty, WAN banwth costs, an atacenter capacty. These works, however, o not conser farness n ther problem formulaton. The concept of Nash barganng games has been apple to networkng problems n other omans. [2] apples t to ensure farness n a network flow control problem. Kelly n [6] has shown that Nash barganng ensures proportonal farness n a TCP settng. [7] stue a far multuser channel allocaton scheme for OFDMA networks base on Nash barganng solutons an coaltons. [21] apple NBS to a resource allocaton problem n cooperatve cogntve rao networks. We conser a new context of clou computng, the specfcs of whch are not capture n these works. VI. CONCLUDING REMARKS In ths paper, we presente a general optmzaton framework to solve the atacenter selecton problem for clou servces. Our framework s base on the concept of NBS farness to proportonally allocate the avalable clou resources across the mappng noes, after the mnmum SLA requrements are met. We aopte a ual ecomposton approach to evelop a strbute algorthm base on the subgraent metho. Our work can be extene n many rectons. One possble recton s to conser the onlne atacenter selecton wth farness conseraton, whch s more ffcult than the offlne problem we solve here. The other recton s to take nto account the stochastc nature of the request traffc, an prove stochastc performance guarantee am such ranom traffc. REFERENCES [1] DynDNS, http://yn.com/ns/. [2] P. Wenell, J. W. Jang, M. J. Freeman, an J. Rexfor, DONAR: Decentralze server selecton for clou servces, n Proc. ACM SIG- COMM, 21. [3] Amazon AWS elastc loa balancng, http://aws.amazon.com/ elastcloabalancng/. [4] R. Kohav an R. Longbotham, Onlne experments: Lessons learne, IEEE Computer, vol. 4, no. 9, pp. 85 87, September 27. [5] D. K. Golenberg, L. Qu, H. Xe, Y. R. Yang, an Y. Zhang, Optmzng cost an performance for multhomng, n Proc. ACM SIGCOMM, 24. [6] F. P. Kelly, A. K. Maulloo, an D. K. H. Tan, Rate control for communcaton networks: Shaow prces, proportonal farness an stablty, J. Operat. Res. Soc., vol. 49, no. 3, pp. 237 252, March 1998. [7] Z. Han, Z. J. J, an K. J. R. Lu, Far multuser channel allocaton for OFDMA networks usng Nash barganng solutons an coaltons, IEEE Trans. Commun., vol. 53, no. 8, pp. 1366 1376, August 25. [8] G. Owen, Game Theory. New York: Acaemc, 21. [9] A. Quresh, R. Weber, H. Balakrshnan, J. Guttag, an B. Maggs, Cuttng the electrcty bll for Internet-scale systems, n Proc. ACM SIGCOMM, 29. [1] UUSee Inc. http://www.uusee.com/. [11] Feeral Energy Regulatory Commsson, U.S. electrc power markets, http://www.ferc.gov/market-oversght/mkt-electrc/overvew.asp, 211. [12] D. Nu, H. Xu, B. L, an S. Zhao, Rsk management for veo-oneman servers leveragng eman forecast, n Proc. ACM Multmea, 211. [13], Qualty-assure clou banwth auto-scalng for veo-oneman applcatons, n Proc. IEEE INFOCOM, 212. [14] G. Song an Y. G. L, Cross-layer optmzaton for OFDM wreless network Part II: Algorthm evelopment, IEEE Trans. Wreless Commun., vol. 4, no. 2, pp. 625 634, March 25. [15] S. Boy an A. Mutapcc, Subgraent methos, Lecture notes of EE364b, Stanfor Unversty, Wnter Quarter 26-27. http://www. stanfor.eu/class/ee364b/notes/subgra metho notes.pf. [16] WebGIS, http://www.webgs.com/. [17] L. Rao, X. Lu, L. Xe, an W. Lu, Mnmzng electrcty cost: Optmzaton of strbute Internet ata centers n a mult-electrctymarket envronment, n Proc. IEEE INFOCOM, 21. [18] Z. Lu, M. Ln, A. Werman, S. H. Low, an L. L. Anrew, Greenng geographcal loa balancng, n Proc. ACM Sgmetrcs, 211. [19] S. Agarwal, J. Dunagan, N. Jan, S. Sarou, A. Wolman, an H. Bhogan, Volley: Automate ata placement for geo-strbute clou servces, n Proc. USENIX NSDI, 21. [2] R. Mazumar, L. G. Mason, an C. Doulglgers, Farness n network optmal flow control: Optmalty of prouct forms, IEEE Trans. Commun., vol. 39, no. 5, pp. 775 782, May 1991. [21] H. Xu an B. L, Effcent resource allocaton wth flexble channel cooperaton n OFDMA cogntve rao networks, n Proc. IEEE INFOCOM, 21. 16