On the impact of heterogeneity and back-end scheduling in load balancing designs

Transcription

1 O the mpact of heterogety ad back-ed schedulg load balacg desgs Ho-L Che Iformato Scece ad Techology Calfora Isttute of Techology Pasadea CA 925 Jaso R Marde Iformato Scece ad Techology Calfora Isttute of Techology Pasadea CA 925 Adam Werma Computer Scece Departmet Calfora Isttute of Techology Pasadea CA 925 Abstract Load balacg s a commo approach for task assgmet dstrbuted archtectures I ths paper we show that the degree of ffcecy load balacg desgs s hghly depedet o the schedulg dscpl used at each of the backed servers Tradtoally the back-ed scheduler ca be modeled as Processor Sharg PS whch case the degree of ffcecy grows larly wth the umber of servers However f the backed scheduler s chaged to Shortest Remag Processg Tme SRPT the degree of ffcecy ca be depedet of the umber of servers stead depedg oly o the heterogety of the speeds of the servers Further swtchg the back-ed scheduler to SRPT ca provde sgfcat mprovemets the overall mea respose tme of the system as log as the heterogety of the server speeds s small I INTRODUCTION Load balacg s a commo approach to task assgmet computer commucato systems such as web server farms database systems ad grd computg clusters I such desgs there s a dspatcher that seeks to balace the assgmet of servce requests obs across the back-ed servers the system so that the respose tme of obs at each server s arly the same Such desgs are popular due to the creased robustss they provde to bursts of traffc server falures etc as well as the heret scalablty they provde However there s also a maor drawback to load balacg desgs some performace s sacrfced Specfcally t s possble to reduce user respose tmes by movg away from load balacg desgs I ths paper we study the mpact of the scheduler at the back-ed servers e the back-ed scheduler Our goal s twofold: we wll characterze the ffcecy of load balacg desgs ad determ how ths ffcecy depeds o the back-ed scheduler; ad we wll study the mprovemets overall mea respose tme that are achevable through chagg the back-ed scheduler Our results apply to a wde varety of back-ed schedulers however our prmary focus wll be o two partcularly mportat os: Processor Sharg PS ad Shortest Remag Processg Tme frst SRPT Uder PS the server s shared evely amog all obs at the server whle uder SRPT the server s devoted fully to the ob wth the least remag work PS s ofte used as a smplfed model of the tradtoal Ths work was supported by NSF CCF Mcrosoft Research ad the Lee Ceter for Advaced Networkg schedulg desgs may computer systems cludg badwdth schedulg web servers flow schedulg routers ad CPU schedulg operatg systems SRPT provdes a mportat comparso because t has recetly bee suggested as a alteratve to PS a varety of applcatos eg [] [4] [6] [8] [23] Further SRPT mzes mea respose tme a sgle server queue [26] However t should be poted out that SRPT s ot cessarly mal load balacg systems sce the arrval process to the back-ed server s depedet o the queue state There are three ma cotrbutos ths paper Frst we show that the back-ed scheduler has a sgfcat mpact o the degree of ffcecy load balacg desgs I partcular whe the back-ed scheduler s PS the degree of ffcecy depeds larly o the parallelsm of the system e the umber of servers but whe the back-ed scheduler s SRPT the degree of ffcecy s less depedet o the system parallelsm ad more depedet o the heterogety of the speed of the servers Theorems 4 ad 5 I fact the case whe ob szes follow a Pareto dstrbuto wth fte varace the degree of ffcecy uder load balacg systems s depedet of the parallelsm of the system ad larly depedet o the heterogety of the system Secod we llustrate that the potetal mprovemet from chagg the back-ed scheduler ca be dramatc; however the degree of mprovemet depeds o the heterogety of the server speeds Corollary 3 I partcular the mprovemet mea respose tme from swtchg from PS to SRPT ca be as dramatc as the mprovemet a smple sgle server queue whe the heterogety of server speeds s small However whe the heterogety of server speeds s large the mprovemet ca be very small or eve o-exstet Thrd we provde results that facltate the aalyss of arbtrary back-ed schedulers load balacg desgs Theorems 6 ad 7 These results provde tools that are straghtforward to apply to determ the performace of a arbtrary schedulg polcy a load balacg system uder a geral servce dstrbuto Our results are eabled va two aalytc techques: algorthmc game theory ad heavy-traffc approxmatos I partcular the aalyss begs by otg that a load balacg dspatcher ca be vewed as equvalet to havg obs make ther ow dspatchg decsos greedly order to mmze ther mea respose tme Thus a deal load balacer ca

2 be vewed as rug at a operatg pot determd by the Nash equlbrum of a selfsh routg game [9] So we ca brg to bear tools from the algorthmc game theory commuty ad further the ffcecy of load balacg desgs ca be measured by the so-called prce of aarchy see Secto II for the defto However eve ths framework uderstadg the exact teracto betwee the back-ed scheduler ad the load balacer s dffcult as we hghlght Secto IV Thus we cosder the heavy-traffc regme where the load of the system s approachg ad we take advatage of recet results eg [5] [30] [32] to smplfy the model cosderably Further the heavy-traffc regme s the regme of terest sce web applcatos are typcally ru at hgh load ad addtoally the ffcecy of load-balacg desgs s tutvely worst uder heavy-traffc The remader of the paper s orgazed as follows We frst formally troduce our model Secto II The Secto III we summarze the pror lterature Next Secto IV we provde the cessary backgroud o PS ad SRPT Secto V presets the frst set of w results whch cotrast the performace of PS ad SRPT back-ed schedulers uder Pareto ob szes The Secto VI presets results characterzg the performace of arbtrary back-ed schedulers uder geral ob sze dstrbutos Fally Secto VII cocludes the paper II MODEL DESCRIPTION To study the effcecy of load balacg desgs we wll use the queueg model pctured Fgure The system cossts of parallel queues Q Q wth servce rates µ µ where µ µ + ad µ = k µ Let X be a radom d ob sze at queue havg pdf f x ad cdf F x Let F x = F x ad E[X ] := /µ The arrval process to the system s Posso wth rate Λ where Λ < µ esures stablty There s a load balacg dspatcher that probablstcally routes arrvals to queues so that the mea respose tme aka soour tme flow tme E[T ] at each queue s the same Ths model follows from the assumpto that routg decsos are made wthout observg the queue legth ad that the load balacg dspatcher reacts to perodc performace measuremets attad from each queue wth the goal of balacg respose tmes across the queues Ths kowledge lmtato of the dspatcher s a commo desg dstrbuted web servers It follows that the resultg arrval rate to Q s Posso wth rate λ ad the load at queue s ρ := λ /µ < Thus each Q s a statoary M/GI/ queue Note Λ = λ Further def the remag servce capacty aka gap at Q as γ := µ λ := γ Ths wll be a mportat cocept our aalyss Fally ote that all comg obs are routed to o of the servers e there s o balkg For reasos that we wll descrbe Secto IV we wll be cosderg the heavy-traffc behavor of ths system That s Fg Λ λ λ 2 λ μ μ 2 μ Q Q 2 Q A dagram of the load balacg model cosdered ths paper we wll aalyze the respose tme as Λ µ whch also esures that each Q s heavy-traffc e ρ We do ot vew ths as a lmtato sce the worst-case ffcecy of load balacg desgs tutvely occurs heavy-traffc Further the heavy-traffc regme s the most relevat scearo for web applcatos Iterestgly we ca take a slghtly dfferet vew of the load balacg model tha we have descrbed to ths pot whch wll prove useful I partcular we ca vew the statoary behavor of the load balacg system as the equlbrum pot of a o-atomc routg game Ths s ofte termed a selfsh routg game [9] A set {λ λ } correspods to Nash equlbrum arrval rates aka the Nash assgmet ths o-atomc routg game f for all Q f λ > 0 the E[T ] E[T ] Iformally a selfsh ftesmal of arrvg flow caot mprove ts mea respose tme f the system s at a Nash equlbrum Note that sce a load balacg dspatcher matas the same E[T ] for all Q such that λ > 0 t s operatg at a Nash equlbrum Further ths Nash equlbrum s uque I the settg of a selfsh routg game the prce of aarchy PoA s defd as the worst-case rato betwee a Nash assgmet ad the global mum Formally we wrte E[T ; λ λ k] max E[T ; λ k] st 0 {λ } < µ µ µ kµ where correspods to the mal routg structure ad λ correspods to the Nash assgmet load balacg routg structure Note the Nash ad mal assgmets are both uque o-atomc routg games We have serted a parameter k that bouds the rato of the server speeds We wll refer to k as the heterogety of the system ad we wll state bouds o the prce of aarchy terms of the umber of servers ad the heterogety k I our cotext the prce of aarchy s equvalet to the effcecy of a load balacg desg Iterestgly addto the stadard terpretato the prce of aarchy takes o aother meag ths cotext t also characterzes the beft achevable by swtchg from PS to SRPT see the dscusso of Corollary 3 for detals III PRIOR WORK No-atomc routg games were frst troduced by Pgou [22] ad later were formally defd by Wardrop [28] For ths

3 reaso equlbra these games are ofte called Wardrop equlbra The prce of aarchy was frst studed ths settg by Roughgarde & Tardos [25] ad the work that followed s surveyed [9] The fudametal result for o-atomc routg games s that the prce of aarchy s depedet of the twork structure uder a wde varety of latecy respose tme fuctos eg uder lar latecy fuctos the prce of aarchy s bouded by 4/3 regardless of the twork structure cosdered [25] However there are very few results the case of latecy fuctos specfed by queueg models as we cosder ths paper I partcular the oly boud o the prce of aarchy uder queueg latecy fuctos that s kow to hold depedetly of the twork structure holds oly whe Λ < m µ see [24] I ths case the prce of aarchy s bouded by + 2 m µ m µ Λ The reaso such results oly hold ths lmted stuato s that outsde of the lght-traffc regme the twork structure matters Recetly the frst result to characterze the mpact of the twork structure outsde of the lght-traffc regme was provded depedetly by Havv & Roughgarde [3] ad Wu & Starobsk [36] who proved that the model of ths paper the prce of aarchy s the umber of queues the system whe ob szes are Expotally dstrbuted ad obs are processed FCFS order I the curret paper our goal s to cotrast the prce of aarchy uder a varety of other backed schedulg dscpls partcularly the prce of aarchy uder PS ad SRPT Our aalyss depeds prmarly o characterzg the heavytraffc behavor of schedulg dscpls There s a large lterature studyg queueg systems heavy-traffc eg [7] [2] [33] [38] Some of these results have bee exploted to study dstrbuted system desgs that use dspatchers such as Roud Rob RR Jo the Shortest Queue JSQ ad SITA- E See [9] [34] [35] ad the refereces there However our focus ths paper dffers from these aalyses for a umber of reasos Frst we focus o characterzg the ffcecy of load balacg dspatchers rather tha smply dervg the performace of load balacg dspatchers though we derve the performace as a sde-effect Secod we derve results that apply whe the back-ed scheduler s SRPT whch has ot bee studed prevously ths cotext Thrd our results apply for arbtrary back-ed schedulers Specfcally f the heavy-traffc behavor of the polcy s kow our techque ca be appled to acheve results The works most closely related to ours are the recet papers by Wu & Dow [9] [34] whch study the heavy-traffc behavor of approxmatos of SRPT dstrbuted archtectures where the dspatcher performs a mult-layered roud rob algorthm IV BACKGROUND ON BACK-END SCHEDULERS We wll prmarly cosder two possbltes for the backed scheduler PS ad SRPT though we wll also dscuss other schedulg dscpls Secto VI PS s mportat E[T] SRPT approx ρ Fg 2 A llustrato of the valdty of the approxmato because t s commoly used as a tractable model for the tradtoal schedulg desgs may computer systems cludg badwdth schedulg web servers flow schedulg routers ad CPU schedulg operatg systems SRPT s mportat because t s kow to mmze the mea respose tme a sgle server queue [26] regardless of the arrval ad servce process Further t has recetly bee suggested as a alteratve to PS may applcatos [] [7] [23] Note however that SRPT s ot cessarly the mal back-ed scheduler for a load balacg desg I partcular because the load balacer teracts wth the schedulg polcy t s ot a pror clear that SRPT s eve cessarly a good choce for a back-ed scheduler There s a large lterature studyg each of these polces ad we refer the reader to [37] for backgroud o PS ad to [5] for backgroud o SRPT For our purposes we wll ed oly results characterzg the mea respose tme uder these polces The terested reader may fd surveys of dstrbutoal results for SRPT ad PS [6] [2] ad studes of the farss of SRPT [3] [29] A Processor Sharg PS I a M/GI/ PS queue E[T ] s [5]: E[T ] = µ λ The mportat observato here s that E[T ] M/GI//P S = E[T ] M/M//F CF S Thus the prce of aarchy uder PS follows mmedately from pror results [3] [36]: Proposto The prce of aarchy a dstrbuted system wth Posso arrvals whe the back-ed scheduler s PS s Further [3] llustrates that ths s tght Recetly ths result has bee exteded to mult-class load balacg systems [] B Shortest Remag Processg Tme SRPT The mea respose tme uder SRPT has a much more complcated form Let m x = E[X X <x] = x 0 t f tdt be a trucated -th momet of ob sze dstrbuto X Let m x = E[mx X ] = x 0 t F tdt be a dfferet trucated -th momet Fally let ρ x = λ m x Now we ca wrte the mea respose tme of SRPT a M/GI/ queue as follows [27]: x E[T ] = 0 0 ρ t dt + λ m 2 x 2 ρ x 2 df x If the dspatcher choces are fxed the SRPT s the mal back-ed scheduler However sce the dspatcher s decsos deped o the back-ed scheduler SRPT s ot mal

4 The complcated form of E[T ] uder SRPT makes t dffcult to study ths polcy drectly Istead we wll cosder the behavor of ths polcy heavy-traffc e as Λ µ I ths case recet results provde a smpler form uder certa ob sze dstrbutos cludg bouded dstrbutos [30] [32] expotal dstrbutos [4] ad Pareto dstrbutos [5] [3] For the bulk of the paper we wll lmt ourselves to Pareto ob sze dstrbutos sce these dstrbutos are commoly foud to be good models of web request dstrbutos eg see [2] [8] [0] However Secto VI we wll dscuss other ob sze dstrbutos The followg proposto follows from combg the results [5] [3] Proposto 2 Cosder a M/GI/ SRPT queue as λ µ ad X P aretoα x L wth α > e F x = x/x L α for some x L > 0 the Θ log ρ f α < 2 E[T ] = Θ log 2 ρ f α = 2 Θ f α > 2 ρ α 2 α Due to lmted space we wll focus oly o the case of α 2 though the case of α = 2 ca be hadled usg the same techques Note that whe α < 2 the ob sze varace s fte Proposto 2 oly specfes the growth rate of E[T ] wth ρ uder SRPT heavy-traffc ad we ed a smple equato to facltate our aalyss So we wll use the followg fuctoal form that ecompasses both E[T ] uder PS ad a approxmato for E[T ] uder SRPT heavy-traffc E[T ] := µ log f α < 2 m µ µ λ f α > 2 α 2 µ λ where 0 < m = α < m = uder PS Ths approxmato ca bee show usg smulatos to be very accurate for SRPT eve outsde of heavy-traffc Fgure 2 llustrates ths fact by comparg the approxmato to results from a smulato of a M/GI/ queue wth a P areto2 ob sze dstrbuto Further the approxmato matches the bouds o SRPT derved [3] Uder ths formulato we see that the cotrbuto of Q to the overall respose tme s gve by λ E[T ] whch heavy-traffc λ /µ becomes log µ C λ := λ f α < 2 m µ λ f α > 2 For a gve set of arrval rates λ λ the expected overall respose tme of the system heavy traffc s E[T ; λ λ ] := C λ V THE CASE OF SRPT AND PARETO JOB SIZES We beg our aalyss by focusg o the ffcecy of load balacg desgs whe ob szes follow a Pareto ob sze dstrbuto ad the back-ed scheduler performs SRPT Table IV-B summarzes the ma results from ths secto Let us frst cocetrate o the prce of aarchy results for SRPT Notce that whe the ob szes are Pareto wth α < 2 fte varace the prce of aarchy s k That s t depeds larly o the heterogety of server speeds k ad s depedet of the parallelsm of the system Ths s stark cotrast wth the result for PS whch states that the prce of aarchy grows larly wth the umber of servers e s Thus t becomes evdet that the overall desg of a load balacg system should be depedet o the back-ed scheduler beg used That s f o s usg PS t s mportat to avod addg too may servers whle uder SRPT t s prmarly mportat to lmt the varato betwee the servers Whe we swtch to cosderg SRPT uder Pareto obs szes wth α > 2 we see the same cotrast wth PS oly to a lesser extet Aga the parallelsm of the system s less mportat for the ffcecy of SRPT load balacg systems tha for PS os but the dfferece shrks as ob sze varablty decreases α m crease It should be poted out that the prce of aarches of SRPT ad PS are tght I partcular they are tght whe µ = kµ ad µ 2 = = µ = µ It s mportat to remember that these results characterze the prce of aarchy of SRPT uder heavy-traffc However tutvely the heavy-traffc regme should provde the worstcase prce of aarchy Our results verfy that ths s deed true uder PS sce the prce of aarchy heavy-traffc matches the overall prce of aarchy Iterestgly the prce of aarchy has a secod terpretato the cotext of ths paper Ths terpretato s the result of the followg corollary whch follows from the results for the Nash ad mal assgmets stated Table IV-B Corollary 3 uder SRPT wth Pareto ob szes havg α < 2 s the same as λ uder PS The mportace of ths Corollary s ot mmedately evdet however t provdes a alteratve terpretato of the prce of aarchy the case of Pareto ob szes wth α < 2 I partcular ths case the prce of aarchy bouds the reducto of the beft attaable by swtchg the back-ed scheduler from PS to SRPT as compared to the beft a M/GI/ queue To see ths ote that because uder SRPT s the same as λ uder PS the rato betwee E[T ] uder SRPT ad E[T ] uder PS s the same as the M/GI/ queue Further the prce of aarchy of SRPT characterzes how much worse E[T ] s tha E[T ] uder SRPT So the prce of aarchy bouds the reducto the mprovemet from swtchg to SRPT as compared to the mprovemet the M/GI/ Applyg ths terpretato of the prce of aarchy t s clear that a dramatc mprovemet s possble by swtchg from PS to SRPT load balacg systems whe k s small e sce the mprovemet of SRPT over PS the M/GI/ s large eg see [3] t s stll large load balacg systems

5 TABLE I SUMMARY OF RESULTS FOR PARETO JOB SIZES THE RESULTS FOR PS WERE FIRST DERIVED IN [3] [36] BUT ALSO FOLLOW IMMEDIATELY FROM THE RESULTS FOR SRPT Back-ed scheduler SRPT PS Job sze dstrbuto Pareto wth α < 2 Pareto wth α > 2 Geral C λ log m µ λ µ λ where m = α 2 α Optmal assgmet µ µ µ m/m+ µ m/m+ µ λ µ µ Lemma 8 Lemma 9 Lemma 9 Nash assgmet λ Prce of aarchy Satsfy λ = λ µ λ st = µ µ m /m µ m /m µ /µ µ wth µ λ /µ µ Lemma 0 Lemma Lemma µ k k m m Theorem 4 Theorem 5 Proposto whe k s small However whe k s large PS ca actually outperform SRPT I the remader of ths secto we prove the prce of aarchy results Table IV-B To prove these results we frst ed to characterze the λ ad These results are summarzed Table IV-B but we defer the dervatos to the appedx We start wth the case of fte-varace Pareto ob szes ad the move to the case of fte-varace Theorem 4 Whe C λ = log s k µ λ the prce of aarchy Proof: We beg by provg a upper boud o the prce of aarchy ad the we llustrate that t s asymptotcally tght Sce the form of λ s mplct we caot smply drectly compare E[T ] ad E[T ] Istead we wll wrte each terms of the remag servce capacty at each queue e the gaps γ = µ λ Def c to be the average respose tme for the -th queue E[T ] otce that every queue has the same average respose tme uder Nash equlbrum The we have E[T ] = c µ Note that the remag capacty at each server Q at the Nash assgmet s γ = µ e cµ Next we wll calculate E[T ] terms of c ad γ Sce the total gap s dstrbuted equally uder the mal allocato see Table IV-B we have = = µ e c Thus recallg that µ = k µ we have that E[T ] s as follows E[T µ ] = log = = log = = log + k µ e cµ k e ck µ logk + log k e ck µ Notg that k e ckµ s decreasg for large eough cµ we ca boud the thrd term above as follows: log log e cµ / whch gves E[T ] k e ck µ = cµ log logk + cµ Now we ca boud the prce of aarchy by c µ logk + cµ = k logk cµ + k k Next we wll show that ths boud o the prce of aarchy s asymptotcally tght Cosder the specfc example where µ = kµ ad µ 2 = = µ = µ The we ca aga calculate E[T ] as E[T ] = c µ = cµk +

6 ad E[T ] as E[T ] = = µ log = log µ e cµ µ µ e cµ = logµ + logk + log kµ + log µ e cµ µ e cµ Sce µ e c for large eough c c ca be chose arbtrarly large heavy-traffc we ca boud the last term above by log log e cµ = cµ µ e cµ Whe cosderg the heavy-traffc regme e c ths yelds the followg lower boud o the prce of aarchy: cµk + logµ + logk + cµ = k + logµ+logk cµ + For costat ths gves Ωk as desred k/ as c Theorem 5 Whe C λ = µ λ m the prce of aarchy s k m m Proof: Sce λ ad are so smlar see Table IV-B we ca compare the mea respose tme uder these polces drectly I partcular straghtforward calculato yelds E[T ; λ k] = E[T ; λ λ k] = m m µ m/m+ = = µ = m+ µ m m whch gves that the prce of aarchy s the soluto to the followg mzato: m We ca wrte the above more succctly usg orms as follows: max k x /m k x /m+ st k k k m x /p where y p = = yp We ca ow boud the soluto to ths mzato: k x /m k m k x /m+ k x /m+ k k /m+ m k m m k m The frst step follows from upper boudg x The secod step follows from lower boudg x k m The thrd step follows from observg that k /m+ k To see that ths boud o the prce of aarchy s asymptotcally tght let us cosder the stuato wth queues where k = k ad k 2 = = k = I ths case = k = k m m m = k m m+ m+ = m + k + k m m + k m m+ m+ m+ + k m + k m/m+ m+ + k = m+ + km/m+ Now suppose k wth k m/m+ the we have + k + km/m+ m+ k k m m+ = m k m max = k m = km /m m+ = km/m+ st k k A upper boud o the soluto to ths mzato s the followg reformulato: max = k m = x/m = k x /m+ m+ st k k k m x VI THE CASE OF ARBITRARY SCHEDULING POLICIES AND GENERAL JOB SIZES I the prevous secto we derved bouds o the prce of aarchy whe the back-ed scheduler performs SRPT ad ob szes follow a Pareto dstrbuto I ths secto we dscuss geralzatos of those results to both geral ob sze dstrbutos ad to arbtrary polces Though geralzg the schedulg polcy ad geralzg the ob sze dstrbuto seem dfferet from o aother they both have the same effect they chage the form of E[T ] For example Basal [4] recetly showed that the heavy-traffc growth rate of SRPT uder Expotal ob szes s E[T ] = θ µ λ logµ /µ λ 2

7 Smlar heavy-traffc results have recetly bee derved for polces such as Preemptve Shortest Job Frst PSJF [3] Foregroud-Backgroud schedulg FB [20] ad mult-class prorty queues [30] uder a varety of ob sze dstrbutos The dervato of these results s a actve area Our goal ths secto s to provde results that ca easly facltate the calculato of the prce of aarchy uder arbtrary schedulg polces ad ob sze dstrbutos That s we would lke to esure that as w heavy-traffc results appear prce of aarchy results follow easly We provde two such results The frst result llustrates that the prce of aarchy s determd oly by the polyomal term the heavy-traffc growth rate ad the secod result characterzes a smple stuato that ca be aalyzed to determ the prce of aarchy a very geral settg To llustrate the usefulss of these results we wll apply them to the case of SRPT schedulg wth Expotal ob szes 2 Theorem 6 Fx µ Cosder C λ such that for all ɛ 0 m there exsts λ ɛ such that for all λ wth λ ɛ < λ < µ m ɛ m C λ 3 µ λ µ λ The the prce of aarchy heavy-traffc uder C λ s the same as the prce of aarchy heavy-traffc uder m µ λ I partcular the prce of aarchy s m k m Proof: Let us refer to the terms 3 as A B ad C respectvely The we wll prove the result by showg that E[TA ] E[TB ] E[TC ] ad E[TA ] E[TB ] E[T C ] The result follows because for ay λ < µ as ɛ 0 we have that E[TA ] E[TC ] ad E[TA ] E[TC ] Frst ote that t s almost mmedate that E[T A ] E[T B ] E[TC ] I partcular λ B ca be used as the arrval rates A ad wll gve smaller mea respose tme tha E[T B ] for hgh eough Λ Further the mal arrval rates wll lead to a eve smaller mea respose tme A parallel argumet shows E[T B ] E[TC ] Secod we ed to argue that E[TA ] E[TB ] E[TC ] We wll oly argue the case of E[TA ] E[TB ] sce the remag argumet wll be symmetrc Let us beg wth the Nash assgmet B Now o at a tme we wll swtch the cost fuctos B to match those A We start wth the slowest queue Cosder what happes whe we swtch the cost fucto of a queue If the arrval rate were to rema uchaged the respose tme would drop for large eough Λ So at Nash equlbrum arrvals from other queues must shft to the queue that ust chaged Thus the overall mea respose tme drops Ths happes wth each chage so the resultg Nash assgmet case A s such that E[TA ] E[TB ] Fally sce the above orderg holds for all ɛ ad the prce of aarchy boud s cotuous m we have that B has prce of aarchy that matches C as desred Notce that a mmedate corollary of the above theorem s that SRPT has a prce of aarchy of the case of Expotal ob szes Our xt result characterzes prce of aarchy by determg a easy to study set of systems that s guarateed to cota the worst case prce of aarchy Thus t provdes a smple way to calculate the prce of aarchy Theorem 7 Cosder a system wth C λ = f ρ for some fxed o-creasg fucto f where fx fcx fy fcy for all c < y > x > 0 Let S be such a system havg queues where the servce rates of these queues are betwee µ ad kµ The there exsts a system S wth a queues of servce rate kµ ad b queues wth servce rate µ where b/a such that the prce of aarchy for S s less tha or equal to prce of aarchy for S Theorem 7 shows that to calculate the prce of aarchy t suffces to study oly systems havg queues wth two dfferet servce rates kµ ad µ Notce that ths theorem both provdes a smple way to calculate the prce of aarchy ad provdes a llustrato of the tghtss of the prce of aarchy I the case of SRPT ad Pareto ob szes the ths result led to the determato that the worst-case scearo was µ = kµ ad µ 2 = = µ = µ It also apples easly the case of SRPT uder Expotal ob szes 2 where a smple calculato shows that the worst-case s aga µ = kµ ad µ 2 = = µ = µ It the follows quckly that the prce of aarchy s The detals of ths argumet as well as the proof of Theorem 7 are omtted due to space costrats VII CONCLUDING REMARKS Server farms are ow the domat eterprse system archtecture ad load balacg dspatchers are by far the most commo desg for such systems However load balacg desgs are well-kow to be ffcet terms of the overall mea respose tme I ths paper we have provded results characterzg the ffcecy of load balacg desgs I partcular we have focused o the desg of the backed scheduler ad show that the ffcecy ad performace of load balacg systems s hghly depedet o the choce of the back-ed scheduler We showed that whe the backed scheduler s PS as s commo tradtoal desgs the ffcecy of the system grows larly wth the umber of back-ed servers I cotrast by swtchg the back-ed scheduler to SRPT the performace of the load balacg system ca be greatly mproved as log as the heterogety of server speeds s small Further the ffcecy of load balacg s less depedet o the umber of servers ad more depedet o the heterogety of the server speeds These results provde terestg maageral-level sght to the desg ad maagemet of server farms I partcular the archtecture of the server farm should be depedet o the back-ed scheduler If the back-ed scheduler s PS the deal load balacg system cossts of fewer faster servers; whereas uder SRPT the deal load balacg system cossts of a large umber of homogeous servers Ths sght begs the questo of whether usg heterogeous back-ed schedulers would be befcal? Specfcally t may be befcal to use SRPT o a large umber of back-ed servers wth smlar speeds ad use PS o the back-ed servers wth very fast/small speeds Studyg ths dea s a curret topc of research

8 The aalyss ths paper s made possble by combg deas from algorthmc game theory wth recet heavy-traffc queueg results Our results Secto VI provde a geral techque for dervg prce of aarchy results for each w heavy-traffc result that appears However o mportat ope questo that remas s how to explot heavy-traffc queueg results more geral twork structures REFERENCES [] E Altma U Ayesta ad B J Prabhu Optmal load balacg processor sharg systems I Proc of GameComm 2008 [2] M Arltt ad C Wllamso Web server workload characterzato: the search for varats I Proc of ACM Sgmetrcs 996 [3] B Av-Itzhak H Levy ad D Raz A resource allocato farss measure: propertes ad bouds Queueg Systems Theory ad Applcatos 562: [4] N Basal O the average soour tme uder M/M//SRPT Oper Res Letters 222: [5] N Basal ad D Gamark Hadlg load wth less stress Queueg Systems 54: [6] O Boxma ad B Zwart Tals schedulg Perf Eval Rev 344: [7] J Cao W Clevelad D L ad D Su Itert traffc teds toward posso ad depedet as the load creases Sprger New York 2002 [8] M Crovella ad A Bestavros Self-smlarty world wde web traffc: Evdece ad possble causes Tras o Networkg 56: [9] D G Dow ad R Wu Mult-layered roud rob routg for parallel servers Queueg Sys 534: [0] A B Dowy A parallel workload model ad ts mplcatos for processor allocato I Proc of Hgh Performace Dstrbuted Computg pages 2 23 August 997 [] M Harchol-Balter B Schroeder M Agrawal ad N Basal Szebased schedulg to mprove web performace ACM Trasactos o Computer Systems 22 May 2003 [2] J M Harrso Browa moto ad stochastc flow systems Joh Wley ad Sos New York USA 985 [3] M Havv ad T Roughgarde The prce of aarchy a expotal mult-server Oper Res Letters 35: [4] M Hu J Zhag ad J Sadowsky A sze-aded opportustc schedulg scheme wreless tworks I Globecom 2003 [5] L Klerock Queueg Systems volume II Computer Applcatos Joh Wley & Sos 976 [6] D Lu P Dda Y Qao ad H Sheg Effects ad mplcatos of fle sze/servce tme correlato o web server schedulg polces I Proc of IEEE MASCOTS 2005 [7] D Lu H Sheg ad P Dda Sze-based schedulg polces wth accurate schedulg formato I Proc of IEEE MASCOTS 2004 [8] R Magharam M Demrha R Rakumar ad D Raychaudhur Sze matters: Sze-based schedulg for MPEG-4 over wreless chals I SPIE & ACM Proceedgs Multmeda Computg ad Networkg pages [9] N Nssa T Roughgarde E Tardos ad V V Vazra Algorthmc game theory Cambrdge Uversty Press New York NY USA 2007 [20] M Nuyes ad A Werma The foregroud-backgroud queue: A survey Performace evaluato 653-4: [2] M Nuyes A Werma ad B Zwart Prevetg large soour tmes usg SMART schedulg Oper Res 56: [22] A Pgou The Ecoomcs of Welfare Macmlla 920 [23] M Rawat ad A Kshemkalya SWIFT: Schedulg web servers for fast respose tme I Symp o Net Comp ad App 2003 [24] T Roughgarde The prce of aarchy s depedet of the twork topology J Comp Syst Sc 672: [25] T Roughgarde ad E Tardos How bad s selfsh routg J ACM 492: [26] L E Schrage A proof of the malty of the shortest remag processg tme dscpl Operatos Research 6: [27] L E Schrage ad L W Mller The queue M/G/ wth the shortest remag processg tme dscpl Operatos Research 4: [28] J G Wardrop Some theoretcal aspects of road traffc research Proc of Isttute of Cvl Egers : [29] A Werma Farss ad classfcatos Perf Eval Rev 344: [30] A Werma Schedulg for today s computer systems: Brdgg theory ad practce PhD thess Carge Mello Uversty 2007 [3] A Werma M Harchol-Balter ad T Osogam Nearly sestve bouds o SMART schedulg I Proc of ACM Sgmetrcs 2005 [32] A Werma ad M Nuyes Schedulg despte xact ob-sze formato I Proc of ACM Sgmetrcs 2008 [33] R J Wllams Dffuso approxmatos for ope multclass queueg tworks: suffcet codtos volvg state space collapse Queueg Systems Theory Appl 30-2: [34] R Wu ad D G Dow O the relatve value of local schedulg versus routg parallel server systems I Proc of Cof o Paralell ad Dst Sys 2007 [35] R Wu ad D G Dow Roud rob schedulg of heterogeous parallel servers heavy traffc I Europea J of Oper Res to appear [36] T Wu ad D Starobsk O the prce of aarchy ubouded delay tworks I Proc of Game Theory for Comm ad Networks 2006 [37] S Yashkov Mathematcal problems the theory of shared-processor systems J of Sovet Mathematcs 58: [38] J Zhag J Da ad B Zwart Dffuso lmts of lmted processor sharg queues submtted for publcato 2007 APPENDIX A CALCULATING THE OPTIMAL ASSIGNMENTS I ths secto we wll calculate the mal routg assgmet the case of SRPT schedulg ad Pareto ob szes The mal dspatcher wll use a routg scheme that mmzes the overall mea respose tme E[T ]; thus the dspatcher eds to determ the m st that solve the followg mzato: C λ λ = Λ; 0 λ < µ We ca solve ths mzato explctly to obta the followg lemmas whch are summarzed Table IV-B Lemma 8 Cosder C λ = log dspatcher uses = µ µ Λ µ λ The mal Proof: Frst ote that the mal arrval rates must satsfy d C λ = d C λ { } 4 dλ dλ Ths smplfes to µ = µ { } 5 Recall that the gap at Q s γ := µ λ So the above equato gves us that the = for all It follows that = / or equvaletly = µ µ Λ

9 Notce the tuto to the above lemma: the total excess servce capacty µ Λ s dvded evely amog the servers I ths xt lemma the total excess servce capacty µ Λ s ot dvded evely amog the servers aymore stead t s dvded proporto to µ m/m+ m Lemma 9 Cosder C λ = µ λ The mal dspatcher uses m/m+ µ = µ µ Λ µm/m+ Proof: We ca aga fd the socally mal arrval rates by solvg 4 Ths gves m+ m µ µ µ = m µ from whch we obta = So we ca wrte = whch gves = µ µ µ m/m+ µ m/m+ m/m+ µ µm/m+ m+ 6 Equvaletly we have m/m+ µ = µ µ Λ µm/m+ Note that whe m = we get the mal arrval rates for PS APPENDIX B CALCULATING THE NASH ASSIGNMENT I ths secto we wll calculate the Nash assgmet the case of SRPT schedulg ad Pareto ob szes We kow that heavy-traffc all queues are used ad thus the arrval rates must satsfy E[T ] = E[T ] { } From ths codto t s possble to derve the arrval rates explctly ad we atta the results summarzed Table IV-B Lemma 0 Cosder C λ = log satsfes µ/µ λ = λ µ µ µ λ where λ s the soluto to /µ µ Λ = µ λ µ The λ Proof: At a Nash assgmet all obs must have the same expected respose tme e f λ λ represet the arrval rates at a Nash assgmet the for all { } µ log µ µ λ = µ log µ µ λ 7 From here we ca calculate λ explctly From the equato above t follows that /µ / µ γ = γ 8 Summg both sdes gves γ / = from whch t follows that γ /µ = µ / µ /µ µ/ Combg 8 ad 9 gves γ = γ = µ µ γ µ µ γ /µ µ µ/µ γ / 9 /µ whch s equvalet to the equato the statemet of the lemma Note that though the form of λ the above lemma s mplct t ca be solved easly may specal cases For stace whe µ = µ for all the Nash assgmet s the same as the mal assgmet The λ the xt lemma are explct I fact ths case λ has arly the same form as Lemma 9 Lemma Cosder C λ = m µ λ The m /m µ λ = µ µ Λ µm /m µ Proof: The Nash codto gves us that m = m µ { } 0 µ λ µ µ λ Notg that ths s parallel to 6 except that m + s ow chaged to m we ca mmedately wrte m /m µ γ = µm /m whch s equvalet to the statemet of the lemma