CPU Provisioning Algorithms for Service Differentiation in Cloud-based Environments

Transcription

1 CPU Provisioning Algorihms for Service Differeniaion in Cloud-based Environmens Kosas Kasalis, Georgios S. Paschos, Yannis Viniois, Leandros Tassiulas Absrac This work focuses on he design, analysis and evaluaion of Dynamic Weighed Round Robin (DWRR) algorihms ha can guaranee CPU service shares in clusers of servers. Our moivaion comes from he need o provision muliple server CPUs in cloud-based daa cener environmens. Using sochasic conrol heory we show ha a class of DWRR policies provide he service differeniaion objecives, wihou requiring any knowledge abou he arrival and he service process saisics. The member policies provide he daa cener adminisraor wih rade-off opions, so ha he communicaion and compuaion overhead of he policy can be adjused. We furher evaluae he proposed policies via simulaions, using boh synheic and real races obained from a medium scale mobile compuing applicaion. Index Terms CPU scheduling, service differeniaion, sochasic conrol, closed loop sysems, servers I. INTRODUCTION Along wih he adverised advanages, like scalabiliy, cos efficiency and rapid deploymen, cloud echnologies bring srong challenges o he designers of he cloud environmens. In his work, we focus on one such design challenge, namely service differeniaion, ha refers o he problem of provisioning resources among compeing users. The new challenge ha he cloud environmen brings o he able is o define and enforce meaningful, easy-o-apply Service Level Agreemens (SLAs). In he emerging virualized compuing environmen he sae-of-he-ar performance SLAs are based on CPU uilizaion, having he objecive o guaranee CPU uilizaion goals per user/server. The mos known objecive of such an SLA is he case where VMs compee for he physical CPU power and guaraneed service o each VM is required [1][2][3][4]. Apar from meeing exac guaranees, wha is encounered in pracice is a performance margin beween a minimum and maximum CPU uilizaion percenage, due o he sochasic naure of he overall server load, muli-enancy effecs, work conserving or non work conserving modes of operaion, how he algorihms handle symmeric muliprocessing and so on [2][4]. Alhough CPU provisioning is a very well invesigaed opic (e.g., in OS operaions and hread schedulers [5], hypervisor operaions [1][2][3][4], queueing sysems [6]) he imporance of dynamic conrol schemes ha are able o guaranee CPU performance/provisioning has emerged again because of he increased complexiies in oday s virualized cloud-based environmens. The reason is ha, in conras o merics like delay, CPU uilizaion is a meric ha is easily observable, even in virualized environmens and CPU power is he dominan resource ha deermines he server/sysem performance. Therefore we believe our sudy is criical o such sysems and applicaions. In his work, we examine he CPU allocaion problem of providing guaraneed service per cusomer class. We define a class of Dynamic Weighed Round Robin (DWRR) scheduling algorihms ha under some saisical assumpions can be used o provision no approximae bu exac CPU cycle percenages beween compeing users in overload. Under his class of policies he SLA is achieved a all imes, even when he demand exceeds he capabiliies of he sysem, while a he same ime hey fulfill a number of design crieria. An example SLA is he following, wihou any knowledge of saisics regarding he arrival and he service process in a cluser of servers, guaranee 20% of CPU power o requess of class A, 30% of CPU power o requess of class B and 50% of CPU power o requess of class C in he long run. Also assume ha no saisical knowledge is available regarding he correlaion beween he reques size and he service ime. The sysem model we examine is generic and hus i can be applied, wih some necessary modificaions, in boh hread scheduling in a OS kernel [5] or a hypervisor, as well as in web server operaions. We presen a moivaing scenario from a professional web applicaion environmen in Secion II. This ype of SLAs may seem rivial o saisfy. In principle, however, when we canno use knowledge abou he arrival/service process, saic algorihms are no suiable. Firs noe ha our sudy focuses on overloaded environmens. In he case of underloaded sysems (wih oal uilizaion less han 1), in a work conserving sysem he uilizaion every class will receive is equal o λ i E[S i ], where E[S i ] is he mean service ime of class i and λ i is arrival rae, independenly of he goal vecor. Then le, for simpliciy, m = 1 be he number of servers in an overloaded sysem and D he se of classes. I is well known ha under a Weighed Round Robin (WRR) policy employing weighs w i, he uilizaion Ui W RR of a given class will converge o he consan U W RR i = wi E[Si] w. j E[S j] j D Then, o achieve he goal under WRR (or similar policies) one needs o se he weighs w i such as Ui W RR equals he goal uilizaion and solve a sysem of linear equaions o find he weighs. Weighed Round Robin [7] and approximaions of he proporional-share GPS/WFQ [8] in which weighs are assigned saically, in he case where he saisics of job service ime are unknown, are no able o provide he required differeniaion and he same conclusion holds rue for

2 probabilisic/mixing policies [6][9] ha need o characerize he performance space firs. Saic open-loop schemes can only reach arbirary defined arges in he case when he arrival and service process are known in advance; as we will show in he moivaing example his is no he case in professional service-oriened web applicaions. Also, an approach based on service ime predicions according o sysem macro behavior raher han he micro behavior is subjec o predicion errors [10] and usually is unable o provide predicable services. The policies used o provide his kind of differeniaion mus, herefore, be dynamic in naure; hey require a closedloop, feedback-based sysem o make heir decisions. Neverheless, dynamic policies incur communicaion and compuaional overhead and are no guaraneed a priori o achieve heir objecive. For example, hey may exhibi oscillaions and hus may never reach a seady sae, while i is possible ha afer long ime any acion aken may have mild effecs on he sysem behavior [11] [12]. This propery ranslaes o slow speeds of convergence. In summary, our modeling assumpions and analysis echniques are as follows. We assume general arrival and general service ime processes wih unknown saisics. Our conrols are feedback-based, dynamic variaions of Weighed Round Robin policies. The reason for his choice is he superior convergence properies of such conrols as compared o probabilisic ones [13]. We noe ha we canno use sae-space models analysis since he parameers of he feedback-based sysem are sochasic [14]. In our approach, we examine convergence properies of our conrols by applying ools from sochasic analysis. In our model here is no need for (fair queueing) virual ime calculaions [8][15], and similar o [12], we focus on work-conserving, non-preempive conrols ha do no depend on fuure arrivals and service imes. Relaed analysis ools (namely Lyapunov analysis in sysems where negaive drif is applied o change prioriies) were used in works like [11], [12] and [16]. Our conribuions are he following. We firs prove analyically ha a rich class of proposed conrols can achieve he desired objecive of he SLA (Theorem 1 and Corollary 1) and we show ha he speed of convergence o he desired goals is sub-linear (Theorem 2). In order o avoid mahemaical echnicaliies in he proofs, we assume heavy raffic (e.g., overloaded siuaions). We hen evaluae he conrols under more realisic assumpions, in erms of heir convergence speed, he required overhead and heir insensiiviy o sysem and saisical assumpions. The evaluaion is done via simulaions; we use boh synheic and real races obained from a medium scale mobile applicaion. The policies are designed wih he daa cener adminisraors in mind: hey are configurable, providing hem wih rade-off opions, so ha he communicaion and compuaion overhead of he policy can be adjused o specific environmens. We show how he weigh selecion in such schemes affecs he magniude of oscillaions bu no convergence o he goal. The paper is organized as follows: in Secion II, we describe a moivaing scenario for he objecive we consider; in Secion Fig. 1: Muliier daacener archiecure. In fron of he servers ha execue he main applicaion processing, ESBs are deployed o perform specialized preprocessing funcions. III, we describe he sysem model, he deailed provisioning objecives and he corresponding mahemaical model; in Secion IV, we define a class of scheduling policies ha mees he objecives and ouline policies of his class; in Secion V, we discuss implemenaion consideraions, namely speed of convergence and overhead-performance rade-offs; in Secion VI, we evaluae he performance of he proposed policies; we summarize previous relaed work in Secion VII and conclude our sudy in Secion VIII. II. A MOTIVATING EXAMPLE We provide he moivaion for his sudy using a real scenario, where a global mobile markeing services company (referred o as MobileBroker) is acing as a service broker beween differen mobile carriers. A. The Service Broker applicaion scenario The MobileBroker houses is applicaions in a muli-ier daa cener and one such applicaion is he launching of a promoion campaign, where subscribers from hree large mobile carriers conribue o burss of requess. The MobileBroker has applicaions developed according o service oriened principles and upon receiving a reques (e.g., a single SMS message) inegraion procedures, XML ransformaions ake place and a workflow is execued. A number of services like examinaion of he profile of he requeser and/or he device are called and based on business crieria a response or muliple responses are sen back o he end user hrough he carrier. The MobileBroker IT deparmen has also adoped virualizaion and cloud compuing echnologies. Virual servers in our example are iered. In he firs ier virual servers are hosing Enerprise Service Bus (ESB)s [17] and are deployed o perform specialized preprocessing funcions (e.g., firewall services, or proocol bridging and inegraion funcions), while in he second ier virual servers execue he main applicaion processing. Fig. 1 provides a high-level descripion of he sysem archiecure ha we consider in his paper. In pracice, he acual sysem ha mus be buil o suppor hese procedures in large scale campaigns, wih millions of subscribers (e.g., during concurren Olympic game evens), is quie complex. All he cusomers from all he carriers reques for a service ha will enable hem o paricipae in he campaign. In Fig. 2(a) inpu raffic paerns (i.e., he number of SMS messages) from all carriers are depiced for a single day, from real saisics from he MobileBroker s campaign applicaion. In Fig. 2(b) he arrival paern is analyzed per mobile carrier for 2

3 (a) (b) (c) Fig. 2: Incoming raffic from hree carriers are depiced in a real mobile promoion campaign. he same day and in Fig. 2(c) he aggregaed inpu saisics are presened for he whole monh 1. B. The user raffic, he SLA and he design challenge The main moivaion behind his work and he seleced SLA is ha, in complex environmens, like a professional ESB environmen in our example, i is no easy o correlae acual processing ime wih he ype of reques or wih he reques/packe size. Saisics abou he iner-arrival and service ime for a cusomer s requess are, in general, unknown in pracice and may no be easily esimaed or correlaed o merics ha are easily observable in his service processing environmen. For example, parsing a small XML file may need more CPU cycles han a huge XML file, depending on he service requesed and he ieraions needed [17][18]. Wih empirical daa abou he service ime, when esimaing he momens from he sample we can derive a good esimae for he mean bu he esimaes for variance and higher momens are no accurae [19]. Wih he feedback-based sochasic conrol approach we develop in his work, we avoid dependencies on predicion errors regarding he service process evoluion. C. The proposed SLA The MobileBroker can agree wih he cliens o offer a differen service qualiy level depending on he sysem load. The main idea is ha under load sress, he sysem canno effecively guaranee he same level of performance as he normal load merics describe. An example of such a normal load meric ha can be quanified under normal load is delay. Alhough i is quie challenging o offer guaraneed end-oend delay performance o each cusomer class, in pracice delay is no a meric ha an adminisraor can guaranee, especially when alking abou overloaded sysems, where queueing heory canno provide safe bounds. In principle, in such a highly dynamic environmen, fine-uned merics like delay, response ime or availabiliy are subjec o unconrollable parameers like overall (physical and virual) server load, memory managemen, OS hread scheduling, muli-enancy effecs, oupu and inpu raffic in queueing services or general nework issues. An SLA wih alernaive merics of performance mus be clearly chosen. In his work, we sudy an objecive where 1 All figures depic acual daa received by he MobileBroker service. guaranees on CPU sharing are provided o he users. An example could be cas along he following lines: during overload condiions, carrier A ges 50% of he oal CPU power in he server ier, carriers B and C ge 25% each. We formalize his problem in he following, using an absrac sysem model. III. SYSTEM MODEL & PROBLEM STATEMENT A. Sysem model The sysem we consider is depiced in Fig. 3 and consiss of one conroller, locaed in fron of he CPU servers, a se M = {1,..., M} of servers and a se D = {1,..., D} of service domains (i.e., cusomer classes). The conroller mainains a FIFO queue for each service domain. Every server works on one reques a a ime. Upon compleing he reques, he server sends a feedback signal o he conroller, which chooses a domain and hen forwards he head-of-line reques from he domain s queue o he server. For modeling simpliciy, we assume ha all feedback signals and all requess are ransmied insananeously. Also preempion a he servers is no allowed and conrol decisions are aken only a ime insances where a server complees a reques. Every domain i has a coninuous ime, GI arrival process wih finie rae λ i. The service ime of a reques of domain i is described by a random variable S i wih general disribuion, mean E[S i ] and finie variance. Moreover, in order o avoid unnecessary echnicaliies in he proofs, we assume ha here exiss finie Si max such ha 0 < S i S max i <, a.s. We furher assume ha he service imes of any wo requess are pairwise independen. In pracical sysems hese assumpions describe well he service ime in professional ESB operaions where subsequen requess from he same domain may execue a differen workflow or refer o differen service, inside he ESB server. B. Problem saemen A scheduling policy (or simply policy) π is a rule ha deermines he conrol acions across ime. As explained above, an acion akes place a any insance ha a reques is compleely served, and he acion consiss of choosing a 3

4 domain whose head-of-line reques is forwarded o he server which jus finished service. Definiion 1 (Uilizaion of domain i): We define Ui π (), he uilizaion of domain i up o ime, under policy π as: Ui π oal received service by i up o in he cluser () M (1) where M servers operae in he sysem. We hen define he performance meric of he alloed CPU o each domain i. Definiion 2 (Average CPU alloed o domain i): The alloed percen of CPU capaciy o domain i is Ũ i (π). = lim inf U π i (), a.s. (2) Technical SLA (T-SLA): Le p i, i = 1,..., D, be given sricly posiive consans ha sum up o 1, where p i represens he percen of CPU resource ha he adminisraor wishes o allocae o domain i over a large ime period. Le p i = λi E[Si] M ; p i denoes he maximum achievable uilizaion for domain i, obained when all of is requess were served, where an infeasible goal represens a resource demand ha exceeds he sysem capabiliy. The T-SLA is formally expressed by he following condiion: Ũ i (π) min{ p i, p i } (3) From a modeling perspecive, a highly overloaded sysem can be absraced as an ideally sauraed sysem in which here is always raffic o be served in all he queues by all he domains. Then he offered load always exceeds he desired allocaion p i, and eq. (3) becomes Ũ i (π) p i. We provide deails on he ideally sauraed assumpion in he following secion. C. Desired policy properies A desirable scheduling policy has several properies: a) i achieves he T-SLA, meaning ha i guaranees specific CPU uilizaion percenages, b) i is agnosic o he arrival and service saisics, c) i converges o he T-SLA fas, and, d) requires a small amoun of calculaions per ime uni. Apar from (a) which is obvious, he knowledge of service saisics (b) and he decision load (d) are boh relaed o he communicaion overhead and CPU coss and are very imporan consideraions for pracical conroller sysems. Also (c) is crucial for achieving he arge wihin shor periods. To he bes of our knowledge, no single scheduling policy exiss ha is superior in all hese properies; we invesigae policies ha excel in some of he above crieria and give he designer he abiliy o rade off in order o saisfy he res. IV. CLASS Π OF DYNAMIC POLICIES ACHIEVING THE SLA We propose a specific class of policies which work using he concep of a round. Insead of making one decision upon each service compleion, we bundle ogeher a number of decisions and fix hem in a vecor. Then as he servers become empy, he nex unused decision in he vecor is chosen. This approach has Fig. 3: Sysem model. One conroller is hosed in a rouer and is locaed in fron of he CPU servers. I mainains a FIFO queue for each service domain, where every server works on one reques a a ime in non-preempive fashion. he obvious disadvanage ha some decisions are made earlier han normal, and hus wih less informaion abou sysem sae. Neverheless, round-based policies are very popular in highspeed schedulers because he round provides he CPU of he scheduler wih enough ime o make he scheduling decisions. This way he scheduling overhead does no sall he sysem operaion. A. Assumpions and Terminology We define n R +, o be he ime insan when round n begins, n N +. A each round n, wi n requess of domain i are serviced, where wi n s are seleced by he scheduling policy. Le W ( n ) = [wi n ] be a conrol vecor responsible for he weigh allocaion decisions aken a any insan n, in he beginning of round n; le X i ( n ) denoe he queue size for domain i a ime insan n. Le k( n, π) be a posiive ineger chosen by policy π in effec, and furher consrained by k( n, π) K where K is an arbirary, finie consan. (The bound consrain is crucial in he proofs.) Definiion 3 (Class of policies definiion): We define Π as he class of policies π whose rules saisfy he following: (i) w n i = 0, if U i ( n ) > p i or X i ( n ) = 0 (ii) w n i = min{k( n, π), X i ( n )}, if U i ( n ) p i, X i ( n ) > 0 In he special case where for all he domains j wih U j ( n ) p j we have X j ( n ) = 0, hen he policy selecs a random a domain i wih wi n = min{k( n, π), X i ( n )}. Noe ha he class Π conains a large number of scheduling policies, since k( n, π) can depend on he round and since he definiion above allows serving he domains in an arbirary order. Fig. 4: Service ime example. We denoe Si n as he oal service ime domain i received during round n (sum of individual service imes). This random variable is independen of how we reached ime n and depends on he policy in effec, while i follows some 4

5 probabiliy disribuion, ha depends on (bu no he same as) he disribuion of he reques service ime. Le us denoe by S i (j) he service ime of he j h reques of domain i receiving service wihin round n; S i (j) is idenically disribued wih S i. By definiion, up o K requess can be served per domain per round so for he sum of he service ime random variables he following inequaliy holds : 0 < S n i K S max i <, a.s. (4) Le L n denoe he service period during round n; hen L n = S n i:wi n 1 i le also l n denoe he idle ime ha may occur beween round n and round n + 1. We assume only sysems where l n < l <, a.s., (where l R + noes he maximum observable idle ime or maximum observable inerarrival ime) and he round lengh is bounded. An example of he round ime evoluion is shown in Fig. 4. An overloaded server sysem, ideally sauraed, is clearly relaed wih he way we define busy periods. In queueing heory, a busy period is he ime beween he arrival of a cusomer o an idle server and he firs ime he server is idle again. In he ideally sauraed case we consider, we assume an infinie busy period where we exend his concep and furhermore we assume a any conrol insan n, X i ( n ) > 0, i D (here is an available reques by every domain waiing for service), wihou however aking ino accoun any load variaions, burs sizes or arrival process and service ime disribuion deails. This modeling assumpion is basic for he proof of convergence analysis of Theorem 1. Definiion 4 (Ideally sauraed condiions): We say ha a domain is sauraed if here always exiss a leas one job waiing o be serviced. The sysem has ideally sauraed condiions if all domains are sauraed. Noe also ha, in ideally sauraed condiions he conrol vecor updae rules are simplified o: { wi n k(n, π), if U = i ( n ) p i (5) 0, if U i ( n ) > p i, B. The main resul: Convergence o he goal vecor From eq. (1), we can easily see ha, for a period of ime R +, < during which domain i is in service for ime T R +, T he uilizaion funcion evolves according o: U i ( + ) = + U i() + T + The policy π samples a poin U i ( n ) from U i () a he beginning of round n; for every domain i he sequence of sampled poins evolves according o: U i ( n+1 ) = where he coefficien (6) n Si n U i ( n ) + 1 {w n i 1} (7) n+1 n+1 n n+1 is he same for all he domains S and he erm (1 {w n i 1} n i n+1 ) depends on he conrol policy. Alhough eq. (7) reminds one of low pass filer operaions, he abiliy o avoid oscillaions and furhermore converge o Algorihm 1 OBG Algorihm Descripion n : beginning of round n X i ( n ) : queue size a n for domain i Calculae U i ( n ) if X i ( n ) > 0 and U i ( n ) p i hen w n i = 1 else w n i = 0 end if if for all i:u i ( n ) < p i, X i ( n ) = 0 hen w n m = 1, for random m where X m ( n ) > 0 end if he goal almos surely is due largely o he designed policy operaions. The following heorem saes ha all he policies ha belong in class Π converge o he goal and hence saisfy he T-SLA. Theorem 1 (Convergence o he objecive): Under ideally sauraed condiions any policy in class Π saisfies he T-SLA defined in eq. (3); moreover he following holds, Ũ i (π) = p i, a.s., π Π. (8) See he Appendix for he proof of his heorem. 1) Example policies: Two example policies ha belong o he class of policies defined are described below. We sudy heir properies exensively, in subsequen secions. Serve All Suffering (Overloaded Below Goal - OOBG) Under he OOBG policy, in round n (ha sars a ime n ), se wi n = 1 for all he domains where Ui OOBG ( n ) p i ; for he res se zero weighs. In oher words, a every round only he domains ha have alloed CPU ime less han or equal o heir T-SLA arge are given one service chance, regardless of any oher consideraion (for example, regardless of how far a domain is from is arge); he res are given none. Requess are served in a limi service fashion according o [20] (ransmi all wi n and move o he nex valid queue). We noe ha he same resuls hold whichever he order of service is. To make OOBG work wih non-sauraed arrivals, W (n ) is chosen in he case ha no domain is below goal according o he general rules defined. The corresponding algorihm is now called OBG and selecs all he suffering domains for a round and serves one reques per domain. The formal descripion is oulined in Algorihm 1. In Secion VI, we show by simulaions ha OBG can be used o mee he T-SLA. Serve The Mos Suffering (Overloaded Only he Mos Suffering - OOMS) Under he OOMS policy, each round is composed of only one domain which is served once. The seleced domain is he one wih he larges amoun of missing service. When more han one domains have he same deviaion from he goal, he policy selecs one of hem a random. More precisely, a a decision insan n when he n round begins, we se wi n = 0, i j and w j ( n ) = 1 where j = arg min k D {Uk OOMS ( n ) p k }. OOMS resembles he family of maximum weigh policies [21],[22]; ha are well- 5

6 known opimal policies and can be hough of as a degenerae case of he Round Robin scheduling policies. V. IMPLEMENTATION CONSIDERATIONS In ypical daa cener implemenaions, he conroller can be housed in any device ha erminaes TCP/UDP connecions (e.g., a core rouer, an hp rouer/sprayer in he swich fabric, or a preprocessing server in he service ier). In addiion, in our model, he queues and he conroller are locaed ouside he server ier. By keeping queues a he conroller side and making decisions a service compleion insans, one makes sure ha he conrol acion is aken wih he full sysem sae known. If insead, we queued requess in he servers, he conrol decisions would be made much earlier han he service insan and laes informaion abou he aggregae CPU imes could no be used. The insananeous feedback and reques ransfer assumpion would hold rue in a pracical deploymen of our sysem, if a small ping-pong buffering mechanism is used in he servers and he server-conroller inerconnecion is fas, compared o CPU processing imes. Such a buffer would hold one o wo requess a a ime, ensuring ha he server does no go idle. A. Rae of convergence analysis In pracical consideraions, we are ineresed no only for he convergence of he sequence, bu also how fas he sequence convergences o is arge. Theorem 2 (rae of convergence): In ideally sauraed condiions, all he policies in class Π experience sub-linear convergence o he arge uilizaions. See he Appendix for he proof of his heorem. Noe ha here exis sandard echniques in he lieraure ha can accelerae convergence [23], bu hey are beyond he scope of his work. B. Time and round of convergence esimaion We have seen ha convergence o he desired arges can be slow (heoreically, sub-linear according o Theorem 2). Quie ofen, sub-linearly convergen sequences approach heir limi fairly fas and hen ake a long ime o achieve he heoreical value. From a pracical sandpoin, i may be saisfacory o know ha wih high probabiliy, we have come close o he arge. In his secion, we invesigae heurisically he ime and he number of rounds ha is required o ener a sphere of convergence around he arge. We will ry o find n 0 such ha, wih high probabiliy, g i ( n ) = U i ( n+1 ) U i ( n ) ɛ, n > n 0 (9) We begin by wriing he following expression, based on eq. (18) and following he analysis in he proof of convergence: g i ( n ) = U i ( n+1 ) U i ( n ) D j K S max j + K S max i n+1 (10) where we remind ha K Si max noes he maximum observable service ime for all he requess of domain i and in all sample pahs. If now we wan U i ( n+1 ) U i ( n ) < ɛ i D, hen from eq. (10), we can wrie (D + 1) max i D {K Si max } ɛ n0 (11) n+1 n0+1 (D + 1) max i D{K Si max } ɛ The ime n0+1 is an esimaion for he convergence ime of all he domains o converge in a sphere of deviaion ɛ. 1) Round of convergence n 0 esimaion: Furhermore, an esimaion on he round of convergence can be also made. If we know ha he ime of convergence is lower bounded when L n is maximized, hen he round of convergence is upper bounded when every round has he minimum duraion (larges possible ime of convergence wih he maximum number of rounds). This happens when only one domain paricipaes in he round and his happens wih he minimum service ime. According o his, in a wors case scenario, he round of convergence n 0 in a sphere of deviaion ɛ is equal o n 0 n0+1 min i D,over all n {Sn i } (12) In he evaluaion secion an exensive se of experimens validaes also ha his round of convergence limi is safe. C. Overhead Analysis and Implemenaion rade-offs Because of he recursive form of eq. (18), we do no need o mainain he whole uilizaion hisory and he enire sample pah of he corresponding random variables U i ( n ). Therefore, he memory requiremen of any policy in he class Π grows linearly wih he number of domains. The compuaion overhead, on he oher hand, depends on how frequenly decision insans n occur; his frequency clearly depends on he policy. Alhough policies like OOBG and OOMS enjoy he bounded-round, negaive-drif characerisics and can be used o saisfy he T-SLA, hey are very dynamic policies and in pracice hey may inroduce a high communicaion and compuaion overhead, especially in daa cener environmens wih housands of servers. Especially for he OOMS policy which mus calculae a decision a every service compleion insan, his overhead can make he implemenaion of he algorihm prohibiive. One way o address hese limiaions and avoid communicaion overhead is o keep he calculaed weighs consan for a fixed period of ime T. Under his principle, OOBGT, a variaion of he policy of OOBG is defined as follows. For an ineger number of k rounds, weighs remain he same, as long as n+k < n + T, meaning ha we se wi n = w n+1 i =... = w n+k i for any domain i. In he ime insan when for he minimum k, n+k n + T is rue, hen new weighs are calculaed for he a new round according o OOBG policy. Similarly o OOBGT, we define OOMST, he T-delayed version of he OOMS policy, in order o provide a rade-off beween overhead and performance. 6

7 (a) (b) (c) (d) (e) (f) (g) (h) (i) Fig. 5: Ideal sauraion: Service Domains requesing for predefined CPU uilizaion. Similar o Theorem 1, one can show he following resul for T-delayed versions of policies. The proof is similar and hus omied. Corollary 1 (Convergence o he objecive): Le π denoe he T-delayed version of a policy π ha belongs in class Π. In ideally overloaded condiions, he policy π saisfies he T-SLA defined in eq. (3); ha is, Ũ i (π) = p i, a.s., π. (13) We noe ha, as in all feedback-based conrol sysems, large gain resuls in quick response o deviaion from he goal bu causes large oscillaions [24]. Finally, noe ha when delayed updaes are made in he case we don have overload condiions, he work-conserving propery is los and he sysem may experience idle ime. VI. EXPERIMENTAL EVALUATION In addiion o he heoreical analysis of he previous secion, we evaluae our policies based on experimens ha address pracical consideraions and overhead issues. The evaluaion procedure is based on exensive experimens using simulaed and real races. The goals of he evaluaion process are o demonsrae how sysem and saisical parameers affec convergence and provide rade offs ha aim o reduce communicaion and processing overhead. A. Ideally Sauraed condiions and performance demonsraion We firs demonsrae how sysem and saisical parameers affec he algorihm performance in ideally sauraed condiions. In all he following scenarios where ideal sauraion is presened, here is no descripion of he arrival process; 7

8 ! )*+,+,+,+- #$$ %&' ( % $ # "! #& -./!./"./#.0 ()) * +, & % $ # "! $'. /!01"01#01$02 )** +, - " " " (a) $' ' $'!' (b) & %!( "( "( (c) ' Fig. 6: Performance Comparison: Service Domains requesing for predefined CPU uilizaion. we assume ha here are always available requess waiing for service in he corresponding queues for all he domains. As we presened heoreically, he class of policies is able o handle any variabiliy on he workload, any bursiness phenomena or any differeniaions in mean values of he service ime disribuions, as long as his assumpion holds. Besides verificaion of he heoreical resuls, his sep will also give us he inuiion for he more realisic scenarios wih sochasic arrivals ha are examined in subsecions VI-B and VI-C. In Fig. 5, we presen example visualizaions of he OOBG and OOMS algorihm behavior where domains uilize a single CPU. In he scenarios presened in Figures 5(a), 5(b) and 5(c), he arge vecor is equal o (10%, 20%, 30%, 40%) and all domains are served by a single server. In Figures 5(a) and 5(b) he service ime in he CPU is exponenially disribued wih mean E[S i ] = 1 for all four domains; in Fig. 5(c) he mean is differen for every domain and varies according o he formula E[S i ] = 1+2 i. (This selecion was made o sress he convergence propery.) All figures demonsrae ha he algorihm is insensiive o service process variaions. A vas number of simulaions performed verified ha he T-SLA is saisfied as i was expeced from he heoreical analysis, and saisical parameers like he service raes, service variance and service probabiliy disribuion affec he rae of convergence bu no convergence o he arge iself. In pracical consideraions, we are ineresed in how sysem parameers like he number of servers and he number of domains affec he convergence speed. In Fig. 5(d), we presen simulaions for an environmen where we invesigae he OOBG algorihm; he number of domains is se o D = 4 and he number of servers varies. In he verical axis he oal absolue deviaion from he arge vecor is noed ( U i () p i ), i D while he T-SLA is P () = [p i ] = (10%, 20%, 30%, 40%). All he domains have exponenial service imes wih E[S i ] = 1. Increasing he number of servers leads o faser convergence since for any given ime, more rounds are compleed and he sysem has more chances of adjusing. The number of domains also has a direc impac on he form and rae of convergence, as we can see in Fig. 5(e), where he oal absolue deviaion from he arge vecor is noed in he verical axis; increasing he number of domains resuls in slower convergence. Wih more domains a round akes more ime o complee, less rounds are execued in he same period of ime and hus he policy exhibis slower convergence raes. The T-SLA for hese simulaions was specified as follows: p 1 = 20% for domain 1 and he res of he domains receive equal share from he remaining 80% of he CPU ime. In he following we invesigae empirically he accuracy of he resuls in secion V-B. In Fig. 5(f) we demonsrae he number of rounds ha are needed o saisfy wo differen SLAs, and also conras i o he lower bound ha eq. 12 provides. We presen he average number of rounds n 0 required unil U i ( n0 ) p i < 0.01 holds for every domain i. The averages are calculaed over 1,000 samples per configuraion and ploed versus he number of domains. In all he scenarios, we use one CPU and service imes wih mean E[S i ] = 1 for all he domains. The firs T- SLA requires equal share beween all he domains, while in he second T-SLA one domain receives 20% and he res share he res 80% of CPU power. We plo he lower bound of eq. 12 for Si max = Si Min. In his case he service imes are consan. From an exended se of simulaions conduced and as we can see in his figure also, differen SLAs require differen number of rounds o converge bu he upper bound ha eq. 12 is sill saisfied. Overhead and rading-off analysis: In Figures 5(g) and 5(h), we invesigae how convergence of he T-delayed versions is slowed by increasing he value of T. The simulaion scenario is D = 4, arge uilizaions are P () = [p i ] = (10%, 20%, 30%, 40%) and he mean service ime is exponenial wih E[S i ] = 1, while all he domains uilize a single server CPU. In Fig. 5(g), we verify ha alhough he conrol decisions are delayed and he sysem performance is degraded, he T-delayed version of OOBG algorihm converges o he goal vecor. A comparison of he T -delayed version wih he OOBG case is demonsraed in Fig. 5(h), where we can see ha as T increases, oscillaions persis for longer. We noe ha he one domain per round sraegy of OOMST is raher penalized in his scenario; OOBGT uilizes he idea of he round and appears superior. This can be seen in Fig. 5(i), where a comparison of convergence speed is presened 8

9 beween he wo policies for he same simulaion environmen. The convergence crierion was U i () p i 0.01 for all domains i for a leas 5,000 service evens. As we can observe, only for small values of T, OOMST is slighly superior han OOBGT. Noe, ha here exiss a value of period T where OOMST and OOBGT experience similar convergence ime and above his value OOBGT offers faser convergence han he corresponding OOMST. B. Sochasic Arrivals: Evaluaion and Comparisons In his subsecion we relax he assumpion of ideal sauraion and we presen he performance comparison beween he following schemes under sochasic arrivals: Saic Weighed Round Robin where he weighs are se proporionally o he SLA (referenced as WRR); Dynamic Weighed Round Robin (he OBG algorihm proposed); Credi-based scheduling (referenced as Credi); and a Dynamic probabilisic scheme proposed in our previous work [25] using a linear approximaion echnique (referenced as Probabilisic). The Credi scheduler is he successor of SEDF and BVT schedulers, used o schedule vcpus in he XEN hypoervisor [3][4]. The performance comparison is made under hree configuraion seups where four domains reques service from a single CPU and he goal vecor is se equal o 10% 20% 30% 40%. Seup 1- Fig.6(a): all domains can achieve heir goal and all have Poisson arrivals and exponenial service ime disribuions wih he same mean. This simple experimen is used o invesigae he speed of convergence meric. Alhough in his seup all policies evenually can achieve he SLA, he wo dynamic schemes (our proposed) OBG and Probabilisic (from [25]) converge faser o he goal. The reason is ha a dynamic scheme is able o quickly adap o load flucuaions, leading o smaller oscillaions. For he Credi-based scheduling we noe ha alhough i is able o differeniae according o he SLA, is performance is heavily dependen on he configuraion of he credi allocaion and he rese procedure [26]. These can be done in numerous ways; we chose o rese when credis for all he domains are equal o zero, allowing negaive credis, where we allocae credis based on he T-SLA objecive raio (credi equals p i muliplied by a consan for all he domains). In he Xen hypervisor, he credi updae period is 30ms. However, depending on he conex of he applicaion (e.g., web server operaion), his period can be configured in order o avoid over-allocaion more ofen han under-allocaion [4]. In some cases, he configuraion simpliciy of Dynamic Weighed Round Robin is an imporan facor when comparing wih a Credi scheduler. Seup 2-Fig.6(b): We consider a scenario wih bursy raffic and a feasible goal vecor (meaning ha λ i E[S i ] p i, i D). The arrivals follow an ON/OFF paern [27], where during he ON period hey have exponenial iner-arrivals and boh ON and OFF periods are exponenially disribued. In his experimen we showcase how fas he algorihms adap o bursy raffic o achieve he CPU goals. The same commens as before hold, where in his case larger convergence periods are required by all schemes because of he OFF periods of some Fig. 7: Implemenaion in a cusomized WSO2 Enerprise Service Bus emulaor. domains. In his case all schemes are able o differeniae (even WRR) since he goal vecor is feasible. Again OBG algorihm and he dynamic probabilisic one ouperform he Credi and he WRR schedulers. Seup 3-Fig.6(c): again a seup wih mixed workloads, where neverheless in his case no all he domains can achieve heir goals (some domains have less demand han heir assigned max uilizaion). Injeced raffic is again composed of burss of requess for some domains, while domain 1 arrivals follow a Pareo disribuion (wih shape parameer equal o 0.8). Also he service ime disribuions alhough exponenial for all, have differen means. Firs we noe ha, in his case, as we can see saic WRR has he wors performance in erms of he deviaion from he goal vecor meric. The reason is ha he policy is giving saic prioriies and hus canno capure he dynamics of load changes. The probabilisic algorihm performs beer han WRR bu worse han OBG because he ime required o esimae he correc probabiliy weighs is increased due o he bursy raffic. The correlaion beween he predicion period used and he acual idle periods experienced, resuls in probabiliy (weigh) allocaion ha leads o wrong and slow response o he idling periods. As we discussed in he analysis of Seup 1, many variaions of credi-based scheduling are possible. The performance of he one we chose is heavily affeced by idle periods. As we presen in he following secion using real races, for a differen, smooher workload variaions, OBG and Credi scheduler experience similar performance. C. Evaluaion using real races The available races a our disposal come from he MobileBroker case. In his secion, we use his daa se in order o benchmark he performance of he OBG algorihm. The implemenaion-emulaion scenario is depiced in Fig. 7. We use a cusom service mediaion applicaion ha can be used o emulae acual ESB operaions, for example WSO2 ESB [17] funcions. The goal of he following analysis is o demonsrae he performance of OBG algorihm in realisic condiions emulaing an operaional environmen. Three service domains (cliens) generae hp/soap raffic (XML-RPC requess over hp) and send i o an hp conroller (Java Servle). The hp conroller is used o a) accep his raffic in a queue per domain, b) perform scheduling decisions and send raffic o he service mediaion applicaion in he cusom ESB. The service mediaor is responsible o: process he requess, forward hem 9

10 o an endpoin service and reurn o he clien he response from he service. The rajecory of he oal daily raffic is shown in Fig. 2(a) and he raffic paern per domain is depiced in Fig. 2(b). In our effor o emulae he condiions of he acual applicaion, we use a cusom mediaion service wih an average service ime of 20ms per reques (which was he one also repored in real condiions). Furhermore, we consider he same ype/size of reques per domain. Implemenaion deails: In pracice, he scheduler could be implemened inside he mediaion applicaion, or ouside by regisering domains and assigning raffic per domain, where in his demonsraion we follow he laer approach and we implemen he scheduler inside he conroller. This is similar o inerposed reques schedulers [28] where he resource conrol is applied exernally and he server is reaed as a black box. The hp conroller is a cusom muli-hreaded Java Servle deployed in Glassfish Applicaion Server. The receiving process is separaed from he scheduling process a he hread level. This way, while one hread pus requess in he queues he oher is able o pull asynchronously. In principle, assigning differen hread pools o differen operaions may lead o unpredicable behavior, since hese operaions rely on he way he OS performs hread scheduling [5]. The servers are hosed in gues VMs ha reside in a single i5-3.2ghz, 8G memory Ubunu server machine. The Servle(scheduler), he mediaion applicaion and he endpoin service are hosed in a single gues VM o avoid communicaion overhead, while he cliens are hosed in differen VM machines (one per clien), also hosed in he same physical machine. Evaluaion scenarios: We benchmark he OBG algorihm in Equal share and Unequal share scenarios: Equal share scenario: In his scenario, we wan all he carriers (domains) o receive equal share from he CPU of he server hosing he mediaion service. Also, according o he SLA defined in eq. 3, when he oal raffic is less han he hroughpu capaciy, every domain will use he maximum achievable CPU cycling share. The backlog evoluion per domain for ha inerval is presened in Fig. 8(a) and in Fig. 8(b) he alloed uilizaion is presened. As we can conclude by hese figures, for he period when all he domains have available requess in queues, domain 3 requess ha end o dominae he sysem, are held back. When he backlog is close o zero for domains 1 and 2 (a approximaely 11:54), he uilizaion is proporional o he arrival rae of every domain and hus domain 3 sars o receive more uilizaion. In his scenario, OBG can be used o offer overload proecion and load balancing funcionaliy, while i converges fas (in less han 1 minue). Unequal share T-SLA (50%-30%-20%) Scenario: In he second scenario, we wan each domain o receive a predefined percenage {50%, 30%, 20%} from he server CPU cycles. As we can see in Fig.9(a), where he backlog evoluion is presened and Fig. 9(b) where he alloed uilizaion is presened, he algorihm again blocks domain 3 and differeniaes according o he T-SLA. For boh scenaria, a comparison is also presened beween OBG, WRR, he probabilisic algorihm based on predicions, presened in [25] and a Credi Scheduling, using he workload races. As we can see, in boh cases where he goal was se equal o {33%, 33%, 33%} Fig.8(c) and {50%, 30%, 20%}, Fig.9(c) respecively, he only algorihm ha fails o mee he objecive is he WRR. The reason is ha is no able o adap o he workload variaions and saisfy he objecives. The remaining algorihms, alhough in a shor ime scale, are able o provide he required differeniaion under heavy load. Noe ha in he comparison presened in Fig. 9, Credi scheduling and predicion echniques were able o differeniae raffic according o he objecive, bu no able o opimally conrol he allocaion accuracy and he relaive error, due o arbirary ON/OFF periods for all he domains. In conras, in he races experimen a single domain dominaes he sysem (domain 3), while he oher wo domains have low arrival rae. In his case of smooh workload changes, boh Credi scheduling and predicion echniques experience almos idenical performance wih OBG. In conclusion, saic WRR policies fail o mee he objecive under dynamic workloads; in he case of complex workloads for muliple domains, Credi schedulers mus be uned for opimal performance, while predicion echniques and DWRR can handle he dynamics of he sysem beer and increased accuracy, a he expense of increased overhead. D. Remark regarding ypes of service Our approach in his work focuses on saeless service designs, where we do no examine any relaionships beween services or dependencies on he sae informaion. Insead we adop an approach where muliple services are exposed as a single aggregae service (for example he campaign paricipaion service). In order o make our approach able o handle session based workloads and handle iner service dependencies, a model of he sae informaion as well as a model of processing chains for modeling he relaionship beween successive requess is required [27]. In he case where services are deployed in muliple servers, in addiion o he sae informaion an enhanced saisics collecion mechanism is required. The goal would be again o model he processing imes of he individual services called (synchronously or asynchronously) and keep rack of he cycles/uilizaion allocaed o every domain. VII. RELATED WORK In addiion o he relaed work referenced in he inroducion, our analysis is relaed o he large family of GPS/WFQ scheduling principle [8][29][30] and Defici/Weighed Round Robin schemes [31][32] [15]. For example, in [15] fair scheduling is applied and a weigh is assigned o each hread based on is prioriy, where fairness aims o bound he posiive and negaive processing ime lags. All he negaive drif policies use he same concep of dynamic weigh allocaion ha updaes he hreads round slice based on new weighs according o he deviaion ha he lag (processing ime) had from he WFQ goal [15]. We use ideas similar wih queue skipping based on some curren meric, bu in conras wih 10

11 !" #$ (a) Backlog evoluion. (b) Uilizaion achieved per domain. (c) Comparison beween scheduling policies. Fig. 8: Sudy of sochasic arrivals using races. The CPU uilizaion goal vecor is se equal o 33%, 33%, 33%.!" (a) Backlog evoluion. (b) Uilizaion achieved per domain. (c) Comparison beween scheduling policies. Fig. 9: Sudy of sochasic arrivals using races. The CPU uilizaion goal vecor is se equal o 50%, 30%, 20%. max packe lengh our skipping crierion is differen, he goal is o achieve a differen objecive (CPU cycles share no hroughpu) and we also use a differen noion of round. Also we provide no an algorihm bu a class of algorihms ha can be used o guaranee CPU shares. Proposals o change prioriies dynamically, using feedback-based sochasic conrol are presened in works like [33] and [34]; he auhors of hese works use a similar approach o ours bu focus on rae conrol hrough scheduling. Service differeniaion based on feedback conrol is also invesigaed in [35], while in [10] feedback conrol ogeher wih rae predicions are used o provide service differeniaion by means of slowdown. The problem formulaion sudied in his work, was also defined in [36] where a preliminary analysis was made for he case of ideally sauraed condiions. The resource provisioning T-SLA problem has also been addressed by he same auhors in [25] and [37] bu in a differen conex. In he former approach a linear approximaion echnique was proposed, while in he laer approach a sochasic queue managemen mechanism wih plain Round Robin scheduling was used o perform negaive drif operaions whenever over-provisioning of CPU resources. Neverheless, in boh [25] and [37] we do no provide proof of convergence and we presen convergence o he goal hrough simulaions. In conras o all our previous works and especially [36], we exend he proof in [36] o a more general class of policies; we provide ime and round of convergence analysis; provide rae of convergence analysis; exend he SLA definiion o cases where he assumpion of ideal sauraion is relaxed; we enhanced he comparison wih oher scheduling schemes under sochasic arrivals and we examine real races from a mobile compuing company o evaluae he DWRR algorihms performance under real condiions. Work in he area of guaraneed CPU performance hrough scheduling, also relaed o our work, is exploied by hypervisor echnologies [1][2][3][4]. Xen plaform uses Credi scheduling, which is he successor of BVT and SEDF schedulers [3][4] and VMware ESX server operaes wih a cusom scheduling mechanism ha uses he concep of reservaions and shares [1]. In he VMWare approach, a reservaions parameer specifies he guaraneed minimum and maximum CPU allocaion and a shares parameer measures he curren configured amoun of shares of he VM (a MHzPerShare meric based on he curren uilizaion of he virual machine). To conclude he sae of he ar secion, schemes where processes are separaed ino caegories based on heir need for CPU cycles are used in Mulilevel feedback queueing [38] sysems. Relaed work in CPU power managemen policies for service applicaions can be found in [39] and [40]. In [39] prioriy scheduling is proposed wih a penaly funcion ha is used o adjus hroughpu per clien; in [40] he auhors propose a mehod for esimaing CPU demand of service requess based on linear regression beween he observed reques hroughpu and resource uilizaion level. 11

12 VIII. CONCLUSIONS AND FUTURE WORK In his work, we defined a class of Dynamic Weighed Round Robin scheduling policies ha can be used o provision CPU resources in he server ier of a daa cener among compeing service domains. The objecive of such provisioning is o guaranee o each domain a pre-specified percenage of he CPU resource. We provided he necessary mahemaical framework and proofs of convergence for a class of policies ha does no require exac knowledge of service ime saisics and has adjusable communicaion and compuaion overhead. Besides heoreical analysis, exensive simulaions were performed based on real and synheic races. A promising direcion for fuure work ha is applicable o cloud environmens would be along he lines of end-oend problems: he conrol policies in his scenario mus ake ino accoun ineracions beween muliple iers of servers. Alhough in many applicaions CPU is he dominan facor ha affecs service delivery, in many applicaions a very large number of facors (neworking, CPU, memory, sofware orchesraions, ec.) affec he performance of services. Invesigaion of more complex SLAs ha ake ino he accoun he relaionship beween various performance merics and he dependencies beween services are lef for fuure work as well as sudy of merics like compleion ime o capure he effecs arised in he case of session based workloads. Furher research mus be also made in order o mee implemenaion consrains in highly dynamic environmens where he ideal sauraion concep does no hold. Acknowledgemens The work of Kosas Kasalis is co-financed by ESF and Greek naional funds hrough he Operaional Program Educaion and Lifelong Learning of he Naional Sraegic Reference Framework (NSRF) Research Funding Program: Heracleius II - Invesing in knowledge sociey hrough he European Social Fund. The work of G. S. Paschos was done when he was a MIT and affiliaed wih CERTH-ITI. IX. APPENDIX Firs we derive a helpful inermediae resul which bounds by how much he uilizaion of every domain exceeds he goal a any round. Since he upper bound decays wih k, we expec ha he uilizaions concenrae around he goal for large k. Lemma 1: For any domain i and round k, he uilizaion is upper bounded by U i ( k ) p i + K Smax i. (14) k Proof [Proof of lemma 1] Firs consider a round n for which he uilizaion is smaller han he goal. Then we readily ge U i ( n ) p i < p i + K Smax i. (15) n Nex we sudy a number of consecuive rounds k 1,..., k ξ for which he uilizaion of domain i is above he goal. Observe ha domain i does no receive service wihin rounds k 2,, k ξ since is uilizaion exceeds he goal (see policy definiion). Therefore we can readily conclude ha U( kξ ) < U( kξ 1 ) < < U( k2 ) < U( k1 ). Therefore i suffices o prove an upper bound only for rounds ha succeed a round where he uilizaion was below goal. For hese specific rounds we perform he following analysis. Suppose k is a round for which U i ( k ) > p i and U i ( k 1 ) < p i. Le i,k be he larges insan wihin round k such ha U i ( i,k ) = p i, clearly such an insance mus exis. Also, le S i,i k be he remaining service ime of domain i in round k, afer he ime insance i,k. Then using eq. (6) for k < k+1 we derive U i () p i i,k U i( i,k ) + = i,k p i + S k i,i S k i,i p i = p i (16) S k i,i i,k p i Since K Si max is a universal upper bound of service wihin a round, we have S i,i k K Smax i and so U i () p i + S k i,i k S k i,i p i + K Smax i. (17) k Proof [Proof of heorem 1] We will prove ha for every domain he uilizaion sequence converges o he goal almos surely. lim U i( k ) = p i a.s. Consider a subse S of he sample space Ω in which Si n < K Si max, such ha P (S) = 1. All he subsequen saemens in he proof hold rue for samples ω in his se; for noaional simpliciy, dependence of sample pahs on ω is omied. The corresponding relaions used for calculaions in he discree ime for he sampled sequence is U i ( n+1 ) = Using lemma 1, firs we noe ha lim sup K S max i lim sup n n+1 U i ( n ) + Sn i n+1, (18) K S max i k = 0 since is by definiion almos surely finie. Then, we have U i ( k+1 ) lim sup p i + lim sup K S max i k = p i a.s (19) 12

13 Addiionally, noe ha lim inf U i( k+1 ) = lim inf (1 1 + lim inf ( = 1 lim sup 1 j D\{i} eq.(19) 1 j D\{i} Since i mus also be lim inf conclude ha lim inf j D\{i} j D\{i} j D\{i} U j ( k+1 )) (20) U j ( k+1 )) U j ( k+1 ) lim sup U j ( k+1 ) p j = p i = lim sup U i ( k+1 ) U i( k ) lim sup U i ( k ), we U i( k ) = lim sup U i ( k ) = p i. Therefore he limi exiss and lim U i( k ) = p i a.s. Using he following lemma, he proof is compleed. Lemma 2: If he sampling sequence U i ( k ) a he beginning of every round converges almos surely, hen he coninuous ime uilizaion U i () converges o he same limi a.s. Proof [Proof of lemma 2] Pick any wihin some round k, i.e., in he inerval [ k, k+1 ], we will show ha U i () converges o he goal. Le T i ( k, ) denoe he service ime received for domain i wihin he ime inerval [ k, ]. Then using eq. (6) U i () = k U i( k ) + T i( k, ) and hence ( ) U i () U i ( k ) = k U i ( k ) + T i( k, ) by he choice of, we have ha k < herefore ( k ) j D K Sj max. Then j D K S max j (21) and U i () U i ( k ) = k U i ( k ) + T i( k, ) (22) k U i( k ) + K Si max K S max j j D + K S max i K Si max + K Sj max j D = Also applying he riangle inequaliy we have U i () p i U i () U i ( k ) + U i ( k ) p i (23) From eq.(22) and eq.(23) we conclude ha lim sup U i () p i lim sup U i () U i ( k ) Since lim inf U i() p i > 0, we conclude ha lim sup U i () p i = lim inf U i() p i = lim U i () p i = 0 from which we conclude ha U i () a.s. p i. Proof [Proof of Theorem 2] Le R i ( n ) = U i( n+1 ) p i U i ( n ) p i (24) he rae of convergence of U i ( n ) sequence can be found by lim R i( n ) (since his limi exiss by Theorem 1). n According o eqs. (7) and (24), we have ha R i ( n ) = n n+1 U i ( n ) + Sn i n+1 p i n 1 n U i ( n 1 ) + Sn 1 i n p i (25) where for simpliciy of noaion Si n in eq. (25) conains he indicaor funcion of eq. (7). The numeraor of he righ par of he equaion is equal o n U i ( n ) + Sn i p i n+1 n+1 = n 1 U i ( n 1 ) + Sn 1 i + Sn i p i n+1 n+1 n+1 (26) Afer we replace eq. (26) in eq. (25) and perform some arihmeic operaions, we have ha: R i ( n ) = n n Si n L n p i n 1 U i ( n 1 ) + S n 1 n p (27) i where we used he fac ha n+1 = n + L n. Le g( n ) = N i(n) D i (n) = Si n L n p i n 1 U i ( n 1 ) + S n 1 (28) i n p i Si n = L n p i n 1 [U i ( n 1 ) p i ] + S n 1 i L n 1 p i where N i (n) denoes he numeraor and D i (n) denoes he denominaor. We wan o show ha lim g( n) = 0. A n minor echnical difficuly arises from he presence of he erm n 1 [U i ( n 1 ) p i ] in he denominaor, which leads o an 0 erm in he limi. To overcome his echnicaliy, we define nex wo random sequences v n ( n ) and h n ( n ) as follows: v n ( n ) = h n ( n ) = S n i L n p i n 1 [U i ( n 1 ) p i + u n i ] + Sn 1 i S n i L n p i n 1 [U i ( n 1 ) p i + h n i ] + Sn 1 i i L n 1 p i L n 1 p i where vi n,hn i are random sequences chosen o make he brackeed erms nonzero and in addiion obey he following rules: 13

14 N i (n) D i (n) ui n hi n 0 > 0 ui n > 0 Di (n) n 1 < hi n < 0 0 < 0 0 < ui n Di (n) < n 1 hi n < 0 < 0 > 0 Di (n) n 1 < ui n < 0 hi n > 0 < 0 < 0 ui n < 0 0 < hi n < Di (n) n 1 These rules guaranee ha we can bound g( n ) as follows: v n ( n ) < g( n ) < h n ( n ), n. We can easily see, however, ha lim v n( n ) = 0 and n lim h n( n ) = 0. (Boh limis go o zero since by definiion n n, S n i < K S max i <, L n = i:w n i 1 S n i D i=1 K S max i < ). By he squeezing principle i follows ha lim n g( n) = 0 and hus we can calculae he limi in eq. (27): lim R i( n ) = = 1 (29) n The uilizaion approaches he arge sub-linearly alhough variable speed may be observed during every sample pah. REFERENCES [1] VMware vsphere 4: The CPU Scheduler in VMware ESX4, [2] R. McDougall and J. Anderson, Virualizaion performance: perspecives and challenges ahead, SIGOPS, ACM, vol. 44, no. 4, pp , Dec [3] D. Ongaro, A. Cox, and S. Rixner, Scheduling I/O in virual machine moniors, SIGPLAN/SIGOPS Proceedings, ACM, pp. 1 10, [4] L. Cherkasova, D. Gupa, and A. Vahda, Comparison of he hree cpu schedulers in xen, SIGMETRICS Perform. Eval. Rev., vol. 35, no. 2, pp , [5] D. Bove and M. Cesai, Undersanding The Linux Kernel. Oreilly & Associaes Inc, [6] E. Gelenbe and I. Mirani, Analysis and Synhesis of Compuer Sysems: Texs), 2nd ed. London, UK, UK: Imperial College Press, [7] J. B. Nagle, On packe swiches wih infinie sorage, Trans. on Communicaions, IEEE, vol. 35, no. 4, pp , [8] A. Demers, S. Keshav, and S. Shenker, Analysis and simulaion of a fair queueing algorihm, in Symposium proceedings on Communicaions archiecures & proocols, ser. SIGCOMM 89. New York, NY, USA: ACM, 1989, pp [9] A. Burns, G. Berna, and I. Broser, A Probabilisic Framework for Schedulabiliy Analysis, Embedded Sofware, Lecure Noes in Compuer Science, Springer, vol. 2855, pp. 1 15, [10] J. Wei, X. Zhou, and C.-Z. Xu, Robus processing rae allocaion for proporional slowdown differeniaion on inerne servers, Compuers, IEEE Transacions on, vol. 54, no. 8, pp , [11] C. ping Li and M. Neely, Delay and rae-opimal conrol in a muliclass prioriy queue wih adjusable service raes, in INFOCOM, 2012 Proceedings IEEE, 2012, pp [12] P. P. Bhaacharya, L. Georgiadis, P. Tsoucas, and I. Viniois, Adapive lexicographic opimizaion in muli-class m/gi/1 queues, Mahemaics of Operaions Research, vol. 18, pp , [13] M. M. N. Aldeer, Performance comparison of packe-level muliplexing algorihms wih bursy raffic, Journal of Engineering Science and Technology, pp , [14] J. L. Hellersein, Y. Diao, S. Parekh, and D. M. Tilbury, Feedback Conrol of Compuing Sysems. John Wiley & Sons, [15] T. Li, D. Baumberger, and S. Hahn, Efficien and scalable muliprocessor fair scheduling using disribued weighed round-robin, SIGPLAN, ACM, pp , [16] C.-P. Li, G. Paschos, L. Tassiulas, and E. Modiano, Dynamic overload balancing in server farms, in Neworking Conference, 2014 IFIP, June 2014, pp [17] S. Ahuja and A. Pael, Enerprise service bus: A performance evaluaion, Communicaions and Nework, vol. 3, pp , [18] T. Lam, J. Ding, and J.-C. Liu, Xml documen parsing: Operaional and performance characerisics, Compuer, vol. 41, no. 9, pp , [19] M. Tanner, Pracical Queueing Analysis, 1s ed. New York, NY, USA: McGraw-Hill, Inc., [20] Z. Wang, Y.-Q. Song, H.-B. Yu, and Y. Sun, Sabiliy analysis for muliclass muli-queue single server sysem under polling able, in American Conrol Conference, Proceedings of he 2002, vol. 6, 2002, pp vol.6. [21] L. Tassiulas and A. Ephremides, Sabiliy properies of consrained queueing sysems and scheduling policies for maximum hroughpu in mulihop radio neworks, Trans. on Auomaic Conrol, IEEE, vol. 37, no. 12, pp , [22] M. Markakis, E. Modiano, and J. Tsisiklis, Max-weigh scheduling in neworks wih heavy-ailed raffic, INFOCOM Proceedings, IEEE, pp , [23] N. Osada, The early hisory of convergence acceleraion mehods, Numerical Algorihms, vol. 60, no. 2, pp , [24] L. Sha, X. Liu, Y. Lu, and T. Abdelzaher, Queueing model based nework server performance conrol, in Real-Time Sysems Symposium, RTSS rd IEEE, 2002, pp [25] K. Kasalis, L. Tassiulas, and Y. Viniois, Disribued Resource Allocaion Mechanism for SOA Service Level Agreemens, NTMS, IFIP/IEEE, pp. 1 6, [26] G. W. Dunlap, Scheduler developmen updae, Xen Summi Asia, [27] D. G. Feielson, Workload modeling for compuer sysems performance evaluaion, Cambridge Universiy Press, [28] J. Zhang, A. Sivasubramaniam, A. Riska, Q. Wang, and E. Riedel, An inerposed 2-level i/o scheduling framework for performance virualizaion, SIGMETRICS Perform. Eval. Rev., vol. 33, no. 1, pp , Jun [29] A. Chandra, W. Gong, and P. Shenoy, Dynamic resource allocaion for shared daa ceners using online measuremens, SIGMETRICS, ACM, vol. 31, no. 1, pp , [30] N. Giroux, R. Liao, and M. Aissaoui, Fair queue servicing using dynamic weighs (DWFQ), 2001, US Paen 6,317,416. [31] M. Shreedhar and G. Varghese, Efficien fair queueing using defici round robin, SIGCOMM Compu. Commun. Rev., vol. 25, no. 4, pp , Oc [32] D.-C. Li, C. Wu, and F. M. Chang, Deerminaion of he parameers in he dynamic weighed Round-Robin mehod for nework load balancing, Compuers and Operaions Research, vol. 32, no. 8, pp , [33] J. Smih, E. K. P. Chong, A. Maciejewski, and H. Siegel, Sochasic- Based Robus Dynamic Resource Allocaion in a Heerogeneous Compuing Sysem, Parallel Processing, ICPP, pp , [34] D. Guo and L. Bhuyan, A QoS aware mulicore hash scheduler for nework applicaions, INFOCOM Proceedings, IEEE, pp , [35] T. Abdelzaher, J. Sankovic, C. Lu, R. Zhang, and Y. Lu, Feedback performance conrol in sofware services, Conrol Sysems, IEEE, vol. 23, no. 3, pp , [36] K. Kasalis, G. Paschos, L. Tassiulas, and Y. Viniois, Dynamic CPU scheduling for QoS provisioning, in Inernaional Symposium on Inegraed Nework Managemen (IM), IFIP/IEEE, May 2013, pp [37], Service differeniaion in Muliier Daa Ceners, in Inernaional Conference on Communicaions (ICC), IEEE, June 2013, pp [38] C. Faissnauer, D. Schmalsieg, and W. Purgahofer, Prioriy roundrobin scheduling for very large virual environmens, Virual Realiy Proceedings, IEEE, pp , [39] A. Sharma, H. Adarkar, and S. Sengupa, Managing qos hrough prioriizaion in web services, Web Informaion Sysems Engineering Workshops, IEEE, pp , [40] C. Zhang, R. Chang, C. Perng, E. So, C. Tang, and T. Tao, Leveraging Service Composiion Relaionship o Improve CPU Demand Esimaion in SOA Environmens, SCC, IEEE, vol. 1, pp ,