Self-Adaptive SLA-Driven Capacity Management for Internet Services

Transcription

1 Self-Adaptve SLA-Drven Capacty Management for Internet Servces Bruno Abrahao, Vrglo Almeda and Jussara Almeda Computer Scence Department Federal Unversty of Mnas Geras, Brazl Alex Zhang, Drk Beyer and Fereydoon Safa Intellgent Enterprse Technology Lab. HP Labs, Palo Alto, CA, USA Abstract Ths work consders the problem of hostng multple thrd-party Internet servces n a cost-effectve manner so as to maxmze a provder s busness objectve. For ths purpose, we present a dynamc capacty management framework based on an optmzaton model, whch lnks a cost model based on SLA contracts wth an analytcal queung-based performance model, n an attempt to adapt the platform to changng capacty needs n real tme. In addton, we propose a two-level SLA specfcaton for dfferent operaton modes, namely, normal and surge, whch allows for per-use servce accountng wth respect to requrements of throughput and tal dstrbuton response tme. The cost model proposed s based on penaltes, ncurred by the provder due to SLA volaton, and rewards, receved when the servce level expectatons are exceeded. Fnally, we evaluate approxmatons for predctng the performance of the hosted servces under two dfferent schedulng dscplnes, namely FCFS and processor sharng. Through smulaton, we assess the effectveness of the proposed approach as well as the level of accuracy resultng from the performance model approxmatons. I. INTRODUCTION Wth the exponental growth n popularty over the past few years, Internet servces have become a popular soluton for busnesses to offer ther tradtonal servces onlne to customers and to deploy nternal busness operatons n a dstrbuted fashon. In lght of ths, busnesses are ncreasngly relyng on computng-based capacty outsourcng [] as a fnancally attractve approach to host ther servces. In ths scenaro, busnesses sgn SLA contracts [2] wth a provder that hosts a large dversty of Internet servces n shared Internet data centers (IDC). On the other hand, n order to be proftable, provders drect ther efforts to manage resources n the most cost-effectve way whle satsfyng the establshed customers servce level requrements. In ths work, we consder a scenaro where a number of dfferent thrd-party transactonal Internet servces are hosted n a shared platform whch employs mechansms to provde servce dfferentaton. The focus of our work s on the capacty management for IDCs n such a way as to explore the avalable resources to the provder s best advantage so that a busness goal s maxmzed. The emergence of new customer demands pose unprecedented operatonal challenges to ths problem. Frst, n order to stay compettve, nstead of smply agreeng on satsfyng an average response tme requrement, provders have been offerng contracts that promse the satsfacton of a tal dstrbuton requrement of response tme [3], meanng that an upper bound on the probablty of the end-to-end response tmes exceedng a gven threshold s requred. Moreover, n addton to establshng end-to-end response tme requrements, customers have also been demandng from the provders a guarantee on the throughput acheved by ther servces. Last but not least, there has been a recent outburst of nterest n servce contracts n whch customers pay only for actual use [4], thereby forcng provders to revew the tradtonal SLA contracts and proposng more flexble servce accountng schemes. The above facts have drect mpact on the capacty management of Internet servces. Frst, the performance modelng of applcatons requres more complex models whch, n some cases, cannot be solved effcently, or a soluton that captures the relevant characterstcs of the system s not known. On the other hand, the satsfacton of the performance requrements has strong mplcatons on the provder s revenue [2]. As a consequence, the capacty management model should be drven wth respect to a cost model, based on per-use accountng, n whch capacty need conflcts are handle n lght of the provder s fnancal objectve, nstead of smply meetng the customers performance requrements. Fnally, to make matters more complcated, these servces exhbt workloads that may present great fluctuatons over tme, whch changes the servce capacty needs durng ther operaton dramatcally. Ths clearly calls for a dynamc approach to self-adapt the system n real tme, as a response to such changes. A. Research Contrbutons Ths paper presents a model for self-adaptve capacty management n shared envronments, drven by a cost model based on SLA contracts. Due to the varablty n level exhbted by the workloads, we propose a two-level SLA specfcaton, namely, normal and surge operaton modes, whch allows customers to pay for extra capacty (than that normally requred) only when needed. In addton to the proposed SLA contract, we also propose a cost model based on penaltes, ncurred by the provder as a result of servce level requrement volatons, and rewards, pad by customers when ther expectatons, expressed as requrements of throughput subject to a response tme guarantee, s exceeded. In order for the IDC to respond to workload changes and, consequently, adapt to changng capacty needs, a real-tme self-adaptve framework for managng capacty wthn an IDC

2 s proposed here. Our approach s based on an optmzaton model, whch, wth the objectve of maxmzng the net result from penaltes and rewards, lnks the proposed cost model wth a queung-based performance model that predcts the performance of the hosted Internet servces. The computaton of the probablty dstrbuton of response tmes, requred when predctng the capacty needs wth respect to the tal dstrbuton requrements, s also challengng when dealng wth queung models. Ths s because ) the exact results for ths metrc are only avalable for specal types of queues, most of whch do not realstcally capture the characterstcs of the system consdered, and ) some of the avalable results make the optmzaton problem hard to solve n real tme. As a consequence, approxmatons are often needed. Therefore, we propose and evaluate dfferent approxmatons to express the probablstc response tme requrement, comparng the level of accuracy resultng from each of them n the context of our problem formulaton under two dfferent servce schedulng dscplne, namely fst come frst served (FCFS) and processor sharng. Last, we assess the effectveness of the proposed approach through dscrete event smulaton of the mult-servce IDC along wth the resultng cost mplcatons on the fnancal affars of the provder. Ths paper s organzed as follows. Secton II descrbes the envronment consdered and the cost model proposed. The self-adaptve framework for capacty management s presented n Secton III. Secton IV descrbes the parts that comprse the capacty management model, and the expermental analyss s presented n Secton V. Fnally, Secton VI dscusses related work, and Secton VII offers our concluson. II. ENVIRONMENT DESCRIPTION Ths secton provdes an overvew of the envronment and the cost model upon whch our autonomc capacty management framework s bult. A. Hostng Platform Descrpton We consder a scenaro where a provder hosts multple thrd-party transactonal Internet servces n a shared IDC. A key feature of IDCs s the ablty to provde performance solaton by preventng the drect contenton for resources between dfferent servces. Accordngly, a far sharng polcy or a vrtualzaton scheme [5], [6] s employed so as to provde servce dfferentaton. These mechansms partton the physcal resources (.e., processng, storage and communcaton resources) nto multple solated vrtual ones, each runnng at a fracton of ts correspondng physcal resource capacty. Hence, nstead of usng the physcal resources drectly, the hosted Internet servces demand servce from a pool of vrtual resources, created and mantaned by an ntervenng vrtualzaton layer. Typcal Internet servces are usually composed of dfferent transacton types, subjected to dfferent workloads, wth dfferent servce demands on the resources and executed by Class Fg.. Class 2... Physcal nfra-structure Class N Vrtual Machnes Vrtualzaton Layer Internet servce hostng platform. ndependent software components. We denote these components as applcaton classes. Under the assumpton that the classes are ndependent, they are analyzed here as ndependent applcatons. The hostng platform consdered s depcted n Fgure. The vrtualzaton layer creates N solated vrtual machnes (VM) (also known as resource contaners or applcaton envronments [5], [7]) on top of the physcal nfra-structure. Each VM s composed of a set of vrtual nstances of each of the IDC s resources and s dedcated to servng a sngle applcaton class only. Ths hostng model solates classes one from another, each usng the vrtual machne as f t were a dedcated server, workng at a fracton of the total (physcal) capacty. Vrtualzaton allows the physcal resources to be proportoned to applcaton classes, each recevng at least as much capacty as has been assgned to t, regardless of the load mposed by other classes. Thus, t enables the IDC to flexbly contract or expand the resource capactes assgned to applcaton classes. Hence, we defne the capacty allocaton decson as the determnaton of the fractons of server capacty each VM ( =,.., N) obtans from the correspondng physcal one. Last, we assume that the VMs employ an admsson control scheme [8] that rejects requests for varous purposes. For example, the VMs may drop some requests to avod servce nstablty due to capacty lmtatons or to guarantee that the requrements of response tme are met. In ths analyss, we focus on the dynamc capacty management model wthn the server nodes, and we make use of a hgh level of abstracton, by consderng each VM as a sngle resource. We left the extenson of the managng each of the VM s devces (.e. CPU, dsk, etc) ndependently, as well as servce replcaton and mult-tered servces for future work. B. Cost Model The servce contracted by customers must meet certan performance expectatons whch the partes agree upon n the SLA contracts. Ths work focuses on the requrements of throughput and response tme. In addton to agreeng on the requrements n the SLA contracts, the provder also establshes a servce rate to charge customers proportonal to the strctness of ther servce level requrements. For nstance,

3 X SSLA NSLA X tps Surge Operaton Mode Normal Operaton Mode Fg. 2. Example of an arbtrary vald throughput scale wth the ranges of normal and surge operaton modes. customers who run crtcal busnesses are wllng to pay more so as to obtan hgh throughput and/or short response tmes whereas other customers sgn up for servce wth loose requrements but lower costs. The Internet offers plenty of examples of servces whch usually receve low to moderate load, but occasonally receve an exceptonally large surge of requests. Common examples are the onlne news servces whch expect a predctable number of users durng normal operaton but, whenever certan specal events occur, a sudden surge of clents overload the ste, changng the capacty needs dramatcally [9]. Due to the hghly dynamc nature of Internet workloads, we propose contracts wth two levels of requrements, whch correspond to two dfferent operaton modes, namely, normal and surge. In the normal operaton mode, customers contract the servce level whch satsfes ther needs for the majorty of operaton tme whereas n the surge operaton mode, a hgher servce level lmt s establshed, up to whch the provder has an ncentve to assgn extra capacty to servces so as to accommodate occasonal workload peaks. Tradtonal contracts wth a sngle performance target would requre customers whose workload presents hgh peak-to-mean rato to pay for the servce level needed to satsfy both the average and the peak of demand durng the entre operaton, even though only part of the capacty the customer pays s actually utlzed most of the tme. Therefore, from the busness standpont, the two-level SLA approach can be advantageous both to customers who pay for extra capacty only when needed, and to provders who are able to offer more attractve servce plans by operatng wth more flexblty. There are several ways of measurng the servce level provded to a customer. In ths work, the SLA performance requrements quantfy the VM s ablty to process transactons, provded the qualty of the processed transacton response tmes satsfy a gven requrement. Accordngly, t s consdered the tal dstrbuton response tme requrement whch states that the response tme of the transactons from class must not exceed a gven threshold R SLA for more than α % of the tme, or equvalently, P (R >R SLA ) α,wherer s the response tme of a sngle transacton of class. Moreover, n lght of the above reasonng, the proposed SLA contracts contan performance targets for the normal operaton mode, under NSLA requrements, and for the surge operaton mode, under SSLA requrements. In order to present the operaton modes, let us frst ntroduce some defntons. The acheved performance of applcatons s computed at the end of perodc ntervals so that the payoff of the servce s executon (.e., the net result from penaltes and rewards) can be determned. In addton, we refer to the the actual processng rate n whch transactons are performed wthn the response tme requrement as the vald throughput. Accordngly, the remanng transactons whch volate the response tme requrement are not consdered n the SLA accountng process. Fgure 2 dsplays an example of an arbtrary scale that shows the range of the normal and the surge operaton modes. The NSLA requrement states that the vald throughput of transactons per second, whch s the upper lmt to the normal operaton mode. Nonetheless, volatons of the NSLA requrements may occur, n whch case the provder agrees to refund part of the servce charge to customers, whch s proportonal to the dfference between the NSLA throughput requrement and the saturated vald throughput, provded ths dfference has been caused by capacty lmtatons. Conversely, the SSLA target s establshed n order to prevent mssng servce or unavalablty n case of occasonal workload peaks. Thus, the applcaton class should be at least X NSLA partes establsh a lmt on the throughput, X SSLA (X SSLA ), whch s the upper lmt to the surge operaton mode, X NSLA up to whch the owner of class agrees to pay a reward to the provder whenever the vald throughput over the nterval exceeds X NSLA. Accordngly, the rewards are proportonal to the extra vald throughput acheved. Gven the framework descrbed, we defne the provder s busness objectve as the provson of capacty to VMs so as to maxmze the net result from penaltes and rewards. Fnally, t should be emphaszed that ths approach s not restrcted to only two SLA levels. Thus, one may defne multple levels whch specfy multple performance targets so as to account for dfferent levels of demands. III. SELF-ADAPTIVE FRAMEWORK In order to provde the system wth the ablty to adapt tself n response to changes, we propose the closed control loop presented n Fgure 3. The heart of the model s the capacty manager component whch, gven the expected workload of each VM, the SLA requrements, and the system characterstcs of the classes, reconfgures the IDC wth the goal of maxmzng the provder s busness objectve. Therefore, the capacty manager component s confgured wth SLA and system parameters of each applcaton class (dscussed n Secton II-B and summarzed n Table I of Secton IV-A) (). These values are updated whenever contract changes occur (.e., a new applcaton class s ntroduced or removed from the data center, or the requrements change).

4 Workload Forecast Workload Forecaster (3) Capacty Manager () SLA and System Parameters (2) Workload Montorng (4) New IDC Confguraton Fg. 3. Class (...) Class N Physcal nfra-structure Closed Control Loop Vrtual Machnes Vrtualzaton Layer The workload forecaster component s responsble for constantly montorng the ntensty of the workload of each class over tme (2). It contans a local storage faclty (.e., a database) to store the past behavor of the workload so that the logged data can be used for perodc analyss. Accordngly, t uses one of the exstng workload forecastng methods [] to predct, based on past observed behavor, the workload that each class s lkely to receve n the near future. Consequently, the predcted workload s also used as nput by the capacty manager (3). After determnng a capacty allocaton decson, the capacty manager component sends the new IDC confguraton to the system (4), adjustng the vrtual resource mappngs n the vrtualzaton layer nto new values at the physcal nfrastructure and the admsson control parameters for each VM. We refer to the nterval between consecutve controller nterventons as the controller nterval. The duratons of these ntervals may be fxed or varable dependng on the characterstcs of the system and ts servces. For example, servces whose workload present a constant workload behavor can make use of adaptatons trggered by unexpected changes n demand. On the other hand, the capacty manager nvocatons can occur at perodc ntervals of tme, whch are usually confgured for perods n whch the workload of servces s unlkely to change. Moreover, lmtatons on the tme requred to adapt the system mght defne the mnmum length of the nterval. Fnally, we consder that the payoff for the executon of the servces (the net result from penaltes and rewards) durng each controller nterval s computed at the end of the correspondng tme perod. The capacty manager component presented n ths work s based on an optmzaton model whch lnks the cost model to a performance model (dscussed n Secton IV-D) and determnes, n lght of the SLA specfcatons and the predcted workload for each applcaton class, a capacty allocaton decson that maxmzes the provder s busness objectve for the next controller nterval. IV. CAPACITY MANAGEMENT MODEL Ths secton descrbes the capacty management model for Internet servces. We frst descrbe the model parameters and Symbol X NSLA X SSLA Descrpton TABLE I MODEL PARAMETERS SLA parameters vald throughput requrement (n tps) for class n normal operaton mode. vald throughput upper lmt (n tps) for class n surge operaton mode. R SLA response tme threshold (n seconds) for class. α lmt on probablty of response tme beng greater than R SLA for class. c penalty cost for a unt of throughput NSLA saturaton for class. π N reward prce for a unt of extra vald throughput above X NSLA for class. System parameters number of applcaton classes, and thus number of VMs. υ maxmum utlzaton planned for VM. E[S ] λ P {Z } average servce tme (n seconds) of class transactons on the physcal server. Workload Forecaster s estmates predcted arrval rate (n tps) of class for next controller nterval. probablty of captalzng rewards from class durng next controller nterval. ts assumptons (Secton IV-A), the penalty and the reward accountng scheme (Secton IV-B), and, fnally, we present the optmzaton model for capacty allocaton (Secton IV-C). A. Man Model Parameters and Assumptons The man model parameters are presented n Table I. Some of the parameters are extracted drectly from the SLA contracts whle others are system parameters and estmates made by the workload forecaster component, used as nputs to the capacty manager. The SLA parameters can be understood n lght of the dscusson presented n Secton II-B. The system parameters consst of: ) the number N of VMs mantaned by the vrtualzaton layer, thereby the number of applcaton classes hosted by the provder, 2) the maxmum utlzaton υ the provder allows VM to reach, and 3) the average servce tme of transactons from class on the physcal server. Parameter υ s ntroduced because both the mean and the varance of response tmes can ncrease wthout lmt when the utlzaton approaches %. Thus, n order to mantan a certan level of stablty n the VMs, a fractonal upper lmt υ ( υ < ) s specfed, up to whch the utlzaton of VM s planned. We assume that transactons from the same applcaton class are statstcally ndstngushable and, thus have the same average servce tme on the physcal server. Therefore, E[S ] s used here as a model parameter to ndcate the mean servce tme of class, whch can be approxmated by the arthmetc

5 average of the observed values of S, measured n a preproducton executon of the applcatons on the physcal server. Snce each VM receves a guaranteed fracton of tme from the physcal server, takng f as the fracton of the server assgned to the VM, we approxmate the average servce tme on the VM as E[S ]/f. Accordngly, the capacty allocaton decson s the determnaton of the capacty fractons (.e., f, =..N) to be assgned to each VM. Therefore, f s the man decson varable of the proposed problem. The last two rows of Table I present the outputs of the workload forecaster component. As mentoned n Secton III, the capacty manager component receves from the workload forecaster component an estmate λ ofthearrvalrateof requests durng the next controller nterval for each applcaton class. The capacty optmzer component can be nvoked n response to arrval rate devatons from the above estmate. However, when consderng fxed controller ntervals, events that are sgnfcantly shorter than the nterval could mslead the manager. For example, a surge of requests wth short duraton can cause the capacty manager to seze the capacty needed by other classes to meet ther NSLA requrements so as to provde extra capacty to a more proftable one. Clearly, ths would ncur penaltes throughout the controller nterval whereas the reward would be captalzed durng a tny fracton of the tme. In order to mnmze ths undesred effect, the workload forecaster component may also produce the estmated probablty of a class recevng a surge of requests, that s, an arrval, durng the next controller nterval. Thus, parameter P {Z } ndcates the certanty level to bet on reward captalzaton for VM. If the system s allowed to adapt to workload changes, ths parameter can be bypassed by settng t to. rate greater than X NSLA We consder λ as the arrval rate of class over the controller nterval. However, due to capacty lmtatons, some of the requests may be rejected, n whch case the actual throughput of the class becomes smaller than λ. Moreover, some of the processed transactons may volate the response tme requrement, n whch case they are not counted (for SLA purposes to compute the vald throughput) as processed. In order to dffer these cases, X s ntroduced to denote the actual throughput, and µ to denote the vald throughput of class, over the controller nterval. B. Penalty and Reward Accountng Ths secton descrbes how penaltes and rewards are computed and how the payoff for the executon of servces s calculated at the end of each controller nterval. We start by defnng Q as the relatve frequency of transactons wth response tmes below R SLA at the end of the controller nterval. Thus, f the requrement has been volated, that s Q < ( α ),thenµ <X, and the vald throughput s gven by the transactons processed wthn the tme lmt plus the tolerance allowed: µ = Q X + α (Q X ) = ( + α )(Q X ).Otherwse,whenQ ( α ), all the TABLE II VALUES OF PENALTY AND REWARD Symbol Condton Value Penalty Y µ <X NSLA Reward Z µ X NSLA mn{x NSLA,λ } µ mn{x SSLA,µ } X NSLA transactons processed durng that nterval were wthn the response tme requrement and, thus, µ = X. Upon defnng the response tme tal dstrbuton requrement as a QoS condton, the NSLA saturaton s consdered here as the VM s nablty to acheve the vald throughput requrement. Consder Y to be the penalty ncurred by the provder due to volatng the performance requrement of VM. As mentoned n Secton II-B, the penaltes are proportonal to the magntude of the dfference between the NSLA throughput requrement and the vald throughput. However, the throughput may have been smaller than the requrement due to capacty lmtatons or due to an arrval rate X NSLA smaller than X NSLA. Snce the provder should be penalzed only for the former case, a penalty s ncurred whenever and µ <λ. In ths case the penalty value = λ µ. However, snce the NSLA target requres the provder to process at least X NSLA transactons over the controller nterval, f λ X NSLA, the ncurred penalty should be Y = X NSLA µ. Thus, the penalty cases just descrbed can be expressed as Y = mn{x NSLA,λ } µ. Conversely, f the vald throughput of an applcaton class has exceeded the throughput NSLA requrement, µ X NSLA, the provder s able to captalze rewards proportonally to the magntude of the dfference between the extra vald throughput and the NSLA throughput requrement. However, the reward value cannot exceed the lmt X SSLA. Let us denote Z to be the reward value pad by class owner and express the reward case as Z = mn{x SSLA,µ } X NSLA. µ <X NSLA s Y The above condtons determne the values of the SLA penaltes and rewards computed at the end of the controller nterval. The penalty or reward value for any other possble condton s. All penalty and reward cases are summarzed n Table II. C. Optmzaton Model Formulaton The optmzaton model evaluates the estmated net result from penaltes and rewards at the begnnng of each controller nterval, usng the workload predcted by the workload forecaster component n order to provde a capacty allocaton decson whch maxmzes the provder s busness objectve. The model objectve functon expresses the sum over all applcaton classes of the expected payoff for ther executon as follows: Maxmze N c Y + π Z P {Z } () = The whole optmzaton model s depcted n Fgure 4 and n order to refer to the model constrants, they are ndexed

6 max s.t. N = c Y + π Z P {Z } Y mn(x NSLA,λ ) X (a) Y (b) Z = δ (mn(x SSLA,X ) X NSLA ) (c) Z, δ {, } (d) λ = λacc + λ rej (e) X = λ acc (f) P (R R SLA ) α (g) ρ = λ acc /λ sat υ (h) N f () f, ρ (j) X,λ acc,λ rej,λ sat (k) Fg. 4. Optmzaton model usng lowercase letters (a) to (k). Constrants (a) to (d) express the cases shown n Table II, replacng the arrval rate λ and the vald throughput µ of the past controller nterval by the estmate λ and the actual throughput X respectvely. Notce that snce Y s stated n constrant (b), Y assumes n cases of reward (.e., when X >X NSLA ). In addton, n constrants (c) and (d), an artfcal bnary varable δ s ntroduced as part of the expresson for Z. Thus, δ s forced to when X <X NSLA, makng Z =n case of penaltes. Otherwse, δ assumes the value. Due to capacty lmtatons, the VMs mpose a lmt on the number of requests that can be processed. However, the arrval rate can be greater than the maxmum achevable processng rate n whch case some of the requests must be dropped. Therefore, we assume that the VMs employ one of the exstng admsson control polces [8] whch splts the arrval rate a VM receves nto λ = λ acc + λ rej,whereλ acc s the rate at whch VM accepts requests, and λ rej s the rate at whch VM rejects requests. Ths s expressed n the optmzaton model by constrant (e), usng λ as the expected arrval rate. Constrant (f) expresses the job flow balance condton [] whch states that all accepted requests are actually processed by the VM, thereby assurng that no accepted request s lost. If the reader wll recall, the vald throughput of a VM s subject to the response tme requrement satsfacton. Therefore, ths s expressed as model constrant (g). Snce the expected response tme s a functon of the VM throughput and the fracton of capacty t receves, R = f(λ acc,f ), constrant (g) forces the VMs to accept only the transactons that would satsfy the response tme requrement for any gven value of f, wth the objectve of makng µ = X at the end of the controller nterval. Notce that α expresses a tradeoff between the throughput and the qualty of the processed transactons. Indeed, a smaller α results n lower throughput but guarantees a hgher degree of response tme requrement satsfacton, whereas a large α ncreases the processng rate of transacton at the expense of ther response tmes. The utlzaton ρ of VM s defned by constrant (h) as the rato of λ acc and the smallest arrval rate λ sat at whch VM becomes saturated (see Secton IV-D). It s also mportant to emphasze that, smlarly to the expected response tme, ρ s a functon of both the acceptance rate of request and the fracton of capacty the VM receves. Thus, ρ = g(λ acc,f ). Moreover, constrant (h) guarantees the stablty condton by lmtng ρ to υ %. A drect consequence of the above constrants s that λ acc s lmted by constrants: (e), (g), and (h), by the expected arrval rate, the response tme satsfacton, and the maxmum allowed utlzaton respectvely. The capacty allocaton constrant s expressed n (). Ths sum lmts the capacty percentages assgned to the VMs to %. Fnally, constrants (j) and (k) delmt the doman of the varables. The optmzaton model presented n ths secton can be combned wth several performance predcton technques to estmate the values of the performance-related varables λ sat and P (R R SLA ), whch result n dfferent levels of accuracy. Wth ths objectve, n the next secton, we dscuss methods based on queung models. D. Estmatng the Performance Metrcs Ths secton presents an analytcal queung model to calculate the values of the performance metrcs used n the optmzaton model presented n Secton IV-C. As dscussed earler, the most challengng part of the performance predcton for the model consdered s the estmaton of the probablty dstrbuton of response tmes. Ths s because ths exact probablty dstrbuton of response tme can only be computed for some types of queues whch do not drectly apply to the system consdered, and some of the avalable expressons are complex and, therefore, lmt the optmzaton model real tme computaton [2]. Thus, we propose and compare dfferent approxmatons for ths purpose. The followng assumptons hold for the queung analyss: Snce there s consderable evdence that the arrval process mposed by users follows Posson [3], [2], [4], we assume Posson request arrvals. Moreover, for the sake of smplcty, we assume that the servce tmes of applcaton classes are exponentally dstrbuted and left the study of other traffc patterns, whch are characterstc of specfc applcatons, as future work. We consder two types of queues for modelng transactonal servce centers, namely M/M/ and M/G/ wth processor sharng (PS). The former works under FCFS schedulng dscplne, whch may be more accurate for

7 modelng, for example, transactonal servers wth wrte operatons, whereas the latter serves customers n a round robbn fashon wth a very small quantum, whch closely mmcs the behavor of mult-threaded servers. Both queues have been frequently consdered as reasonable abstractons for transactonal servce centers [2], []. Let us start by fndng the saturatng throughput of VM. Applyng the Utlzaton Law [] results n the followng expresson for the utlzaton: ρ X. Thus, the saturatng throughput s determned to be X = ρf f E[S ] = E[S] f E[S ] = λ sat. Notce that wth ths result, constrant (h) of Fgure 4 mposes the followng lmtaton on the rate of acceptance of requests: λ acc υ f /E[S ]. Let us now turn our attenton to approxmatons to compute the tal probablty dstrbuton of the response tme. We consder s Inequalty [5] as a frst approxmaton, requrng only the expected response tme of the classes, whch s computed usng the same expresson for both M/M/ and M/G/ (PS) queues [6]: E[R ]= E[S]/f ρ. Thus, usng the mean response tme and guaranteeng the stablty condton, t s possble to approxmate the tal dstrbuton response tme requrement as: P (R R SLA ) E[R ] R SLA = E[S ] R SLA f λ acc E[S ] α (2) The advantage of usng s Inequalty s that t only requres the average response tme and can be appled to both M/M/ and M/G/ (PS) nterchangeably. Nevertheless, Equaton 2 often provdes a loose bound and, therefore, t could result n overly strct requrements of response tme. Often, t s possble to mprove upon s nequalty to obtan a tghter bound by usng s Inequalty [5]. However, ths result depends on both the average and the varance of the response tme. The latter metrc s queue dependent and, for the M/M/ case, s gven by Var[R ]= (E[S ]/f ) 2 ( ρ ) [6]. For M/G/ (PS) queues, the computaton of 2 the response tme varance uses an ntegraton term whch makes the optmzaton problem hard to solve. However, t can be shown that Var[R ] ρ(e[s2 ]/f 2 ) ( ρ ) gves a tght upper 3 bound of the varance for a PS queue [7], especally for large jobs. Notce that the latter equaton requres both the frst and second moment of the servce tmes, whch we assume can be drectly measured n a pre-producton executon of the applcaton on the physcal server. Gven the mean and the varance of the response tme of applcatons, s nequalty lmts the probablty of a transacton s response tme beng greater than the SLA threshold as: P (R R SLA ) Var[R ] (R SLA E[R ]) 2 α (3) Although Equaton 3 often gve more precse bounds than Equaton 2, t requres more nformaton, such as the queue type and the second moment of servce tmes. The last proposed approxmaton uses the response tme CDF exact soluton for calculatng the percentle of the response tmes for the M/M/ queue [6] so as to express the tal dstrbuton response tme requrement as: P (R R SLA )=e RSLA (f /E[S ])( ρ ) α (4) Notce that Equaton 4 s only exact when there s no rejecton of requests. Thus, snce we are assumng a general admsson control polcy, ths expresson s also an approxmaton. The equvalent result for M/G/ (PS) queue s stll an open problem [7]. In Secton V, we compare the level of accuracy provded by each of the approxmatons,, and Percentle, presented n ths secton. V. EXPERIMENTAL ANALYSIS In ths secton, we evaluate the effcacy of the capacty management model as well as the approxmatons proposed to model the classes performance. Our results are derved from the dscrete event smulaton of the envronment consdered. The smulator executes the classes concurrently, each runnng on a separate VM. The capacty manager component s called perodcally n order to change the settngs of each VM, that s, f and λ acc.we have also nstrumented the smulator n order to collect performance measurements, such as throughput, response tme, queue length and utlzaton of VMs. Usng the approxmatons presented n Secton IV-D, the optmzaton model becomes a non-lnear problem (due to the non-lnear constrant of response tme). Thus, we have conducted the experments usng the non-lnear solver DONLP2 [8]. Experments wth up to 5 applcaton classes resulted n soluton tmes under one second n a Athlon 2.2 GHz wth 52 MB of man memory. A. Expermental Setup In the experments descrbed n ths secton, as t suffces to assess the quanttatve results, we present results for two competng VMs mantaned on top of the shared physcal nfra-structure. We have run experments varyng the number of VMs up to 5, and the qualtatve results are smlar. It s known that dfferent admsson control polces have dfferent effects on the performance of the system [8]. Snce the man concentraton n ths study s on the resultng effect of employng the capacty management model, for the sake of smplcty, n the followng set of experments, a strct and conservatve admsson polcy based on tokens s employed. That s to say that at each second a lmted number of tokens s set up, whch transactons need to acqure n order to enter the system. Ths lmt s confgured (dynamcally) at each VM, usng the value of varable λ acc, after solvng the optmzaton model. In the experments nvolvng adaptaton, the selecton and evaluaton of the workload forecastng method s out of the scope of ths dscusson. In fact, one can use one of the exstng

8 Arrvals Tme (a) Class Arrvals Tme (b) Class 2 Fg. 5. Synthetc perodc step-lke Posson arrval processes submtted to a) applcaton class b) applcaton class 2. tme seres forecast methods [] and obtan results that vary wth dfferent degrees of accuracy. Instead, we assume an deal forecastng, meanng that the future request arrval rate s known apror. We also assume that there s no tme lmtaton for adaptng the system. The set of experments conducted to assess the adaptve approach were drven by synthetc workloads. Workload generators are used to submt requests followng a perodc steplke non-homogeneous Posson processes [5] whose shape s smlar to the ones presented n Fgure 5. Ths fgure presents the arrvals submtted to each of the classes where the average arrval rate n the traces ranges from to arrvals per second, each perodc cycle lasts for seconds, and each step lasts for seconds. Notce that we ntroduce a shft of 5 seconds n the arrval process presented n Fgure 5(b) wth respect to that n Fgure 5(a) n order to produce a dsplacement of ther perods. Ths scenaro makes for an nterestng analyss of the ablty of the system to redeploy ts capacty, by assgnng the dle capacty of the VM experencng low workload actvty to the overloaded one. Gven the characterstcs of the workload, the system s confgured to adapt at the end of each workload step, that s seconds, and, snce the average request rate s stable over the steps, we bypass P {Z } by settng t to. In addton, as the man focus s on the ablty of the autonomc approach to adapt the system n response to workload changes, equal requrements are set up for the classes, whch are summarzed n Table III. Last, snce we am at studyng the performance of the vrtual machnes only, the vrtualzaton layer allows us to completely abstract the physcal topology of the data center. For the case n whch the performance of the classes s approxmated as a M/M/ queue, we smulate a physcal server n whch both applcatons are served wth exponentally dstrbuted servce tme whose parameter s E[S ]= 3, usng FCFS schedulng dscplne. For the M/G/ (PS) case, the applcatons experence the same servce tme dstrbuton wth the same parameter, but wth processor sharng schedulng dscplne. Notce that the maxmum throughput of each applcaton usng the entre physcal server capacty s tps. As a consequence, the server s underprovsoned to serve the two applcaton workloads smultaneously as ther average request rates may vary from to tps each. TABLE III SYSTEM CONFIGURATION υ α X NSLA X SSLA R SLA c π As a reference, we have run the classes wthout the capacty manager nterventon, referred to here as the approach. In the latter, the system s started up wth the results obtaned from the optmal soluton obtaned from the best capacty allocaton model found for each queue (see Secton V-B), consderng the average value of the workload over the entre smulaton perod. B. Numercal Results In sequence, the effect of the concurrent executon of the classes for the M/M/ and M/G/ (PS) modelng cases, usng the three approxmatons, namely,, and Percentle s analyzed. We present the average payoff over runs of the executon of the classes for each modelng case, throughout a perod correspondng to half an hour, along wth the 9% confdence nterval of the averages n Fgure 6. The results presented n Fgure 6 show that for the M/M/ case, usng the approach, the payoff for the executon vares approxmately from 4 to 2 whle usng the approxmaton ths value vares between 24 and. The payoff obtaned from the approxmaton s slghtly smaller than the payoff obtaned by the Percentle approach even though the two curves overlap most of the tme. These latter two approxmatons produced values that range approxmately from to 2. In the M/G/ (PS) modelng case, the approach produces payoffs varyng approxmately from 3 to 3, whle usng the approxmaton these values range from 25 to, and usng from 9 to 24. Notce that the repeatng pattern of the curves are produced by the perodc behavor of the workloads submtted to the VMs. In ths case, the peaks of payoff values for the statc case corresponds to perods of tme n whch the two workloads approach ther average values together. Accordngly, these are perods where the adaptve approaches obtan ther smaller payoff, as there s lttle opportunty to assgn dle capacty of one VM to another. Indeed, perods where the peak of the workload submtted to one VM complements the valley of the other are the cases n whch the adaptve approaches acheved the hghest payoff values. It s nterestng to analyze the performance behavor of the classes by analyzng the acheved response tme, queue length and throughput throughout the runs. Fgures 7 and 8 show the CDFs of the queue lengths at both VMs, collected at each transacton departure. The two fgures show that, for the M/M/ case, when the system s adapted usng any of the approxmatons, the queue length consstently remans at nearly 5, whereas when there s no adaptaton, the queue lengths vary from 3 to 48 n approxmately 75%

9 Average Payoff Percentle Controler Interval (a) M/M/ Average Payoff Controller Interval (b) M/G/ (PS) Fg. 6. Average payoff for the executon of applcatons produced by the adaptve approach, usng the Percentle, and approxmatons, compared wth the approach a) for the M/M/ case b) for M/G/ (PS) case. P(QL <= x) Percentle Queue Lenght (a) VM P(QL <= x) Percentle Queue Lenght (b) VM 2 Fg. 7. CDF of the queue lengths over tme of a) VM and b) VM 2 for the M/M/ case P(QL <= x) Queue Lenght (a) VM P(QL <= x) Queue Lenght (b) VM 2 Fg. 8. CDF of the queue lengths over tme of a) VM and b) VM 2 for the M/G/ (PS) case. of the cases. For the M/G/ (PS) case, usng the approxmaton, the queue length stays at most of the tme; usng the approxmaton, ths value s 5; andwhen no adaptaton takes places, t vares between 3 and 48 n approxmately 75% of the cases. Notce that n the cases where there s adaptaton, the queue lengths stay wthn the theoretcal value of the average number of customers for M/M/ and M/G/ (PS) queues, gven by

10 P(R <= x) Percentle R (sec) (a) VM P(R 2 <= x) Percentle R (sec) (b) VM 2 Fg. 9. CDF of the response tmes of a) VM and b) VM 2 for the M/M/ case. P(R <= x) R (sec) (a) VM P(R 2 <= x) R 2 (sec) (b) VM 2 Fg.. CDF of the response tmes of a) VM and b) VM2 for the M/G/ (PS) case Q = ρ2 ρ. Replacng ρ by the maxmum utlzaton values υ =.95, at whch we planned the VMs operaton, gves Q =8.5. In order to analyze the satsfacton of the SLA tal dstrbuton response tme requrement durng the runs, we have plotted the CDFs of the response tmes for the VMs n Fgure 9 for both modelng cases. Notce that the response tme requrement s met for both VMs, n the two modelng cases, when usng the adaptve approaches wth any approxmaton. Accordngly, the response tme of the transactons were shorter than. seconds more than 9% of the tme. By contrast, usng the statc approach the response tme threshold s respected approxmately 55% and 6% of the tme for the VMs and 2 respectvely n the M/M/ case, and approxmately 57% for both VMs n the M/G/ (PS) cases. Up to ths pont, we have shown that the response tme requrements of the classes are attaned by usng any of the approxmatons for the two modelng cases and, therefore, they are equally effectve for guaranteeng the response tme requrement satsfacton. In contrast, the resultng throughput explans why one approxmaton results n hgher payoffs than the others. In order to observe ths effect, let us analyze the magntude of the penaltes and rewards acheved by each class separately for both modelng cases. We present the CDFs of the magntude of the penaltes and rewards over the controller ntervals for each VM, for the M/M/ modelng case n Fgure and for the M/G/ (PS) case n Fgure 2. Clearly, n all the plots, the statc approach resulted n sgnfcantly hgher penaltes and smaller rewards n comparson to the results provded when the system s adapted. In addton, the values resultng from the approxmaton are also smaller than the ones acheved when usng and Percentle. These latter two approaches produced curves that almost concde. For the M/M/ case, ths s clear n the values correspondng to rewards for both Fgures (a) and (b), and s also observed n the values of penaltes n the Fgure (b) only. Ths s because, n ths experment, the classes have the same requrements, and, thus, the solver favored VM n these cases, snce ths decson produces a equally optmal soluton as dstrbutng the capacty evenly between the classes. For the M/G/ (PS) queue, ths dfference s also observable, especally n reward values. Ths result

11 P(Magntude <= x) Percentle Magntude (a) VM P(Magntude <= x) Percentle Magntude (b) VM 2 Fg.. CDFs of the magntude of penaltes and rewards produced by a) VM and b) VM 2, for the M/M/ case. P(Magntude <= x) Magntude (a) VM P(Magntude <= x) Magntude (b) VM 2 Fg. 2. CDFs of the magntude of penaltes and rewards produced by a) VM and b) VM 2, for the M/G/ (PS) case. conforms to Fgure 6(b) where the payoff dfference produced by and approxmatons s greater n the hgher values whle they nearly concde n the smaller values. In concluson, we can see that the use of the adaptve approach proposed s able to sgnfcantly ncrease the net result from penaltes and rewards n comparson to the approach. We can observe that the approxmaton s more conservatve than the others, because t over-estmates capacty needs due to the response tme requrements. On the other hand, the approxmaton produces average payoffs slghtly smaller than the Percentle soluton for the M/M/ case. However, we have observed through other experments that when the rato λ λ ncreases, thereby alterng acc the Posson characterstc of the arrval rate, approxmaton s able to produce better results than Percentle. Moreover, can also be appled to the M/G/ (PS) modelng case, producng mprovements n comparson wth the approxmaton. VI. RELATED WORK There have been several proposals to maxmze the captalzaton on the strategc advantages that shared data centers may delver. For nstance, Ross and Westerman [] study the mpact of the maturty of both busness and technologcal strands on the revenue of provders; n [2], the authors present evdences, showng that busness-orented desgn for shared envronments reduces the fnancal rsks assocated wth servce level volatons. Rappa [9] proposes servce contracts for IDCs smlar to the ones used for publc utltes, wheren customers would pay only for actual use, and shows the potental of these agreements for future computng servces. Regardng the systems ssues, the authors of [4] present the man technologcal challenges n the path to the maturty of data centers, whch am at adaptng applcatons to ths new paradgm; Graupner et al [6] dscuss the mpact of vrtualzaton on the management of shared envronments. The manual management of resources operated by humans has become ncreasngly unsutable for modern computng systems. In the lght of ths fact, autonomc (or self-managng) approaches appeared as a soluton for adequately admnster the complexty of such systems. In ths drecton, prevous work consdered autonomc closed control loops usng dfferent technques for a dversty of purposes. For nstance,

12 a model based on a queung model that perodcally reconfgures the parameters of a web server so as to ncrease ts performance s presented n [2]. The authors n [7] consdered mappng the workload level of a data center to ts observed nfluence on the system. Amng at maxmzng the sum of utlty functons of the performance, they rely on a state-space based on the past behavor so as to provde nformaton for future capacty allocaton decsons. The authors n [2] further revsted the latter framework, proposng a combnatoral search technque together wth queung models to mprove the effcency of the table-drven approach. Other analyses propose a control theoretcal approach [22], [3]. However, the approach proposed n ths paper s based on an optmzaton model whch s able to express the mportant propertes, constrants and, more mportantly, bnd the performance of the hosted servces to an SLA cost model drven by penaltes and rewards. Moreover, unlke ours, these prevous work manage resources wth respect to determnstc requrements of response tme (.e., maxmum average response tme). resource management optmzaton problems n IDCs has been dealt wth n several studes. However, they usually have dfferent goals. For example, [23] optmzes the resource allocaton n order to mnmze network traffc and resource underutlzaton; studes [2] and [3] deal wth the problem of resource allocaton for maxmzng SLA profts of an e- commerce provder. However, they assume statc workload, abstract the whole servers as dscrete unts of capacty allocaton, and work wth response tme requrements alone. In contrast, our work presents an SLA busness model to handle the operatonal challenges posed by new customer demands and presents dfferent approxmatons to capture the realstc performance behavor of the servces, measured through the processng rate subjected to a response tme guarantee. VII. CONCLUSIONS In ths work we have consdered a self-adaptve capacty management approach for a mult-servce envronment drven by a cost model based on SLA contracts wth the goal of best explorng the IDC s resources. At the busness level, we propose a two-level SLA specfcaton for dfferent operaton modes that works wth a cost model based on penaltes and rewards, whch allows the per-use servce accountng wth respect to the dual SLA requrement of throughput and tal dstrbuton response tme. In the system level, we evaluate approxmatons based on queung theoretcal formulas for predctng the performance of the hosted servces under two dfferent schedulng dscplnes, namely FCFS and Processor Sharng. The system and busness levels are lnked by an optmzaton model whch allows the capacty manager to adapt the IDC to changng capacty needs n real tme so as to maxmze a provder s fnancal objectve. Fnally, we have demonstrated that the use of the proposed self-adaptve model can effectvely manage the capacty allocaton so as to sgnfcantly ncrease the fnancal value derved from the IDC. In addton, we have also presented the dfference n the level of accuracy resultng from the use of dfferent approxmatons to express the tal dstrbuton requrement of response tme usng queung models. ACKNOWLEDGMENTS Ths work was developed n collaboraton wth Hewlett Packard Brazl R&D (Project CAMPS HP-UFMG-25). REFERENCES [] J. W. Ross and G. Westerman, Preparng for utlty computng: The role of IT archtecture and relatonshp management, IBM Systems Journal, vol., no. 43, pp. 5 9, 24. [2] M. J. Buco, R. N. Chang, L. Z. Luan, C. Ward, J. L. Wolf, and P. S. Yu, Utlty computng SLA management based upon busness objectves, IBM Systems Journal, vol., no. 43, pp , 24. [3] X. Lu, X. Zhu, S. Snghal, and M. Arltt, Adaptve enttlement control of resource contaners on shared servers, n 9th IFIP/IEEE Int l Symposum on Integrated Network Management, 25. [4] J. Wlkes, J. Mogul, and J. Suermondt, Utlfcaton, Hewlett-Packard Laboratores, Palo Alto, CA, USA, Tech. Rep., 24, hpl [5] G. Banga, P. Druschel, and J. Mogul, Resource contaners: A new faclty for resource management n server systems, n 3rd USENIX Symposum on Operatng Systems Desgn and Implementaton, 999. [6] S. Graupner, R. Köng, V. Machraju, J. Pruyne, A. Saha, and A. V. Moorsel, Impact of vrtualzaton on management systems, Hewllet- Packard, Tech. Rep. HPL-23-25, 23. [7] W. E. Walsh, G. Tesauro, J. Kephart, and R. Das, Utlty functons n autonomc computng, n IEEE Internaltonal Conf. Autonomc Computng (ICAC 4), 24. [8] H. G. Perros and K. H. Elsayed, Call admsson control schemes : A revew, IEEE Magazne on Communcatons, vol. 34, no., pp. 82 9, 996. [9] M. Arltt and T. Jn, Workload characterzaton of the 998 world cup web ste, Hewllet-Packard Labs, Tech. Rep. HPL R, 999. [] B. Abraham and J. Ledolter, Statstcal Methods for Forecastng. John Wley and Sons, 983. [] D. Menascé, V. Almeda, and L. Dowdy, Performance by Desgn. Prentce Hall, 23. [2] D. Vllela, P. Pradhan, and D. Rubensten, Provsonng servers n the applcaton ter for e-commerce systems, n IEEE Int l Workshop on Qualty of Servce, June 24. [3] Z. Lu, M. Squllante, and J. L. Wolf, On maxmzng servce-levelagreement profts, n ACM Eletronc Commerce Conference, October 2. [4] V. Paxson and S. Floyd, Wde-area traffc: The falure of posson modelng, IEEE/ACM Transactons on Networkng, vol. 3, no. 3, pp , 995. [5] A. Papouls and S. U. Plla, Probablty, Random Varables, and Stochastc Processes, 4th ed. McGraw-Hll, 22. [6] L. Klenrock, Queueng Systems Volume I: Theory. Wley Interscence, 975. [7] S. F. Yashkov, Processor-sharng queues: Some progress n analyss. Queung Systems, vol. 2, pp. 7, 987. [8] P. Spellucc, An SQP method for general nonlnear programs usng only equalty constraned subproblems, Mathematcal Programmng, vol. 3, no. 82, pp , 998. [9] M. A. Rappa, The utlty busness model and the future of computng servces, IBM Systems Journal, vol., no. 43, pp. 32 4, 24. [2] D. A. Menascé and M. N. Bennan, On the use of performance models to desgn self-managng computer systems, IEEE Dstrbuted Systems, vol. 6, no., 25. [2] M. Bennan and D. Menascé, Resource allocaton for autonomc data centers usng analytc performance models, n IEEE Int l Conference on Autonomc Computng, 25. [22] Y. Dao, N. Gandh, J. Hellersten, S.Parekh, and D. M. Tlbury, Usng MIMO feedback control to enforce polces for nterrelated metrcs wth applcaton to the apache web server, n IEEE/IFIP Network Operatons and Management Symp., 22. [23] X. Zhu, C. Santos, J. Ward, D. Beyer, and S. Snghal, Resource assgnment for large-scale computng utltes usng mathematcal programmng, Hewllet-Packard Labs, Tech. Rep. HPL R, 23.