Service level management, SLA, performance model, muti-tier applications. Copyright 2008 Hewlett-Packard Development Company, L.P.

Transcription

1 Systematcally Translatng Servce Level Objectves nto Desgn and Operatonal Polces for Mult-Ter Applcatons Yuan Chen, Akhl Saha, Subu Iyer, and Dejan Mlojcc HP Laboratores Palo Alto HPL February 20, 2008* Servce level management, SLA, performance model, mut-ter applcatons We propose a systematc and practcal approach that combnes fnegraned performance modelng wth regresson analyss to translate Servce Level Objectves (SLOs) nto desgn and operatonal polces for mult-ter applcatons. These polces can then be used for desgnng a servce to meet the SLOs and montorng the servce thereafter for volatons at runtme. We demonstrate that our approach s practcal and can be appled to commonly used mult-ter applcatons wth dfferent topologes and performance characterstcs. Our approach handles both request-based and sesson-based workloads and deals wth workload changes n terms of both request volume and transacton mx. Our approach s non-ntrusve n the sense that t requres no specalzed proflng,.e., the data used n our approach s readly avalable from normal system and applcaton montorng. We valdate our approach usng both the RUBS e-commerce benchmark and a trace-drven smulaton of a busness-crtcal enterprse applcaton. These results show the effectveness of our approach. Internal Accesson Date Only Approved for External Publcaton Copyrght 2008 Hewlett-Packard Development Company, L.P.

2 Systematcally Translatng Servce Level Objectves nto Desgn and Operatonal Polces for Mult-Ter Applcatons Yuan Chen, Akhl Saha, Subu Iyer, and Dejan Mlojcc Hewlett Packard Labs ABSTRACT A Servce Level Agreement (SLA) contans one or more Servce Level Objectves (SLOs) that descrbe the agreed upon qualty requrements at the servce-level. In order to manage the servce to meet the agreed upon SLA, t s mportant not only to desgn a servce of the requred capacty but also to montor the servce thereafter for volatons at runtme. Ths objectve can be acheved by undertakng SLA Decomposton,.e., translatng SLOs specfed n the SLA nto lower-level polces that can then be used for desgn and enforcement purposes. Such desgn and operatonal polces are often constrants on thresholds of lower level metrcs. Tradtonally, doman experts and admnstrators brng ther knowledge to bear upon the problem of SLA decomposton. Ths practce s ad-hoc, manual, and statc (.e., done once). Ths s both costly, and not well suted to dynamc workloads. In the past, there has been a number of efforts to develop more automated and dynamc solutons, but these approaches have many lmtatons and hence pose major challenges to ther applcablty n practce. In ths paper, we propose a systematc and practcal approach that combnes fne-graned performance modelng wth regresson analyss to translate servce level objectves drectly nto desgn and operatonal polces for mult-ter applcatons. We demonstrate that our approach s practcal and can be appled to commonly used mult-ter applcatons wth dfferent topologes and performance characterstcs. Our approach handles both request-based and sesson-based workloads and deals wth workload changes n terms of both request volume and transacton mx. Our approach s non-ntrusve n the sense that t requres no specalzed proflng,.e., the data used n our approach s readly avalable from normal system and applcaton montorng. We valdate our approach usng both the RUBs e-commerce benchmark and a trace-drven smulaton of a busness-crtcal enterprse applcaton. These results show the effectveness of our approach.. ITRODUCTIO A Servce Level Agreement (SLA) captures the agreed upon guarantees between a servce provder and ts customer. The ablty to delver accordng to a pre-defned SLA s an ncreasngly crtcal need n today s hghly complex and dynamc IT envronments. One of the key tasks to SLA management s SLA decomposton, translatng the hgh level servce objectves to low level desgn and operatonal polces that can be then used to ensure the Servce Level Objectves (SLOs) are met. Gven an applcaton/servce and ts correspondng SLOs, the IT operatons team undertakes SLA decomposton by determnng the desgn parameters that nclude dentfyng the operatonal level objectves that are relevant and the healthy ranges for varous operatonal metrcs to satsfy the SLAs. For example, for a gven set of SLOs for an e-commerce applcaton (e.g., response tme requrements), doman experts make decsons about low level desgn and operatonal polces. A desgn polcy settng usually contans system desgn parameters such as how many web servers, applcaton servers and database servers must be allocated to satsfy the specfed SLOs. An operatonal polcy specfes detals of low level metrcs (e.g., system resource utlzaton) to montor, healthy ranges of these metrcs and actons to take when healthy ranges are volated. Such an operatonal polcy can be used for montorng potental volatons and enforcement of SLOs at run tme. Once the system s put nto producton, workloads and assocated SLOs may change durng operaton. As a result, desgn polces need to be adjusted to ensure current system capacty s suffcent to handle the future workload. Operatonal polcy confguratons need to be adjusted as well. Tradtonally, doman experts and admnstrators brng ther knowledge to bear upon the problem of SLA decomposton. Most of the tme, these decsons are made n an ad-hoc manner based on past experence. Ths process nvolves substantal manual effort and adds to the cost of servce desgn and operaton. Hence effectve and effcent SLA decomposton n an automated fashon s a key requrement n SLA management. In the past, researchers have made many efforts to address SLA decomposton usng technques such as automated provsonng, capacty plannng, and montorng [6, 7, 20, 28, 29]. Prevous studes have utlzed performance models to gude resource provsonng and capacty plannng [6, 20]. Urgaonkar et al. propose a dynamc provsonng technque for mult-ter applcatons [6, 7]. All these research efforts separate desgn and operatons nto two phases and mostly descrbe the capacty plannng and resource provsonng aspects of the desgn phase. In addton, these research efforts make several smplfyng assumptons. As a result, the practcalty and effectveness of these approaches pose major challenges to ther applcablty. We have dentfed four man problems assocated wth exstng solutons that are descrbed below. Frst, workloads n real applcatons are dynamc and vary over tme. Unfortunately, most exstng solutons take nto account the change n the volume of demand only, and assume a fxed or statonary transacton mx [6, 7, 28]. Changes n the volume of transactons (e.g., request rate) or the mxture of transacton types can dramatcally alter an applcaton s performance and resource. Hence, a practcal decomposton approach must handle workload changes n both the volume and transacton mx. Second, exstng solutons model enterprse applcaton workloads as ether request-based (open workload) or sesson-based (closed workload) [6, 7, 26, 28]. In realty, workloads are typcally

3 sem-open, whch s sgnfcantly dfferent than ether an open or a closed model [25]. Hence, a sngle model approach n most exstng solutons s not suffcent to handle the dversty n realstc workloads. A practcal approach should support multple models and choose an approprate model that s based on the propertes of the real workload. Thrd, buldng accurate performance models typcally requres nput parameters such as resource demand. However, most exstng solutons cannot provde the needed model parameters drectly. Instead, such nformaton must be obtaned through applcaton or system nstrumentaton. In practce, nstrumentaton of producton applcatons s rarely done, as t s dffcult, costly, and may ntroduce overhead that degrades the applcaton s performance [29]. Hence, a practcal approach should be nonntrusve and passvely utlze data that s already avalable on most systems. Lastly, most exstng solutons are not applcable to the dverse range of desgn and mplementaton choces. Many of them make smplfyng assumptons about the applcaton nfrastructure, such as consderng only one server per ter [7, 26] or unformly dstrbutng the requests across the dfferent ters [26]. To cope wth the dversty and complexty n real applcatons, a model must be suffcently general to capture the behavor of applcatons wth dfferent confguratons, workloads and performance characterstcs. In ths paper, we propose a systematc, non-ntrusve and practcal SLA decomposton approach to address the above ssues. Our approach combnes a fne-gran performance model and a regresson-based proflng approach to derve low-level operatonal polces from hgh-level objectves for mult-ter applcatons. We formalze the decomposton as a constrant optmzaton problem, and develop a constrant solver to solve t. Our approach provdes the followng four key contrbutons. Frst, our approach formally characterzes both request-based and sesson-based workloads. Ths enables us to choose an approprate model based on the workload characterstcs of the applcaton. Second, our approach models workload as a transacton mx, and systematcally creates a resource profle for each transacton type. Ths fne-graned model enables us to deal wth dynamc and non-statonary workloads. Thrd, we use regresson analyss to estmate the model parameters. The data used n our approach s readly avalable from regular system and applcaton montorng and requres no addtonal nstrumentaton. It s hence practcal to apply our approach to producton envronments. Fnally, the proposed modelng technque can model mult-ter applcatons wth dfferent topologes (.e., any number of ters and any number of servers at each ter), and dfferent workloads (open and/or closed). As a result, our performance model and decomposton approach can be appled to a vast varety of common mult-ter applcatons. The remander of ths paper s organzed as follows. Secton 2 provdes an overvew of our approach and our workload model. In Secton 3, we descrbe proflng n detal. We present an analytcal performance model for mult-ter applcatons n Secton 4 and our decomposton approach n Secton 5. The expermental valdaton of our approach s presented n Secton 6. Related work s dscussed n Secton 7. Secton 8 concludes the paper and dscusses future work. 2. OVERVIEW OF OUR APPROACH 2. Defnton of a Mult-ter Applcaton Mult-ter applcatons are common n modern enterprses. Such applcatons are comprsed of a large number of components, whch nteract wth one another n complex patterns. Typcally, mult-ter applcatons are structured nto multple logcal ters. Each ter provdes certan functonalty to ts precedng ter and uses the functonalty provded by ts successor to carry out ts part of the overall request processng. At each ter, a load balancer dstrbutes the overall load among all servers of that ter accordng to certan schedulng algorthms. Consder a mult-ter applcaton consstng of M ters, T, T M. In the smplest case, each request s processed exactly once by each ter and forwarded to ts succeedng ter for further processng. Once the result s processed by the fnal ter T M, the results are sent back by each ter n the reverse order untl t reaches T, whch then sends the results to the clent. In more complex processng scenaros, each request at ter T can trgger zero or multple requests to ter T +. For example, a statc web page request s processed by the Web ter entrely and wll not be forwarded to the other ters. On the other hand, a keyword search at a Web ste may trgger multple queres to the database ter. Type Open Closed Table. Workload defnton Workload Parameters : number of transacton types (λ, λ 2, λ ): transacton mx where λ ( = ) s the arrval rate of requests of transacton type durng certan tme nterval : number of transacton types C: number of users Z: thnk tme π (p, p 2, p,p ) : transacton mx dstrbuton where p ( =, ) s the percentage of requests of transacton type 2.2 Workload Model Defntons There are typcally a number of transacton types n any mult-ter applcaton. For example, an onlne aucton applcaton has transacton types such as logn, browse, bd, etc. In most cases, dfferent transacton types have dfferent servce demands on resources. For example, bd transactons n an aucton ste typcally requre more CPU tme than browse transactons. As prevously dscussed, emprcal workloads tend to be partallyopen, whch means a user arrves and stays for a certan amount of tme (and ssues a number of requests) before they leave. Prevous work has shown that partly-open workloads can be approxmated usng an open workload f the number of requests n a sesson s small, and a closed workload otherwse [25]. We consder these two types of workloads n our workload model Open Workload In an open (request-based) workload, a new request to the applcaton s only trggered by a new user arrval. The requests are ndependent of each other and the arrval rate s not nfluenced by the number of requests that have already arrved and are beng processed. The number of users who nteract wth the applcaton at any tme may range from zero to nfnty. An open workload s

4 characterzed by an average arrval rate of requests or more generally by an arrval dstrbuton. A typcal open workload s a transacton mx of dfferent transacton types. In real producton systems, the transacton mx changes over tme [26]. Assume the total number of transacton types s. We defne an open workload durng a certan nterval (e.g., 5 mnutes) as a vector (λ, λ 2, λ ) where λ s the arrval rate of transacton type durng that nterval Closed Workload In a closed (sesson-based) workload, a fxed number of users nteract wth the applcaton and each of these users ssues a successon of requests. A new request from a user s only trggered after the completon of a prevous request by the same user. A user submts a request, wats for the response of that request, thnks for a certan tme and then sends a new request. The average tme elapsed between the response from a prevous request and the submsson of a new request by the same user s called the thnk tme, denoted by Z. The next request sent by a user s usually determned by a state transton matrx that specfes the probablty to go from one transacton type to another. Assume the number of transacton types s. The state transton matrx has rows and columns where p j represents the transton probablty from transacton type to transacton type j. Let P denote a state transton matrx of a closed workload and π = (p, p 2 p ) denote the statonary transacton dstrbuton n a user sesson where p presents the percentage of requests of transacton type sent by the user based on P. We have πp = π and p. We can use the workload wth a statonary transacton mx π to approxmate the behavor of a closed workload wth state transton matrx P [27]. A closed workload s characterzed by the number of concurrent users C, the statonary dstrbuton of transacton mx π, and the thnk tme Z. The open and closed workload models are summarzed n Table. Unlke many open workload models that assume a statc transacton mx and hence use an aggregate request rate to characterze the workload, our transacton vector model captures request rate per transacton type and hence can characterze dynamc transacton mxes. Smlarly, by explctly ncorporatng the transacton mx dstrbuton as part of the workload parameter n a closed workload, we can capture dfferent behavors wth dfferent transacton dstrbuton. 2.3 Our Approach An SLA s comprsed of multple Servce Level Objectves (SLOs). The task of SLA decomposton s to translate SLOs nto desgn parameters and bounds on low-level system resources such that the hgh-level SLOs are met. Gven a hgh-level performance SLO and a workload for a mult-ter applcaton (n terms of ether a transacton mx for an open workload or a transacton dstrbuton for a closed workload), decomposton provdes the resource requrements (e.g., number of servers) to handle the workload and meet the specfed SLO. It also fnds the healthy state of each component nvolved n provdng the servces (e.g., resource utlzaton). The decomposton process can be summarzed as (R, W) (ŋ web θ web-cpu, ŋ app, θ app-cpu,, ŋ db, θ db-cpu ) Hstorcal or Benchmark Data transacton mx open or closed workload SLOs (e.g., resp. tme) other constrants resource utlzaton Regresson-based Proflng resource demand per transacton type model parameters Constrant Optmzaton Solver Performance Modelng Queung etwork Model Fgure. Conceptual archtecture where R and W denote the response tme and workload respectvely and ŋ s the number of servers at a ter and θ s the resource utlzaton. SLA decomposton problem s the opposte of a typcal performance modelng problem, where the overall system s performance s predcted based on the confguraton and resource consumpton of the sub-components. For example, gven the performance goal of a 3-ter onlne e- commerce applcaton (e.g., response tme<0 seconds), and any workload n term of transacton mx (e.g., browsng=0 reqs/s, add-to-cart=5 reqs/s, and checkout=4 reqs/s), the decomposton approach determnes how many Web servers, applcaton severs and database severs are requred to handle the workload whle satsfyng the specfed response tme requrement. Decomposton further determnes the healthy ranges of the resource utlzaton of each server (e.g., CPU, I/O, network, etc.) under the confguraton durng operaton. Our SLA decomposton approach s llustrated n Fgure. We undertake SLA decomposton n a systematc way. We use analytcal performance models to capture the relatonshp between hgh-level performance goals (e.g., response tme of the overall system), the applcaton topology, and the resource usage of each component (e.g., CPU utlzaton). In partcular, we develop two queueng network models for a mult-ter archtecture, where each ter s modeled as a mult-staton queueng center. One of the two models s chosen based on the propertes of the real workload. We profle the applcatons and generate the resource demand of each transacton type at each resource. Ths s obtaned by performng a statstcal regresson analyss on the hstorcal or benchmark data. The proflng results are stored as the applcaton resource profle n a repostory. Combnng the performance model and the applcaton resource profle, the decomposton problem becomes a constrant satsfacton problem. Gven a performance goal (e.g., response tme), a workload (open or closed n terms of transacton mx) and any other constrants (e.g., CPU utlzaton < 50%), the solver takes the applcaton resource profle and the analytcal model as nputs and generates a lowlevel polcy settng. The output ncludes the resource requrements, such as how many servers at each ter are requred to meet the SLO and the healthy bounds of resource utlzaton for each component. The resource requrement s then used for the desgn and reconfguraton of the applcaton accordngly, whle the healthy range s used for montorng the systems durng model resource requrements (e.g. number of servers) resource utlzaton desg mon

5 operaton. The developed analytcal models and applcaton resource profles are archved for future reuse. If the workload or response tmes change, we only need to re-solve the constrant satsfacton problem wth new parameters to generate a new polcy settng. 3. PROFILIG 3. Proflng Overvew Proflng creates detaled resource profles of each component n the applcaton. A resource profle captures the demand of each transacton type at that resource. Per-transacton type resource demand (e.g., a browse transacton s CPU demand at an applcaton server) s ndependent of the overall transacton mx and hence remans stable despte any changes n a workload. The profle only needs to be created once, and can be used to drve resource demand for dfferent transacton mxes. In order to obtan the resource profle, we frst acqure the measurements on the transacton nformaton and the resource consumpton n dfferent transacton mxes ether from the hstorcal data or through benchmarkng. The latter s used for new applcatons (e.g., n desgn phase) where no system and applcaton logs are avalable. We deploy a test envronment, apply a varety of transacton mxes to the applcaton and collect the transacton and resource consumpton nformaton. Regresson analyss s then appled to the data, to derve the per transacton-type resource demands. The resultng resource profle for the applcaton s then stored n a repostory. The data requred by the regresson analyss ncludes system resource utlzaton, such as CPU usage, and the applcaton workload nformaton, such as transacton mx. The data s readly avalable from system and applcaton montorng logs. Ths way, our proflng s nonntrusve snce t does not requre changes to exstng applcatons and systems for nstrumentaton purposes and hence avods most of the dsadvantages of nstrumentng the applcaton. Further, our fne-graned profles capture the resource demand at pertransacton level and hence can handle dynamc changes n transacton mx. The proflng detal s dscussed below. 3.2 Resource Demand Estmaton Two key objectves of proflng are to accurately estmate the resource demands for the applcaton, and to dentfy the nput parameters for the performance model. The accuracy of a performance model depends drectly on the qualty of ts nput parameters. Our regresson-based proflng s based on the followng observatons. () Typcally, the aggregated resource demand of all transacton types n a workload are measured durng the proflng stage. In most cases, the transacton mx s assumed to be statc. Hence, the results obtaned for a workload only hold for that partcular workload wth the same transacton mx. Ths approach cannot be appled to realstc workloads where the percentage of transacton types changes over tme. (2) The resource demands of dfferent transacton types are usually dfferent but the resource demand of a transacton type s relatvely fxed rrespectve of the transacton mx. Hence, t s better to create a profle for each transacton type (e.g., CPU demand for browse transacton, bd transacton, etc.) nstead of creatng an aggregated profle for the entre workload. The pertransacton type profle remans stable across dfferent transacton mx. (3) Few applcatons are currently nstrumented to measure fnegraned transacton resource nformaton. Hence, accurately measurng the servce demand of each component requres sgnfcant nstrumentaton of the orgnal applcaton. Ths s unrealstc n practce. Snce the resource demand of each transacton type s relatvely statc across dfferent transacton mxes, we can derve the parameter of per transacton types usng regresson-based approaches [26, 27]. (4) The average resource demand of a request n a workload s determned by the dstrbuton of dfferent transacton types n the workload and the servce demand of each transacton type. Once we have per-transacton type profles, gven a new transacton mx, the aggregated resource demand can be derved from pertransacton resource demand. Durng a certan nterval, a resource s usage s the sum of all transacton types demand at that resource, plus a base utlzaton to account for background actvtes that are present n real systems (even when the applcaton s completely dle). Hence, a resource s utlzaton can be obtaned as follows. () U D0 D λ = + where U s the resource utlzaton, denotes the number of transacton types, D 0 represents the background utlzaton of the resource, D represents the resource demand of a request of transacton type at that resource, and λ s average request rate of transacton type. In order to obtan the demand D (, ) at each resource (e.g., CPU, I/O, network), we collect utlzaton data from each resource U as well as the arrval rates of dfferent transacton types λ (, ) over multple tme ntervals (e.g., 5 mnutes, hour). These nputs are generally avalable va system and applcaton montorng logs. The goal of proflng s to compute the resource demand of each transacton type at that resource D, D. Ths problem can be solved by usng lnear regresson on a set of equatons () at multple ntervals. There are numerous dfferent lnear regresson technques that could be utlzed. In ths work, we used Least Squares Regresson (LSR) to obtan the resource demands D ( =,). Other approaches, such as Least Absolute Devatons Regresson (LADR) could also be appled, as they may provde some advantages over LSR (e.g., ncreased accuracy and robustness). We repeat the above steps for each resource and generate the applcaton resource profle. The profle conssts of a set of resource demands of each transacton type. 4. PERFORMACE MODEL Our performance model captures the relatonshp between the overall applcaton performance as a functon of transacton workload, the applcaton confguraton, and the resource performance characterstcs. We utlze a queung network model of mult-ter applcatons. M/M/ queung network model s used to evaluate the performance for open workloads, whle closed queueng network model s used for closed workloads. Our model s suffcently general to model any commonly used mult-ter e- commerce applcaton wth dfferent applcaton topologes and workloads. Our model also handles mult-class users. The performance model s dscussed n detal next.

6 S, V Q S 2, V 2 Q 2... S M, V M Fgure 2. Basc queueng network model K, D Q K 2, D 2 Q 2 4. Basc Model An applcaton wth M ters s modeled as a queueng network of M queues Q, Q 2,...Q M. (see Fgure 2). Each queue represents an ndvdual ter of the applcaton and the underlyng server t runs on. A request, after beng processed at queue Q ether proceeds to Q + or returns to Q -. A transton to the clent denotes a request completon (.e., response to the clent). We use V to denote the average number of vsts to queue Q by a request. Our model can handle multple vsts to a ter. Gven the user request arrval rate λ, the request arrval rate at ter can be approxmated as V λ. Gven the servce demand D of a request per vst to ter, the average servce demand per user request at ter can be approxmated as V D. Realstc mult-ter applcatons typcally utlze a multserver/processor archtecture to hand a large number of requests. The applcaton server ter for example may nvolve one or more applcaton servers (e.g., JBoss). A smlar noton s applcable to the database ter whch may consst of one or more database servers (e.g., MySQL). In order to capture the multserver/processor archtecture, we enhanced the basc model by usng a mult-queue center to model each ter (see Fgure 3). In ths model, each server/processor n the ter s represented by a queue. The mult-queue model thus s a general representaton of a ter. We use K to denote the number of servers at ter. Ths model represents the mult-server archtecture commonly utlzed by mult-ter applcatons. 4.2 Performance Model for Open Workloads Consder the followng notaton. K M, D M M: number of ters (e.g., Web, APP, DB) : number of transacton types (e.g., Browse, Bd) R: number of resources types (e.g., CPU, DISK) η k : number of servers at ter k (k =, M) (λ, λ λ ): open workload where λ s the average request rate of transactons type D k : servce demand of transacton type at a server of ter k ( =,, k =,M) U kj : utlzaton of resource type j at ter k (j=r, k =, M) Q M Fgure 3. Mult-queue model Q M : ndex of transacton type j: ndex of resource type k: ndex of ter Assume we have a perfect load balancer that evenly dstrbutes the load among all servers of each ter. We model a ter wth K servers as K M/M/ queues. The total servce tme of a request at ter k s the weghted sum of each transacton type s servce tme λ Dk η k R 2 U jk s j= Ujk. The watng tme on a resource type k at ter j. The total resdence tme of all requests at ter k s the servce tme plus the watng tme R λ Dk η + k j= 2 Ujk Ujk. The average response tme s the sum of the resdence tmes at each ter dvded by the overall request rate. R 2 Ujk m λ Dk+ η k j= U jk RT = (2) λ k= η k Gven the applcaton profle, the utlzaton U of each resource can be obtaned as follows U D0 D λ η = + (3) The overall resource demand D of a transacton type at a server s the sum of all resource demand (e.g., CPU, DISK) at that server. Ths model s suffcently general to capture typcal mult-ter applcatons wth multple transactons types and multple servers at each ter. Gven the parameters of the applcatons, the applcaton resource profle and an open workload M: number of ters (e.g., Web, APP., DB) : number of transacton types (e.g., Browse, Bd) R: number of resources types (e.g., CPU, DISK) η k : number of servers at ter k (k =, M) D k : servce demand of transacton type at a server of ter k (λ, λ λ ): transacton mx Equaton (2) s used to predct the response tme and Equaton (3) s used to derve the resource utlzaton. Unlke most performance models, our model takes nto account the multserver structure and represents mult-ter applcatons at a fnegranular level (.e., per transacton type per resource characterzaton). As a result, our performance model can be appled to general mult-ter applcatons wth dfferent applcaton topology and open workload wth dynamc transacton mx. 4.3 Performance Model for Closed Workloads Consder a closed workload wth C users and thnk tme Z. In order to capture the closed workload and the concurrency of multple users, we use a closed queueng network, where we model C concurrent users as C delay resources wth each of them exhbtng a servce demand Z. Fgure 4 shows the closed multstaton queueng network model (QM) of a mult-ter applcaton. Each ter s modeled as a mult-staton queueng center, wth the number of statons beng the ter s total number of

7 servers and each user s a delay center wth servce tme equalng thnk tme Z. We use K to denote the number of servers at ter. Smlarly, the servce demand at a server of ter s denoted by D. Gven any closed workload n terms of number of users C and the transacton mx percentage π = (p, p 2 p K ), the average servce demand of the workload D can be computed from the applcaton profle as the weght average of the servce demand of each =. ndvdual transacton D p D Gven the parameters {C, Z, M, K, D }, the proposed closed queueng network model can be solved analytcally to predct the performance of the underlyng system. For example, an effcent algorthm such as the Mean-Value Analyss (MVA) can be used to evaluate the closed queueng network models wth exact solutons [5]. MVA algorthm s teratve. It begns from the ntal condtons when the system populaton s and derves the performance when the populaton s from the performance wth system populaton of (-), as follows Dk delay resource Rk ( ) = Dk ( + Qk ( ) queueng resource X ( ) = K k= R ( ) Q ( ) = X R ( ) k C, Z, V 0 Users Fgure 4. Closed mult-staton queueng network model, Z, V 0 k k K, S, V Q V, D, DD Q K 2, S 2, V 2 where Rk ( ) s the mean response tme (mean resdence tme) at server k when system populaton s ; Rk ( ) ncludes both the queueng tme and servce tme; X() the total system throughput when system populaton s ; and Qk ( ) s the average number of customers at server k when system populaton s. Tradtonal MVA has a lmtaton that t can only be appled to sngle-staton queues. In our model, each ter s modeled wth a mult-staton queueng center. To solve ths problem, we adopt an approxmaton proposed by Sedmann et al. [6] to get the Q 2 V 2, D 2, DD 2 K M, S M, V M Users Fgure 5. Approxmate model for MVA analyss Q 2 Q M V M, D M, DD M Q M Input: C, Z, M, K, D, ( =,.. M) Output: R, X //ntalzaton R 0 = Z; D 0 = Z; Q 0 = 0; for = to M { // Tandem approxmatons for each ter Q = 0; qrd = D /K ; drd = D ( K -)/K ; } //ntroduce C users one by one for = to C { for j = to M { R j = qrd j ( + Q j); // queueng resource RR j = drd j; //delay resource } X = m R + ( R+ RR) } 0 j= for j = to M Q j = X R j; M R = ( R + RR) Fgure 6. Modfed MVA algorthm approxmate soluton of performance varables. In ths approxmaton, a queueng center that has m statons and servce demand D at each staton s replaced by two tandem queues. The frst queue beng a sngle-staton queue wth servce demand D/m, and the second queue s a pure delay center, wth delay D (m- )/m. It has been shown that the error ntroduced by ths approxmaton s small [7]. By usng ths approxmaton, the fnal queueng network model s shown n Fgure 5. The modfed MVA algorthm used to solve our queueng network s presented n Fgure 6. The algorthm takes the followng set of parameters of a mult-ter applcaton as nputs: C: number of users Z: thnk tme M: number of ters K : number of statons at ter ( =,, M) D : servce demand of a server at ter ( =,, M) The MVA algorthm computes the average response tme R and throughput X of the applcaton. 4.4 Handlng Mult-class Users There are typcally multple classes of users or sessons n real applcatons, representng dfferent SLAs (e.g., Gold customers, Slver customers and Bronze customers) and heterogeneous workloads (e.g., browsng-heavy transactons, purchase-heavy transactons). Constructng multple-class models for a heterogeneous workload can accurately model heterogeneous workloads and dfferentate SLA requrements of dfferent classes. Such classfcaton enables flexble admsson control based on the mportance of the class, e.g., preferentally schedulng requests from more mportant classes and droppng less mportant requests durng overload. We extend our model to handle multple classes of users. Interested readers, please refer to Appendx for the detals.

8 5. DECOMPOSITIO Gven an SLO (e.g., response tme) and a workload, the goal of decomposton s to determne the desgn parameters (e.g., number of servers at each ter) to guarantee that the system has enough capacty for processng the specfed workload and meetng the proposed SLO. The output of decomposton contans operatonal polcy settngs such as how many servers are requred for each ter what s the CPU, Memory, IO utlzaton of each server As we dscussed before, we generate the profle based on the hstorcal data or benchmarkng data wth varyng workloads. The servce demand of each ndvdual transacton type s retreved from the archve, as shown n Fgure. Gven any workload and a response tme requrement, the task of decomposton s then to fnd the set of model nput parameters such as number of servers that satsfy the response tme requrement and further derve the resource utlzaton. Decomposton thus becomes a constrant satsfacton problem. We have developed a smple constrant satsfacton solver to solve ths problem. The solver takes performance goal, workload, resource profles and performance model as nputs and constructs a set of constrant equatons. Varous constrant satsfacton algorthms, such as lnear programmng and optmzaton technques, are avalable to solve such problems [2]. Typcally, the soluton s non-determnstc and the soluton space s large. However, for the problems we are studyng, the search space s relatvely small. For example, f we consder assgnng the number of servers at each ter, we can effcently enumerate the entre soluton space to fnd a soluton. Also, we are often nterested n fndng a sngle feasble soluton (rather than the optmal soluton), so we can stop the search once one s found. Other heurstc technques can also be used durng the search. For example, the hnt that the response tme typcally decreases wth respect to the ncrease of allocated resources can also reduce the search space. One advantage of our approach s that once the profle and model are created, they can be repeatedly used to perform decomposton for dfferent SLOs and workloads. That s, f the response tme or workload changes, we only need to resolve the constrant satsfacton problem wth the new parameters. Smlarly, f the applcaton s deployed to a new envronment, we only need to regenerate the profle n that envronment usng regresson analyss. Further, gven hgh-level goals and resource avalablty, we can apply our decomposton approach for automatc selecton of resources and for the generaton of szng specfcatons that could be used durng system deployment. 5. Decomposton for open workloads The performance model for open workload can be represented as follows. RT = m k= R U λ Dk+ ηk j= U λ ηk U D0 D λ η jk 2 jk (2) = + (3) Gven an open workload n terms of transacton mx dstrbuton (λ, λ λ ) and a specfed SLO of RT < r, the decomposton problem s to fnd a set of η, η 2, η M that satsfy the constrant RT < r as well as determne the resource utlzaton U jk under the confguraton. Other constrants can be added, such as U cpu < 50%, U dsk < 60%. To fnd the soluton of the above equatons, our current solver smply enumerates all combnatons of dfferent number of servers that satsfy the constrant and then chooses the combnaton such that the number of servers s mnmzed. Once we get the η, η 2, η M, the resource utlzaton can be computed based on equaton (3). Implementng a more effcent solvng algorthm (e.g., from Zhang et al. [2]) s left for future work. 5.2 Decomposton for closed workloads For closed workloads, the performance model does not have a closed form (as does the open model), but the model can be conceptually represented as follows. RT = g M C Z η η D D (,,,,..., M,,..., M) where M s the number of ters, and varables RT and C denote response tme and the number of concurrent users respectvely. Varables ŋ j and D j represents the number of servers and average servce demand at ter j respectvely. Please see Secton 4 for the defntons of the other varables. The average servce demand D j can be estmated usng the weghted average resource demand of each transacton type n a user sesson. That s, gven a user sesson s transacton mx dstrbuton (p, p 2, p,p k ) and the resource demand of each transacton type at the resource T : D, T 2 :D 2,,T :D, the average resource demand s estmated as D= p D Gven RT < r and a closed workload (n terms of number of users and the transacton mx dstrbutons π = (p, p 2 p K ) of an M- ter applcaton), the decomposton problem s to fnd a set of ŋj (j =, M). Smlar to the decomposton of an open workload, the solver enumerates all combnatons of dfferent number of servers that satsfy the constrant and then chooses the combnaton, such that the number of servers s mnmzed. Once η, η 2, η M are determned, the resource utlzaton s derved accordng to equaton (3). Gven a new workload, the average servce demand s recomputed and the constrant satsfacton problem s solved agan, usng the new servce demand parameters. 6. EXPERIMET EVALUATIO We evaluated our approach wth two applcatons, the popular RUBS e-commerce applcaton wth synthetc workloads and a real busness-crtcal servce wth real traces. 6. RUBS Testbed RUBS s an ebay-lke onlne aucton ste developed at Rce Unversty []. We use a 3-ter EJB-based mplementaton of RUBS consstng of an Apache Web server 2.0, a JBOSS applcaton server, and a MySQL 5.0 database server, each runnng on dfferent servers. The RUBS mplementaton defnes 26 nteractons, has,000,000 users and 60,000 tems. The testbed ncludes multple Lnux servers. Each server has 2.4 GHz CPU, 4 GB of RAM, and a Gb/s Ethernet nterface. We developed a workload generator that can produce both open and closed workloads. For open workloads, the workload generator sends requests accordng to a specfed request rate and

9 (a) One JBOSS Server transacton mx. For closed workloads, the workload follows a gven transton matrx to smulate multple concurrent users nteractons wth RUBS. The workload generator runs on a separate server node from any of the RUBS systems. 6.. Performance Predcton To valdate the correctness and accuracy of our model, we compare the response tmes predcted by our model and actual measurements wth dfferent workloads under dfferent confguratons. We use the workload generator to produce varable workloads wth fluctuatons n request rate and transacton mx. Applcaton data s obtaned from Apache and JBoss log. System utlzaton s collected every one mnute usng the SAR montor. The data set records two knds of data about RUBS, applcaton-level data such as transacton request rate of each transacton type, and system level resource utlzaton (e.g., CPU utlzaton). We then apply the regresson analyss descrbed n Secton 3 to generate the applcaton s resource profle. Gven any open or closed workload, we use the resource demand nformaton obtaned durng proflng as model nput parameters, and apply the performance model descrbed n secton 4 to derve the response tme. In the frst experment, we change the workload by varyng the (b) Two JBOSS Server Fgure 7. Performance wth dfferent number of users number of concurrent users generated by the workload generator. Each run lasts 20 mnutes, followng a 5 mnute warm-up perod. Fgure 7(a) shows the results of the average response tmes predcted by the model for 50 to 300 concurrent users on our RUBS testbed. Ths fgure also shows the actual results from ths testbed. From the fgure we can see that the performance model predcts the performance of RUBS accurately, as the maxmum relatve error s less than 5%. In the second experment, we reconfgure the RUBS tested wth two JBOSS servers and repeated the experment. The Web server evenly dstrbutes workload among these two JBOSS servers. The results are shown n Fgure 7(b). In ths case, our model has a maxmum relatve error of 20%. It s less accurate than sngle applcaton server confguraton due to the error ntroduced by mult-staton queueng model and the load balancer overhead. The above results show that the regresson-based proflng and the queung network model can model the performance behavor of RUBS applcaton. In the next set of experments, we evaluate the effectveness of our approach for dfferent mxes of browse and bd transactons. Frst, we defne three typcal closed workloads of 200 users wth dfferent transacton mxes: CW: browse domnant, CW2: balanced and CW3: bd domnant. For each workload, we use the profle and model to predct the average response tme, and then (a) Closed Workload Fgure 8. Performance wth dfferent workloads (b) Open Workload

10 We also compare the predcted CPU utlzaton wth the actual CPU utlzaton. A gudelne regardng resource utlzaton s to keep peak utlzatons of resources, such as CPU, below 70% [4]. In practce, enterprse system operators are typcally even more cautous than ths conservatve gudelne. Hence, we also put addtonal constrants of CPU utlzaton to be less than 60% n our evaluaton. Table 2. Decomposton results for closed workload Fgure 9. CPU utlzaton wth dfferent workloads compare the results wth the actual performance. The results are depcted n Fgure 8(a). The results show that our model can accurately predct the performance for dfferent closed workloads wth dfferent transacton mxes. Smlarly, we defne three typcal open workloads wth dfferent transacton mxes: OW, OW2 and OW3 and compare the accuracy of response tme predcted by our model for each workloads. The results depcted n Fgure 8(b) ndcate that our model can also work well wth open workloads. These results clearly demonstrate that our model can use the same proflng results (.e., model nput parameters) obtaned durng proflng to predct the performance of any unforeseen transacton mxes. We also conducted a smlar evaluaton of the RUBS confguraton wth 2 JBOSS servers. We obtaned smlar results, and thus do not nclude the fgures here Dervng Operatonal Polces One of the goals of our SLA decomposton s to derve healthy ranges of system metrcs and confgure lower level operatonal polces accordngly. In ths set of experments, we evaluate how well our model can be used to derve such low-level polces gven a workload or transacton mx. Resource consumpton at the Web ter and database ter are neglgble n our testbed, so we focus on the resource utlzaton of the applcaton server ter only. Gven a workload, we derve the CPU utlzaton as descrbed n Secton 4. Fgure 9 compares the CPU utlzaton predcted and measured for three dfferent transacton mxes. As shown n ths fgure, the maxmum relatve error s less than 0%. We also have smlar results for mult-server RUBS, whch we do not nclude here Decomposton Effectveness In ths secton, we evaluate the effectveness of our SLA decomposton. Gven any workload and SLOs, our decomposton module constructs a set of constrants and then solves the correspondng constrant satsfacton problem. The output of decomposton contans the number of servers requred at each ter to meet the response tme requrements, as well as the resource utlzaton of the confguraton. In the followng experments, gven any SLO n terms of workload and a response tme requrement, we generate the number of applcaton servers needed, and predct the average CPU utlzaton. We then confgure RUBS based on these derved settngs. We valdate our desgn by applyng the workload, and measurng the actual performance of RUBS and comparng the results wth the SLO. Input Output Measurement Workloads and SLOs um. of App. Servers Resp. Tme CPU Utl. Resp. Tme CPU Utl. User=00 Browse Intensve Response tme<5 sec 3.49 s 2.8% 3.03 s 24.5% User=00 Bddng Intensve Response tme< 5 sec 4.03 s 35.6% 4.36 s 33.2% Users=200 Browse Intensve Response tme < 5 sec. 4.77s 43.2% 4.67 s 47.8% User=200 Bddng Intensve Response tme< 5 sec s 37.4% 4.43 s 32.9% In these experments, we frst consder the hgh level SLOs n terms of the number of concurrent users, the transacton mx and the average response tme. Table 2 summarzes the nput and output of decomposton for four dfferent SLOs. The frst column shows the nput to our decomposton and the second column descrbes the output of decomposton such as the system desgn parameter (.e., number of JBOSS servers) and the healthy range of CPU utlzaton under the proposed confguraton. The measurement column shows the actual measurement of response tme and the CPU utlzaton of the system wth the desgn. As shown n the frst row, for the SLO of 00 users wth browsentensve transacton mx and response tme < 5 seconds, decomposton determnes that only one server s requred to ensure the SLO and the response tme and CPU utlzaton are 3.49 seconds and 2.8% respectvely. The actual measurements of response tme and CPU utlzaton are 3.03 seconds and 24.5%. Ths shows that the desgn can meet SLOs and the utlzaton predcton s close to real system measurement. The second SLO has 00 users but wth a dfferent transacton mx (.e., bddng ntensve). We can see from ths experment that the decomposton results are close to the actual measurements. The thrd nput nvolves 200 concurrent users and browsng ntensve transactons, the decomposton result shows that only one server s needed to meet the SLO. The fourth nput has 200 users wth bddng ntensve workload, whch s more resource demandng. The decomposton module determnes that 2 servers are requred to handle the workload and meet the response tme requrement. The actual performance shows that the desgn can meet the requrement and the predcton of response tme and CPU utlzaton are relatvely accurate. From the above results, we can see that our decomposton approach can be effectvely appled to desgn and montor such mult-ter applcatons wth dfferent SLOs. In order to further check the applcablty of our approach, we also apply the decomposton to SLOs nvolvng open workloads. We expermented wth four dfferent SLOs. In the experment, the workload s specfed n terms of request rate and transacton mx.

11 These results are summarzed n Table 3. The results show that our approach can also work well wth open workloads. Table 3. Decomposton results for open workload Input Output Measurement Workloads and SLOs um. of Resp. CPU Resp. CPU App. Servers Tme Utl. Tme Utl. 30 reqs/s Browse Intensve Response tme<5 sec 3.88 s 23.4% 3.67 s 25.% 30 reqs/s Bddng Intensve Response tme< 5 sec 4.53 s 37.3% 4.75 s 42.0% 40 reqs/s Browse Intensve Response tme < 5 sec. 40 reqs/s Bddng Intensve Response tme< 5 sec. 4.47s 40.% 4.8 s 44.5% s 32.3% 4.33 s 36.7% 6.2 Producton Applcaton We also evaluate the ablty of our decomposton approach to generate low-level resource utlzaton polces for a real busnesscrtcal enterprse applcaton. Ths servce conssts of roughly 20 servers and processes tens of mllons of applcaton-level transactons per day. The servce s CPU and network ntensve and ts performance s crucal to many other servces. In the evaluaton, we run the servce wth a 24 hour request trace from one of the actual servers. As descrbed n Secton 3, the profle captures the CPU and network demand for each transacton type. We then extract two typcal workloads wth dfferent transacton mxes: a lghtweght one and a heavyweght one. Gven these two workloads, we apply the decomposton approach to generate the CPU and network bounds and further create montorng polces based on the derved CPU and network bounds. The montorng polces are accordng to the utlzaton predcted by the decomposton model. We apply the workloads and measure the actual CPU and network utlzaton. The actual CPU resource utlzaton and the montorng polces are shown n Fgure 0. As shown n the fgures, the montorng polces accurately capture the healthy range of the applcaton for dfferent workloads. These polces can be contnuously used to assess how the system s performng and evaluate whether t wll volate any of the goals t was desgned for. For example, for the frst workload, t should warrant CPU Utlzaton to be around 20%. Ths metrc has to be montored to make sure that the hgher level SLA s met. For example, approprate actons can be taken when the threshold s volated. Ths can be defned n an operatonal polcy. In addton, such montorng polces wll also provde a mechansm usng whch we could predct or even avod future SLA volatons by provsonng the system accordngly n desgn or capacty plannng phase. 7. RELATED WORK Our prevous work proposes an SLA decomposton approach based on proflng and a queueng network model [28]. Although t shares some common features wth approach presented n ths paper, the basc assumpton and modelng technques are qute dfferent. Our early work focused on managng resource assgnment of vrtual machnes, and the proflng and modelng were relatvely smple. The approach presented n ths paper ams to develop a practcal and advanced model that can be appled to real mult-ter applcatons wth complex, dynamc, non-statonary workloads and varyng topologes. Compared to our earler work, the novel contrbutons and sgnfcant enhancements can be summarzed as follows. Frst, the new performance model s much more advanced and t models mult-ter applcatons n a much fner-gran manner. Through explctly modelng pertransacton resource demand, the new approach can handle any unforeseen workloads wth dfferent transacton mxes. Ths s very mportant mprovement snce workloads n real producton applcatons are typcally non-statonary [25]. Second, our new approach s non-ntrusve. The proflng n our early work requres sgnfcant efforts to nstrument the system and conduct controlled benchmarkng n order to collect montorng data, whle our new approach can perform proflng from readly avalable montorng data. Thrd, by defnng the workload as transacton mx and ntroducng open queueng and closed queung models, our approach can handle both open and closed workloads n a consstent manner. Fourth, our model can drectly derve the healthy range of low level system metrcs and further develop montorng polces from the profles. We also ncreased the (a) CPU Utlzaton Fgure 0. Montorng Polces (b) etwork Usage