A Scalable Application Placement Controller for Enterprise Data Centers

Transcription

1 W WWW 7 / Track: Perforance and Scalability A Scalable Application Placeent Controller for Enterprise Data Centers Chunqiang Tang, Malgorzata Steinder, Michael Spreitzer, and Giovanni Pacifici IBM T.J. Watson Research Center 9 Skyline Drive, Hawthorne, NY 532, USA {ctang, steinder, speitz, giovanni}@us.ib.co ABSTRACT Given a set of achines and a set of Web applications with dynaically changing deands, an online application placeent controller decides how any instances to run for each application and where to put the, while observing all kinds of resource constraints. This NP hard proble has real usage in coercial iddleware products. Existing approxiation algoriths for this proble can scale to at ost a few hundred achines, and ay produce placeent solutions that are far fro optial when syste resources are tight. In this paper, we propose a algorith that can produce within seconds high-quality solutions for hard placeent probles with thousands of achines and thousands of applications. This scalability is crucial for dynaic resource provisioning in large-scale enterprise data centers. Our algorith allows ultiple applications to share a single achine, and strives to axiize the total satisfied application deand, to iniize the nuber of application starts and stops, and to balance the load across achines. Copared with existing state-of-the-art algoriths, for systes with achines or less, our algorith is up to 34 ties faster, reduces application starts and stops by up to 97%, and produces placeent solutions that satisfy up to 25% ore application deands. Our algorith has been ipleented and adopted in a leading coercial iddleware product for anaging the perforance of Web applications. Categories and Subject Descriptors K.6.4 [Coputing Milieux]: Manageent of Coputing and Inforation Systes Syste Manageent General Ters Algorith, Manageent, Perforance Keywords Application Placeent, Perforance Manageent. INTRODUCTION With the rapid growth of the Internet, any organizations increasingly rely on Web applications to deliver critical services to their custoers and partners. Enterprise data centers ay run thousands of achines to host a large nuber of Web applications that are resource deanding and Copyright is held by the International World Wide Web Conference Coittee (IW3C2). Distribution of these papers is liited to classroo use, and personal use by others. WWW 7, May 8 2, 7, Banff, Alberta, Canada. ACM /7/5. process client requests at a high rate. Previous studies have shown that the Web request rate is bursty and can fluctuate draatically in a short period of tie [4]. Therefore, it is not cost-effective to over provision data centers to handle the potential peak deands of all the applications. To utilize syste resources ore effectively, odern Web applications typically run on top of a iddleware syste and rely on it to dynaically allocate resources to eet their perforance goals. Soe iddleware systes use clustering technology to iprove scalability and availability, by integrating ultiple instances of an application, and presenting the to the users as a single virtual application. Figure is an exaple of clustered Web applications. The syste consists of one front-end request router, three backend achines (A, B, and C), and three applications (x, y, and z). The applications, for exaple, can be catalog search, order processing, and account anageent for an online shopping site. The request router receives external requests and forwards the to the application instances. To eet the perforance goals of the applications, the request router ay ipleent functions such as adission control, flow control, and load balancing. These functions decide how to dynaically allocate resources to the running application instances, which are well-studied topics in the literature [4, 4]. This paper studies an equally iportant proble that receives relatively less attention in the past: Given a set of achines with constrained resources and a set of Web applications with dynaically changing deands, how any instances to run for each application and where to put the? We call this proble dynaic application placeent. We assue that not every achine can run all the applications at the sae tie due to liited resources such as eory. Application placeent is orthogonal to adission control, flow control, and load balancing, and the quality of a place- a p p x a c h i n e A a p p y Request Router a p p x eb Requests a c h i n e B a p p y a p p x a c h i n e C a p p z Figure : An exaple of clustered Web applications. 33

2 M WWW 7 / Track: Perforance and Scalability placeent sensor app d eand esti ator conf i g D B placeent ex ecu tor C u rrent placeent atri x : I C P U d eand v ector of apps : M eory d eand v ector of apps: C P U capaci ty v ector of ach i nes: eory capaci ty v ector of ach i nes: R estri cti on atri x : R inpu ts application placeent contr oller ou tpu ts N ew placeent atri x : I L oad d i stri b u ti on atri x : L Figure 2: Control loop for application placeent. ent solution can have profound ipact on the perforance of the entire syste. In Figure, suppose the request rate for application z suddenly surges. Application z ay not eet the deands even if all the resources of achine C are allocated to application z. A sart iddleware syste then ay react by stopping application x on achines A and B, and using the freed resources (e.g., eory) to start an instance of application z on both A and B. The application placeent proble can be forulated as a variant of the Class Constrained Multiple-Knapsack Proble [2, 3]. Under ultiple resource constraints (e.g., CPU and eory) and application constraints (e.g., the need for special hardware or software), a placeent algorith strives to produce placeent solutions that optiize ultiple objectives: () axiizing the total satisfied application deand, (2) iniizing the total nuber of application starts and stops as they disturb the running syste, and (3) balancing the load across achines. The placeent proble is NP hard. Existing approxiation algoriths [6, 8] can scale to at ost a few hundred achines, and ay produce placeent solutions that are far fro optial when syste resources are tight. In this paper, we propose a approxiation algorith that significantly and consistently outperfors existing state-of-the-art algoriths in ters of both solution quality and scalability. Our algorith has been ipleented and adopted in a leading coercial iddleware product []. The reainder of the paper is organized as follows. Sections 2, 3, and 4 forulate the application placeent proble, describe our algorith, and present its perforance, respectively. Section 5 discusses related work, and Section 6 concludes the paper. 2. PROBLEM FORMULATION Figure 2 is a siplified diagra of the control loop for application placeent. For brevity, we siply refer to application placeent as placeent in the rest of this paper. The inputs to the placeent controller include the current placeent of applications on achines, the resource capacity of each achine, the projected resource deand of each application, and the restrictions that specify whether a given application can run on a given achine, e.g., soe application ay require achines with special hardware or software. Taking these inputs collected by the auxiliary coponents, the placeent controller coputes a placeent solution that optiizes certain objective functions, and then passes the solution to the placeent executor to start and stop application instances accordingly. Periodically every T inutes, the placeent controller produces a placeent solution based on the current inputs (e.g., T =5 inutes). Estiating application deands is a non-trivial task. We use online profiling and data regression to dynaically estiate the average CPU cycles needed to process one Web request of a given application []. The product of the estiated CPU cycles per request and the projected request rate gives the CPU cycles needed by the application per second. We use a peer-to-peer infrastructure [6] to gather perforance etrics fro a large nuber of achines in a scalable and reliable fashion. In the past, we have developed a iddleware syste [,,, 9] that includes a superset of the control loop in Figure 2. In this paper, we focus on the design and evaluation of the placeent controller. The rest of this section presents the foral forulation of the placeent proble. We first discuss the syste resources and application deands considered in the placeent proble. A running application instance s consuption of CPU cycles and IO bandwidth depends on the request rate. As for eory, our syste periodically and conservatively estiates the upper liit of an application s near-ter eory usage, and assues that this upper liit does not change until the next estiate update point, because of several practical reasons. First, a significant aount of eory is consued by an application instance even if it receives no requests. Second, eory consuption is often related to prior application usage rather than its current load due to data caching and delayed garbage collection. Third, because an accurate projection of eory usage is difficult and any applications cannot run when the syste is out of eory, it is ore reasonable to use the conservatively estiated upper liit for eory consuption. Aong any resources, we choose CPU and eory as the representative ones to be considered by the placeent controller. For brevity, the description of our algorith only considers CPU and eory, but it can deal with other types of resources as well. For exaple, if the syste is networkbounded, we can use network bandwidth as the bottleneck resource whose consuption depends on the request rate. This introduces no changes to our algorith. Next, we present the forulation of the placeent proble. Table lists the sybols used in our discussion (see Figure 2 for their roles). The inputs to the placeent controller are the current placeent atrix I, the placeent restriction atrix R, the CPU/eory capacity of each achine (Ω n and Γ n), and the CPU/eory deand of each application (ω and γ ). The outputs of placeent controller are the updated placeent atrix I and the load distribution atrix L. The placeent controller strives to find a placeent solution that axiizes the total satisfied application deand. In addition, it also tries to iniize the total nuber of application starts and stops, because placeent changes disturb the running syste and waste CPU cycles. In practice, any J2EE applications take a few inutes to start or stop, and take soe additional tie to war up their data cache. The last optiization goal is to balance the load across achines. Ideally, the utilization of individual achines should 332

3 WWW 7 / Track: Perforance and Scalability stay close to the utilization ρ of the entire syste. P ρ = P M n N L,n () Pn N Ωn As we are dealing with ultiple optiization objectives, we prioritize the in the foral proble stateent below. Let I denote the placeent atrix, and I denote the placeent atrix. X X (i) axiize L,n (2) such that (ii) (iii) iniize M n N X X I,n I,n (3) M n N P X iniize M L,n ρ Ω (4) n N n M, n N I,n = or I,n = (5) M, n N R,n = I,n = (6) M, n N I,n = L,n = (7) M, n N L,n (8) X n N γ I,n Γ n (9) n N M M X L,n Ω n () M X L,n ω () n N This proble is a variant of the Class Constrained Multiple- Knapsack proble [2, 3]. It differs fro the prior forulation ainly in that it also iniizes the nuber of placeent starts and stops. This proble is NP hard. We will present an online approxiation algorith for solving it. 3. THE PLACEMENT ALGORITHM This section describes our placeent algorith. Before presenting its details, we first give a high-level description of the algorith, a definition of ters, and the key ideas behind the algorith. Our algorith repeatedly and increentally optiizes the placeent solution in ultiple rounds. In each round, it first coputes the axiu total application deand that can be satisfied by the current placeent solution. The algorith quits if all the application deands are satisfied. Otherwise, it shifts load across achines (without placeent changes), and then considers stopping unproductive application instances and starting ore useful ones in order to increase the total satisfied application deand. The loadshifting step before the placeent-changing step is critical as it draatically siplifies subsequent placeent changes. Note that, in the algorith description, placeent change, application start/stop, and load shifting are all hypothetical. The real placeent changes are executed after the placeent algorith terinates. 3. Definition of Ters A achine is fully utilized if its residual (i.e., unused) CPU capacity is zero (Ω n = ); otherwise, it is underutilized. An application instance is fully utilized if it runs on a fully utilized achine. An instance of application running on an underutilized achine n is copletely idle if it has no load (L,n=); otherwise, it is underutilized. The load of an underutilized instance of application can be increased N The set of achines. n One achine in the set N. M The set of applications. One application in the set M. R The placeent restriction atrix. R,n = if application can run on achine n; R,n = otherwise. I The placeent atrix. I,n = if application is running on achine n; I,n = otherwise. L The load distribution atrix. L,n is the CPU cycles per second allocated on achine n for application. L is an output of the placeent algorith; it is not easured fro the running syste. Γ n The eory capacity of achine n. Ω n The CPU capacity of achine n. γ The eory deand of application, i.e., the eory needed to run one instance of application. ω The CPU deand of application, i.e., the total CPU cycles per second needed for application throughout the entire syste. ω The residual CPU deand of application, i.e., the deand not satisfied by the load distribution atrix L: ω = ω P n N L,n. Ω n The residual CPU capacity of achine n, i.e., the CPU capacity not consued by the applications running on achine n: Ω n = Ω n P M L,n. Γ n The residual eory capacity of achine n, i.e., the eory not consued by the busy applications (L,n>) running on achine n: Γ n = Γ n P :L γ.,n> Table : Sybols used in the placeent algorith. if application has a positive residual (i.e., unsatisfied) CPU deand (ω > ). The CPU-eory ratio of a achine n is defined as its CPU capacity divided by its eory capacity, i.e., Ω n/γ n. Intuitively, it is harder to fully utilize the CPU of achines with a high CPU-eory ratio. The load-eory ratio of an instance of application running on achine n is defined as the CPU load of this instance divided by its eory consuption, i.e., L,n/γ. Intuitively, application instances with a higher load-eory ratio are ore productive. 3.2 Key Ideas in Load Shifting Figure 4 is the high-level pseudo code of our algorith. The details will be explained later. The core of the place() function is a loop that increentally optiizes the placeent solution. Inside the loop, it first solves the ax-flow proble [2] in Figure 3 to copute the axiu total deand ˆω that can be satisfied by the current placeent atrix I. Aong any possible load distribution atrices L that can eet this axiu deand ˆω, we eploy several load-shifting heuristics to find the one that akes later placeent changes easier. We classify the running instances of an application into three categories: idle, underutilized, and fully utilized. The idle instances are preferred candidates to be shut down. We opt for leaving the fully utilized instances intact as they already ake good contributions. 333

4 WWW 7 / Track: Perforance and Scalability Through proper load shifting, we can ensure that every application has at ost one underutilized instance in the entire syste. Reducing the nuber of underutilized instances siplifies the placeent proble, because the strategy to handle idle instances and fully utilized instances are straightforward. We strive to co-locate residual eory and residual CPU on the sae achines so that these resources can be used to start application instances. For exaple, if one achine has only residual CPU while another achine has only residual eory, neither of the can accept applications. We strive to ake idle application instances appear on achines with relatively ore residual eory. By shutting down the idle instances, ore eory will becoe available for hosting applications that require a large aount of eory. 3.3 Key Ideas in Placeent Changing The load-shifting subroutine in Figure 4 prepares the load distribution in a way that akes later placeent changes easier. The placeent-changing subroutine further eploys several heuristics to increase the total satisfied application deand, to reduce placeent changes, and to reduce coputation tie. The algorith walks through the underutilized achines sequentially and akes placeent changes to the one by one in an isolated fashion. When working on a achine n, the algorith is only concerned with the sate of achine n and the residual application deands. This isolation drastically reduces the coputation tie. The isolation of achines, however, ay lead to inferior placeent solutions. We address this proble by alternately executing the load-shifting subroutine and the placeent-changing subroutine for ultiple rounds. As a result, the residual application deands released fro the application instances stopped in the previous round now have the chance of being allocated to other achines in the later rounds. When sequentially walking through the underutilized achines, the algorith first considers achines with a relatively high CPU-eory ratio (see the definition in Section 3.). As it is harder to fully utilize the CPU of these achines, we prefer to process the first when we still have abundant choices. When choosing applications to run on a achine, the algorith tries to find a cobination of applications that lead to the highest CPU utilization of this achine. It prefers to stop unproductive running application instances with a relatively low load-eory ratio to accoodate application instances. To reduce placeent changes, the algorith does not allow stopping application instances that already deliver a sufficiently high load. We refer to these instances as pinned instances. The intuition is that, even if we stop these instances on their current hosting achines, it is likely that we will start instances of the sae applications on other achines. Our algorith dynaically coputes the pinning thresh for each application. s o u r c e w z x y app w app x app y app z ac h i n e A ac h i n e B ac h i n e C B (r B A (r A C (r C Figure 3: This figure shows two network flow probles. () When the link costs (i.e., r A =, r B =, and r C =2) are not used, this figure is an exaple of the axflow proble whose solution gives the axiu total deand that can be satisfied by the current placeent atrix I. (2) When the link costs are used, it is an exaple of the in-cost ax-flow proble solved by the load-shifting subroutine to copute a load distribution that akes later placeent changes easier. Below, we present in detail the load-shifting subroutine, the placeent-changing subroutine, and the full placeent algorith that utilizes these two subroutines. 3.4 The Load-Shifting Subroutine Given the current application deands, the placeent algorith solves a ax-flow proble [2] to derive the axiu total deand that can be satisfied by the current placeent atrix I. Figure 3 is an exaple of this axflow proble, in which we consider four applications (w, x, y, and z) and three achines (A, B, and C). Each application is represented as a node in the graph. Each achine is also represented as a node. In addition, there are a source node and a sink node. The source node has an outgoing link to each application, and the capacity of this link is the CPU deand of the application (ω ). Each achine n has an outgoing link to the sink node, and the capacity of this link is the CPU capacity of the achine (Ω n). The last set of links are between the applications and the achines that currently run those applications. The capacity of these links is unliited. In Figure 3, application x currently runs on achines A and B. Therefore, x has two outgoing links: x A and x B. When load distribution is forulated as this ax-flow proble, the axiu volue of flows going fro the source node to the sink node is the axiu total deand ˆω that can be satisfied by the current placeent atrix I. (Recall that I,n = if application is running on achine n. See Table for the notations.) If all application deands are satisfied, no placeent changes are needed. Otherwise, we ake placeent changes in order to satisfy ore application deands. Before doing so, we first adjust the load distribution atrix L produced by solving the ax-flow proble in Figure 3. (Recall that L,n is the CPU cycles per second allocated on achine n for application.) The goal of load shifting is to achieve the effects described in Section 3.2, e.g., co-locating residual CPU and residual eory on the sae set of achines, and ensuring that each application has at ost one underutilized instance in the entire syste. The task of load shifting is accoplished by solving the in-cost ax-flow proble [2] in Figure 3. We sort all the achines in increasing order of residual eory capacity Γ n, and associate each achine n with a rank r n that reflects its position in this sorted list. The achine with rank has the least aount of residual eory. In Figure 3, the link s i n k 334

5 WWW 7 / Track: Perforance and Scalability function place () { for (i = ; i < K; i++) { // K= by default. calc ax deand satisfied by current placeent (); if (all deands satisfied) break out of the loop; load shifting (); // No placeent changes here. placeent changing (pin app=false); placeent changing (pin app=true); choose the better one as the solution; // Pin or not. if (no iproveent) break out of the loop; } balance load across achines (); } function placeent changing (boolean pin app) { // outerost loop---- // Change the placeent on one achine at a tie. for (all underutilized achines n) { if (pin app==true) identify pinned app instances(); // Suppose achine n currently runs c not-pinned // app instances (M, M2,..., Mc) sorted in // increasing order of load-eory ratio. }}} // interediate loop---- for (j=; j < c; j++) { if (j > ) stop j apps on achine n (M,M2,...,Mj); // innerost loop---- // Find apps to consue n s residual resources // that becoe available after stopping the j apps. for (all apps x with a positive residual deand) { if (app x fits on achine n) start x on n (); } if (is the best solution for achine n so far) record it(); Figure 4: High-level pseudo code of our algorith. between a achine n and the sink node is associated with the cost r n. The cost of all the other links is zero, which is not shown in the figure for brevity. In this exaple, achine C has ore residual eory than achine A, and achine A has ore residual eory that achine B. Therefore, the links between the achines and the sink node have costs r B =, r A =, and r C = 2, respectively. The load distribution atrix L produced by solving the in-cost ax-flow proble in Figure 3 possesses the following good properties that ake later placeent changes easier: () An application has at ost one underutilized instance in the entire syste. (2) Residual eory and residual CPU are likely to co-locate on the sae set of achines. (3) The idle application instances appear on the achines with relatively ore residual eory. Theore. In the load distribution atrix L produced by solving the in-cost ax-flow proble in Figure 3, each application has at ost one underutilized instance in the entire syste. Proof: We prove this by contradiction. Suppose there are two underutilized instances of the sae application running on two underutilized achines A and B, respectively. Without loss of generality, we assue that achine A has less residual eory than achine B, i.e., r A < r B. Because achine A still has residual CPU capacity and the cost of using achine A is lower than that of using achine B (r A < r B), the in-cost ax-flow algorith can further reduce the total cost of the axiu flow by oving load fro achine B to achine A, which contradicts with the fact that the current solution given by the in-cost ax-flow algorith already has the lowest cost. Theore 2. In the load distribution atrix L produced by solving the in-cost ax-flow proble in Figure 3, if application has one underutilized instance running on achine n, then () application s idle instances ust run on achines whose residual eory is larger than or equal to that of achine n; and (2) application s fully utilized instances ust run on achines whose residual eory is saller than or equal to that of achine n. Proof: The proof is siilar to that for Theore. 3.5 The Placeent-Changing Subroutine The placeent-changing subroutine takes as input the current placeent atrix I, the load distribution atrix L generated by the load-shifting subroutine, and the residual application deands not satisfied by L. It tries to increase the total satisfied application deand by aking placeent changes, for instance, stopping unproductive application instances and starting useful ones. The ain structure of the placeent-changing subroutine consists of three nested loops (see Figure 4). The outerost loop iterates over the achines and asks the interediate loop to generate a placeent solution for one achine n at a tie. Suppose achine n currently runs c not-pinned application instances (M, M 2,, M c) sorted in increasing order of load-eory ratio (see the definition in Section 3.). The interediate loop iterates over a variable j ( j c). In iteration j, it stops on achine n the j applications (M, M 2,, M j) while keeping the other running applications intact, and then asks the innerost loop to find appropriate applications to consue achine n s residual resources. The innerost loop walks through the residual applications, and identifies those that can fit on achine n. As the interediate loop varies the nuber of stopped applications fro to c, it collects c + different placeent solutions for achine n, aong which it picks the best one as the final solution. Below, we describe the three nested loops in detail. The Outerost Loop. Before entering the outerost loop, the algorith first coputes the residual CPU deand of each application. We refer to the applications with a positive residual CPU deand (i.e., ω > ) as residual applications. The algorith inserts all the residual applications into a right-threaded AVL tree called residual app tree. The applications in the tree are sorted in decreasing order of residual deand. As the algorith progresses, the residual deand of applications ay change, and the tree is updated accordingly. The algorith also keeps track of the iniu eory requireent γ in of applications in the tree, γ in = in γ, (2) residual app tree where γ is the eory needed to run one instance of application. The algorith uses γ in to speed up the coputation in the innerost loop. If a achine n s residual eory Γ n is saller than γ in, the algorith can iediately infer that this achine cannot accept any applications in the residual app tree. 335

6 WWW 7 / Track: Perforance and Scalability The algorith excludes fully utilized achines fro the consideration of placeent changes, and sorts the underutilized achines in decreasing order of CPU-eory ratio. Starting fro the achine with the highest CPU-eory ratio, it enuerates each underutilized achine, and asks the interediate loop to copute a placeent solution for each achine. Because it is harder to fully utilize the CPU of achines with a high CPU-eory ratio, we prefer to process the first when we still have abundant choices. The Interediate Loop. Taking as input the residual app tree and a achine n given by the outerost loop, the interediate loop coputes a placeent solution for achine n. Suppose achine n currently runs c notpinned application instances. (Application instance pinning will be discussed later.) We can stop a subset of the c applications, and use the residual resources to run other applications. In total, there are 2 c cases to consider. We use a heuristic to reduce this nuber to c +. Intuitively, we prefer to stop the less productive application instances, i.e., those with a low load-eory ratio (L,n/γ ). The algorith sorts the not-pinned application instances on achine n in increasing order of load-eory ratio. Let (M, M 2,, M c) denote this sorted list. The interediate loop iterates over a variable j ( j c). In iteration j, it stops on achine n the j applications (M, M 2,, M j) while keeping the other running applications intact, and then asks the innerost loop to find appropriate applications to consue achine n s residual resources that becoe available after stopping the j applications. As the interediate loop varies the nuber of stopped applications fro to c, it collects c + placeent solutions, aong which it picks as the final solution the one that leads to the highest CPU utilization of achine n. The Innerost Loop. The interediate loop changes the nuber of applications to stop. The innerost loop uses achine n s residual resources to run soe residual applications. Recall that the residual app tree is sorted in decreasing order of residual CPU deand. The innerost loop iterates over the residual applications, starting fro the one with the largest aount of residual deand. When an application is under consideration, the algorith checks two conditions: () whether the restriction atrix R allows application to run on achine n, and (2) whether achine n has sufficient residual eory to host application (i.e., γ Γ n). If both conditions are satisfied, it places application on achine n, and assigns as uch load as possible to this instance until either achine n s CPU is fully utilized or application has no residual deand. After this allocation, s residual deand changes, and the residual app tree is updated accordingly. The innerost loop iterates over the residual applications until either () all the residual applications have been considered once; or (2) achine n s CPU becoes fully utilized; or (3) achine n s residual eory is insufficient to host any residual application (i.e., Γ n < γ in, see Equation 2). 3.6 The Full Placeent Algorith The full placeent algorith is outlined in Figure 4. It increentally optiizes the placeent solution in ultiple rounds. In each round, it first invokes the load-shifting subroutine and then invokes the placeent-changing subroutine. It repeats for up to K rounds, but quits earlier it sees no iproveent in the total satisfied application deand after one round of execution. The last step of the algorith balances the load across achines. We reuse the load-balancing coponent fro an exiting algorith [6], but oit its detail here. Intuitively, it oves the application instances between achines to balance the load, while keeping the total satisfied deand and the nuber of placeent changes the sae. The placeent algorith deals with ultiple optiization objectives. In addition to axiizing the total satisfied deand, it also strives to iniize placeent changes, because they disturb the running syste and waste CPU cycles. Our heuristic for reducing unnecessary placeent changes is not to stop application instances whose load (in the load distribution atrix L) is above certain thresh. We refer to the as pinned instances. The intuition is that, even if we stop these productive instances on their current hosting achines, it is likely that we will start instances of the sae applications on other achines. Each application has its own pinning thresh ω pin. The value of this thresh is crucial. If it is too low, the algorith ay introduce any unnecessary placeent changes. If it is too high, the total satisfied deand ay be low due to insufficient placeent changes. The algorith dynaically coputes the pinning thresh for each application using inforation gathered in a dry-run invocation to the placeent-changing subroutine. The dry run pins no application instances. After the dry run, the algorith akes a second invocation to the placeent-changing subroutine, and requires pinning the application instances whose load is higher than or equal to the pinning thresh. The dry run and the second invocation use exactly the sae inputs: the atrices I and L produced by the load-shifting subroutine. Between the two placeent solutions produced by the dry run and the second invocation, the algorith picks as the final solution the one that has a higher total satisfied deand. If the total satisfied deands are equal, it picks the one that has less placeent changes. Next, we describe how to copute the pinning thresh of the corresponding application, i.e., L,n ω pin using inforation gathered in the dry run. Intuitively, if the dry run starts a application instance, then we should not stop any instance of the sae application whose load is higher than or equal to that of the instance. This is because the instance s load is considered sufficiently high by the dry run so that it is even worthwhile to start a ω pin instance. Let ω denote the iniu load assigned to a instance of application started in the dry run. ω = in {L,n after the dry run} I,n { instances of app started in the dry run} (3) Here I,n represents a instance of application started on achine n in the dry run. L,n is the load of this instance. In addition, the pinning thresh also depends the largest residual application deand ωax not satisfied in the dry run. ωax = ax ω (4) {residual app tree after the dry run} Here ω is the residual deand of application after the dry run. We should not stop the application instances whose load is higher than or equal to ω ax. If we stop these instances, they would iediately becoe the applications that we try to find a place to run. The pinning thresh for application is coputed as follows. ω pin = ax (, in (ω ax, ω )) (5) 336

7 WWW 7 / Track: Perforance and Scalability Because we do not want to pin copletely idle application instances, Equation 5 stipulates that the pinning thresh should be at least one CPU cycle per second. ω pin 3.7 Coplexity and Practical Issues The coputation tie of our placeent algorith is doinated by the tie spent on solving the ax-flow proble and the in-cost ax-flow proble in Figure 3. One efficient algorith for solving the ax-flow proble is the highest-label preflow-push algorith [2], whose coplexity is O(s 2 t), where s is the nuber of nodes in the graph, and t is the nuber of edges in the graph. One efficient algorith for solving the in-cost flow proble is the enhanced capacity scaling algorith [2], whose coplexity is O((s log t)(s + t log t)). Let N denote the nuber achines. Due to various resource constraints, the nuber of applications that a achine can run is bounded by a constant. Therefore, in the network flow graph, both the nuber s of nodes and the nuber t of edges are bounded by O(N). The total nuber of application instances in the entire syste is also bounded by O(N). Under these assuptions, the coplexity of our placeent algorith is O(N 2.5 ). In contrast, under the sae assuptions, the coplexity of the state-of-the-art placeent algorith proposed by Kibrel et al. [6, 8] is O(N 3.5 ). This difference in coplexity is the reason why our algorith can do online coputation for systes with thousands of achines, while their algorith can scale to at ost a few hundred achines (see the results in Section 4). For the sake of brevity, our forulation of the placeent proble and our algorith description oit several practical issues. For instance, an adinistrator ay ipose restrictions on the iniu/axiu nuber of instances of a given application allowed in the entire syste. It is also possible that ultiple instances of the sae application need to run on a single achine because, for exaple, one instance of the application cannot utilize all the CPU power of the achine due to internal bottlenecks in the application. Moreover, the optiization objective can be axiizing certain utility function instead of axiizing the total satisfied application deand. Finally, the actual start and stop of application instances should be carefully coordinated to ipleent a fast transition and avoid stopping all instances of an application at the sae tie. One version of our algorith adopted in a leading coercial iddleware product [] addresses these practical issues, but we oit a detailed discussion here due to space liitations. 4. EXPERIMENTAL RESULTS This section studies the perforance of our placeent algorith, and copares it with the state-of-the-art algorith [6, 8] proposed by Kibrel et al. We use this algorith as the baseline because the evaluation [8] shows that it outperfors two variants of another state-of-the-art algorith [2, 3]. For brevity, we siply refer to our algorith and the algorith proposed by Kibrel et al. as the and algoriths, respectively. Because the two algoriths use the sae technique for load balancing, we oit its results here, and refer interested readers to [6]. Both algoriths have been ipleented in a leading coercial iddleware product []. In this paper, we evaluate only the placeent controller coponent (as opposed to the entire iddleware), by feeding a wide variety of workloads directly to the placeent algoriths. Soe of the workloads are representative of real-world traffic (e.g., application deands that follow a power-law distribution), while others are extree configurations for stress test. In all the experients, we assue no placeent restriction for applications, i.e., M n N R,n =. The placeent controller works in cycles. At the beginning of a cycle, the placeent algorith is given a set of achines, a set of applications, the current deands of the applications, and the placeent atrix I left fro the previous cycle. The placeent algorith then produces a placeent atrix I and a load distribution atrix L, which are used for perforance evaluation. The evaluation etrics include the execution tie, the nuber of placeent changes (i.e., application starts/stops), and the deand satisfaction (i.e., the fraction of the total application deands satisfied by the placeent solution: P P M n N P M L,n ). ω In the experients, the configuration of achines is uniforly distributed over the set {GB:GHz, 2GB:.6GHz, 3GB:2.4GHz, 4GB:3GHz}, where the first nuber is eory capacity and the second nuber is CPU speed. The eory requireent of applications is uniforly distributed over the set {.4GB,.8GB,.2GB,.6GB}. A syste configuration includes a fixed set of achines and applications. All the reported data are averaged over the results on randoly generated syste configurations. For each configuration, the placeent algorith executes for cycles (including an initial placeent) under changing application deands. Hence, each reported data point is averaged over, placeent results (excluding the initial placeent). 4. Proble Hardness We copare the algoriths while varying the size of the placeent proble (i.e., the nuber of achines and applications) and the hardness of the proble. The hardness is defined fro four diensions: CPU load, eory load, application CPU deand distribution, and deand variability. CPU Load Factor L cpu. It is defined as the ratio between the P total CPU deand and the total CPU capacity, L cpu = P M ω, where n N Ωn ω is the CPU deand for application, and Ω n is the CPU capacity of achine n. Meory Load Factor L e. Let γ denote the average eory requireent of applications, and Γ denote the average eory capacity of achines. The average nuber of application instances that can be hosted on N achines is Γ N. The eory load factor is defined as γ L e = M/ Γ N = Mγ, where M is the nuber of applications. Note that L e and the proble is ost γ NΓ difficult when L e =. In the experients, we vary the CPU load factor L cpu, the eory load factor L e, and the nuber N of achines. The nuber M of applications is configured according to N and L e: M = Γ NLe = 2.5NLe. γ Application Deand Distribution. Once the CPU load factor L cpu and the total CPU capacity Ω = P n N Ωn are deterined, the total application CPU deand is set to Ω L cpu. We experient with two different ways of partitioning this total deand aong applications. With the unifor distribution, each application s initial deand is generated uniforly at rando fro the range [, ]. With the powerlaw distribution, application s initial deand is set to j α, where α = 2.6 and j is application s rank in a rando perutation of all the applications. For both unifor and power-law distributions, the deand of each application is noralized proportionally to the total application deand. 337

8 WWW 7 / Track: Perforance and Scalability Application Deand Variability. Given a syste configuration, the placeent algorith executes for cycles. The application deands in the first cycle follow either a unifor distribution or a power-law distribution. Starting fro the second cycle, the application deands change fro cycle to cycle. The placeent proble is harder to solve if this change is drastic. We experient with four different deand changing patterns. With the vary-all-apps pattern, each application s deand changes randoly and independently within a ±% range of its initial deand. With the vary-two-apps pattern, we keep the deands of all the applications constant except for the two applications with the largest deands. The su of these two applications deands is kept constant, but the allocation between the randoly changes % fro cycle to cycle. With the resetall-apps pattern, the deands in two consecutive cycles are independent of each other. This unrealistic pattern represents the ost extree deand change. With the add-apps pattern, the placeent algorith executes for M placeent cycles, where M is the nuber of applications. Starting with an epty, idle syste, the deand for one application is introduced into the syste in every cycle. Below, we concisely represent the hardness of a placeent proble as (L cpu, L e, deand-distribution, deandvariability ), e.g., (L cpu=.9, L e=.4, power-law-apps, vary-all-apps). 4.2 Perforance Results To choose the best algorith for the coercial product [], we copared ore than a dozen different variants of the placeent algoriths (including soe variants not described in this paper), and generated thousands of perforance graphs. Due to space liitations, we present in this paper only soe representative results. Overall, when the placeent proble is easy to solve (i.e., the syste has abundant resources), both algoriths can satisfy alost all the application deands. However, when the placeent proble is hard, the algorith significantly and consistently outperfors the algorith. Figure 5 shows the execution tie of the two algoriths for the setting (L cpu=.99, L e=, unifor-apps, resetall-apps). For hard probles, the execution tie of the algorith is alost negligible copared with that of the algorith. As an online controller, the algorith can only scale to at ost a few hundred achines and applications, while the algorith can scale to thousands of achines and applications. Figure 6 reports results on the scalability of the algorith. We vary the nuber of achines fro to 7, and the nuber of applications fro 2 to 7,. The algorith takes less than seconds to solve the difficult 7,-achine, 7,-application placeent proble under the tight resource constraints and the extree deand changes fro cycle to cycle. This execution tie is easured on a.8ghz Pentiu IV achine. The deand satisfaction in Figure 6 stays around.946 as the syste size increases, showing that the algorith can produce high-quality solutions regardless of the proble size. The nuber of placeent changes is high because this resetall-apps configuration for stress test unrealistically changes application deands between cycles in an extree anner. The experient in Figure 7 introduces the deand for one application into a -achine syste in every placeent cycle. This figure reports the nuber of placeent changes occurred when adding the i-th application rather than the aggregated nuber of placeent changes occurred before adding the (i+)-th application. Because the resources are not very tight (L cpu=.9 and L e=.4), both algoriths can satisfy all the deands, but the algorith introduces a uch larger nuber of placeent changes. For exaple, to handle the added deand for the last application, the algorith akes 5.3 placeent changes on average, while the algorith akes only.6 placeent changes on average. In a real syste, this difference can have draatic ipact on the whole syste perforance. The experients in Figures 8, 9, and use different cobinations of CPU load factor (H cpu=.99 or.6), eory load factor (H e= or.6), deand distribution (unifor or power-law), and deand variability (vary-two-apps, varyall-apps, or reset-all-apps). Under all these settings, the algorith consistently outperfors the algorith: it iproves deand satisfaction by up to 25%, and reduces placeent changes by up to 9%. The experient in Figure varies the eory load factor L e fro. to for a -achine syste, while fixing the CPU load factor L cpu=.9. As the hardness of the proble increases, the deand satisfaction of the algorith drops faster than that of the algorith. More iportantly, the nuber of placeent changes in the algorith increases draatically. The experient in Figure 2 varies the CPU load factor L cpu fro. to for a -achine syste, while fixing the eory load factor L e=.4. The algorith and the algorith have siilar perforance when the CPU load factor is below.8. However, when the CPU load factor is between.8 and.9, the nuber of placeent changes in the algorith increases alost exponentially. The situation could get even worse as the CPU load factor further approaches. To deal with this pathological case, the iproved version [6] of the algorith (the version used in the coparison) added a 9% load reduction heuristic whenever the CPU load factor is above.9, the algorith first (artificially) reduces it to.9 by scaling all the application deands proportionally, and then executes the core of the algorith on the reduced deands. This heuristic helps the algorith to reduce placeent changes, but it also decreases the deand satisfaction (see the dip in Figure 2). In contrast, the algorith achieves a better perforance even without using such hard-coded rules to handle the corner cases. In suary, the algorith significantly and consistently outperfors the algorith in all three aspects: execution tie, deand satisfaction, and placeent changes. The algorith s ability to achieve a higher deand satisfaction is ainly owing to its load-shifting heuristics and the strategy that first does placeent changes to the achines with a high CPU-eory ratio. The algorith s fast speed is ainly owing to the strategy that does placeent changes to achines one by one in an isolated fashion. Two heuristics in the algorith help reduce placeent changes: application instance pinning and achine isolation. For hard placeent probles, the algorith ay siultaneously free a large nuber of application instances on different achines, and then try to place the, which ay produce solutions that siply shuffle application instances across achines (see Figure 7). In contrast, due to the algorith s achine isolation strategy, it never siultaneously frees application instances running on different achines and hence avoids unnecessary shuffling. 338

9 WWW 7 / Track: Perforance and Scalability Execution tie (s) Figure 5: Execution tie with configuration (L cpu=.99, L e=, unifor-apps, reset-allapps). Execution tie (sec) Deand satisfication Placeent changes 8 Figure 6: Scalability of the algorith with configuration (L cpu=.99, L e=, unifor-app, reset-all-apps). 7 Deand satisfication Added applications Placeent changes Added applications Figure 7: Deand satisfaction and placeent changes with configuration: (L cpu=.9, L e=.4, unifor-app, add-apps), achines, and applications. Deand satisfication Placeent changes Figure 8: Deand satisfaction and placeent changes with configuration (L cpu=.99, L e=, unifor-app, vary-two-apps). Deand satisfication Placeent changes Deand satisfication Placeent changes Figure 9: Deand satisfaction and placeent changes with configuration (L cpu=.99, L e=, power-law-app, vary-all-apps). Figure : Deand satisfaction and placeent changes with configuration (L cpu=.9, L e=.6, unifor-app, vary-all-apps). Deand satisfication Placeent changes Deand satisfication Placeent changes Meory Load Factor (Le) Meory Load Factor (Le) CPU Load Factor (Lcpu) CPU Load Factor (Lcpu) Figure : Vary L e fro. to. Configuration: (L cpu=.9, L e=x, unifor-apps, reset-allapps), achines. Figure 2: Vary L cpu fro. to. Configuration: (L cpu=x, L e=.4, unifor-apps, reset-all-apps), achines. 339

10 WWW 7 / Track: Perforance and Scalability 5. RELATED WORK The proble of dynaic application placeent in response to changes in application deands have been studied before. The algorith proposed by Kibrel et al. [6, 8] is the closest to our work. The coparison in Section 4 shows that our algorith significantly and consistently outperfors this algorith. The biggest difference between the two algoriths is that, in the previous algorith, the placeent decisions for individual achines are not isolated stopping one application instance on one achine ay lead to reconsideration of the placeent decisions for all the other achines. A popular approach to dynaic server provisioning is to allocate full achines to applications as needed [3], which does not allow applications to share achines. In contrast, our placeent controller allows this sharing and is optiized for it. The algorith proposed by Urgaonkar et al. [7] allows applications to share achines, but it does not dynaically change the nuber of instances of an application, does not try to iniize placeent changes, and only considers a single bottleneck resource. Placeent probles have also been studied in the optiization literature, including bin packing, ultiple knapsack, and ulti-diensional knapsack probles [7]. The special case of our proble with unifor eory requireents was studied in [2, 3], and soe approxiation algoriths were proposed. These algoriths have been shown to be inferior to the algorith proposed by Kibrel et al. [8]. Our algorith further significantly outperfors an iproved version [6] of the algorith proposed by Kibrel et al. One liitation of our algorith is that it akes no attept to co-locate on the sae achine the set of applications that have high-volue internal counication. This issue has been studied before [5, 5], but it still reains a challenge to design for coercial product a fully autoated algorith that does not rely on anual offline profling. 6. CONCLUSION In this paper, we proposed an application placeent controller that dynaically starts and stops application instances in response to changes in application deands. It allows ultiple applications to share a single achine. Under ultiple resource constraints, it strives to axiize the total satisfied application deand, to iniize the nuber of application starts and stops, and to balance the load across achines. It significantly and consistently outperfors the existing state-of-the-art algorith [6, 8]. Copared with [6, 8], for systes with achines or less, our algorith is up to 34 ties faster, reduces unnecessary application starts and stops by up to 97%, and produces solutions that satisfy up to 25% ore application deands. We believe that our algorith is the first online algorith that, under ultiple tight resource constraints, can efficiently produce high-quality solutions for hard placeent probles with thousands of achines and thousands of applications. This scalability is crucial for dynaic resource provisioning in large-scale enterprise data centers. The outstanding perforance of our algorith stes fro our novel optiization techniques such as application pinning, load shifting, and achine isolation. Our algorith has been ipleented and adopted in a leading coercial product []. 7. REFERENCES [] WebSphere Extended Deployent, 6.ib.co/software/webservers/appserv/extend/. [2] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin, editors. Network Flows: Theory, Algoriths, and Applications. Prentice Hall, New Jersey, 993. ISBN [3] K. Appleby, S. Fakhouri, L. Fong, G. Gszidt, M. Kalantar, S. Krishnakuar, D. Pazel, J. Pershing, and B. Rochwerger. Oceano SLA based anageent of a coputing utility. In Proceedings of the International Syposiu on Integrated Network Manageent, pages 4 8, Seattle, WA, May. [4] A. Fox, S. D. Gribble, Y. Chawathe, E. A. Brewer, and P. Gauthier. Cluster-Based Scalable Network Services. In Syposiu on Operating Systes Principles (SOSP), 997. [5] G. C. Hunt and M. L. Scott. The Coign Autoatic Distributed Partitioning Syste. In OSDI, 999. [6] A. Karve, T. Kibrel, G. Pacifici, M. Spreitzer, M. Steinder, M. Sviridenko, and A. Tantawi. Dynaic Application Placeent for Clustered Web Applications. In the International World Wide Web Conference (WWW), May 6. [7] H. Kellerer, U. Pferschy, and D. Pisinger. Knapsack Probles. Springer Verlag, 4. [8] T. Kibrel, M. Steinder, M. Sviridenko, and A. N. Tantawi. Dynaic Application Placeent Under Service and Meory Constraints. In International Workshop on Efficient and Experiental Algoriths, 5. [9] R. Levy, J. Nagarajarao, G. Pacifici, M. Spreitzer, A. N. Tantawi, and A. Youssef. Perforance anageent for cluster based web services. In Proceedings of the International Syposiu on Integrated Network Manageent, 3. [] G. Pacifici, W. Seguller, M. Spreitzer, M. Steinder, A. Tantawi, and A. Youssef. Managing the response tie for ulti-tiered web applications. Technical Report RC 2365, IBM, 5. [] G. Pacifici, W. Seguller, M. Spreitzer, and A. Tantawi. Dynaic Estiation of CPU Deand of Web Traffic. In Proceedings of the First International Conference on Perforance Evaluation Methodologies and Tools (VALUETOOLS), 6. [2] H. Shachnai and T. Tair. Noah s bagels - soe cobinatorial aspects. In Proc. st Int. Conf. on Fun with Algoriths, 998. [3] H. Shachnai and T. Tair. On two class-constrained versions of the ultiple knapsack proble. Algorithica, 29(3): ,. [4] K. Shen, H. Tang, T. Yang, and L. Chu. Integrated Resource Manageent for Cluster-based Internet Services. In Proc. of OSDI, 2. [5] C. Stewart and K. Shen. Perforance Modeling and Syste Manageent for Multi-coponent Online Services. In Proc. of the Second USENIX Syposiu on Networked Systes Design and Ipleentation (NSDI), 5. [6] C. Tang, R. N. Chang, and E. So. A Distributed Service Manageent Infrastructure for Enterprise Data Centers Based on Peer-to-Peer Technology. In Proc. the International Conference on Services Coputing, 6. Winner of the Best Paper Award. [7] B. Urgaonkar, P. Shenoy, and T. Roscoe. Resource overbooking and application profiling in shared hosting platfors. In Proc. of OSDI, 2. 3