Dynamic Placement for Clustered Web Applications

Transcription

1 Dynaic laceent for Clustered Web Applications A. Karve, T. Kibrel, G. acifici, M. Spreitzer, M. Steinder, M. Sviridenko, and A. Tantawi IBM T.J. Watson Research Center ABSTRACT We introduce and evaluate a iddleware clustering technology capable of allocating resources to web applications through dynaic application instance placeent. We define application instance placeent as the proble of placing application instances on a given set of server achines to adjust the aount of resources available to applications in response to varying resource deands of application clusters. The objective is to axiize the aount of deand that ay be satisfied using a configured placeent. To liit the disturbance to the syste caused by starting and stopping application instances, the placeent algorith attepts to iniize the nuber of placeent changes. It also strives to keep resource utilization balanced across all server achines. Two types of resources are anaged, one load-dependent and one load-independent. When putting the chosen placeent in effect our controller schedules placeent changes in a anner that liits the disruption to the syste. Categories and Subject Descriptors K.6.4 [Coputing Milieux]: Manageent of Coputing and Inforation Systes Syste Manageent General Ters Algoriths, Manageent, erforance Keywords Dynaic application placeent, perforance anageent 1. INTRODUCTION Many organizations rely on web applications to deliver critical services to their custoers and partners. These service providers require application iddleware platfors capable of dynaically allocating resources to eet the perforance goals of these web applications. Today, iddleware platfors for web applications already provide functions like onitoring, access control, and interoperability. Soe of the iddleware platfors also provide clustering functionality. Middleware clustering technology integrates ultiple instances of the sae application running on ultiple server achines so they appear to be a single virtual application. Middleware platfors use clustering technology to provide scalability, high availability and load balancing. Copyright is held by the International World Wide Web Conference Coittee (IWC2). Distribution of these papers is liited to classroo use, and personal use by others. WWW2006, May 22 26, 2006, Edinburgh, UK. ACM /06/0005. To change the aount of coputing resources allocated to clustered web applications requires a iddleware with the ability of dynaically changing the size of each application cluster as well as deciding which server achine each application instance of each cluster will use. We assue that the iddleware hosts ultiple clustered applications. Because of resource liitation, a given server achine cannot run instances for all application clusters at the sae tie. There ay also be hardware or other liitations that ake a achine unsuitable for running soe application clusters. In this paper, we propose the design of and evaluate a iddleware clustering technology capable of dynaically allocating resources to web applications through dynaic application instance placeent. We define application instance placeent as the proble of placing application instances on a given set of server achines to satisfy the resource deand of each application cluster. Our iddleware uses a placeent controller echaniss that dynaically configures the size and placeent of application instances while obeying user-supplied constraints. The controller places applications based on the available server achine resources and the application deands. Each server achine has two resources, one load-dependent and one load-independent. The usage of the load-dependent resource depends priarily on the intensity of application load. The usage of the loadindependent resource does not strongly depend on the current intensity of the application workload. Correspondingly, an application has a load-dependent and a load-independent deand. In the case of web applications, typical exaples for load-dependent and load-independent resources are CU and eory, respectively. The proble of application placeent has been studied before [1, 2, 3, 4]. In [5] we have introduced a placeent algorith that iproves on the prior art in several aspects. (1) It allows ultiple types of resources to be anaged, (2) it ais at axiizing satisfied application deand while iniizing placeent changes copared to the previous placeent, and (3) it is efficient enough to be applicable in online resource allocation. Although soe of prior techniques also consider these factors, none of these techniques addresses all factors siultaneously. In this paper, we extend our prior work in two ways. First, we odify the algorith such that it produces application placeent that allows application load to be better balanced across server achines. To the best of our knowledge this objective has not been investigated in prior solutions to the placeent proble. However, placeents that allow load to be balanced across servers allow applications to perfor better. Second, we investigate two variants of the algorith that iprove its effectiveness in axiizing the aount of satisfied deand and iniizing the placeent churn. The paper is organized as follows. In Section 2 we introduce

2 the architecture of a workload anageent syste in which our controller is ipleented. In Section 3 we forulate and present the algorith for the application placeent proble. In Section 4 we evaluate placeent techniques adopted by the controller. In Section 5 we copare our approach with prior work on this subject. 2. SYSTEM DESCRITION We have ipleented the placeent controller presented in this paper as a part of iddleware technology for anaging the perforance of web applications [6, 7]. Web applications handle web requests and generate static or dynaic web content. In the case of J2EE, runtie coponents of web applications include HTT servers, servlet engines, Enterprise Java Beans (EJB) containers and databases. Web adinistrators deploy these runtie coponents on several or ore server achines, foring a distributed coputing environent. Web application iddleware presented in this paper anages applications running on this distributed coputing environent and allocates resources to the in real-tie to eet their deand. Client Client Client Scheduling L7 roxy Tier Manageent Console Load Balancing Counication Network Application Tier 1 App C-1 Node 1-1 App B-1 Node 2-1 App B-1 Node 3-1 Application Tier 2 Monitoring Control ath Request Flow ath App A-2 App C-2 Node 1-2 App B-2 App C-2 Node 2-2 Figure 1: Manageent syste architecture in a saple Web infrastructure. Figure 1 illustrates the typical architecture of web infrastructure. We show three applications A, B and C deployed across two tiers (Application Tier 1 and Application Tier 2). For exaple, in the case of a J2EE application the first tier hosts servlets and EJBs while the second tier hosts the database. In this exaple, the first tier uses three different server achines while the second uses two server achines. In this paper, we use the ter node when referring to a server achine. In a given tier, a particular application ay run on ultiple nodes. We refer to the presence of an application on a node as an application instance and we use the ter application cluster to indicate the set of all instances of the sae application. By application we ean a set of software entities that for a logical web application unit. Thus, an application ay involve ultiple servlets, ultiple EJB odules, etc. In our experience, a large enterprise web datacenter hosts any different web applications deployed in clusters with a few tens to several hundred application instances spread across a few to a hundred nodes. In alost every case datacenter nodes do not have enough capacity to have all deployed applications be running on the node at the sae tie (in the case of J2EE applications eory is the bottleneck, with each application instance requiring on average 1 2GB of real eory to handle requests). Therefore web adinistrator partitions applications across the available nodes. In the exaple of Figure 1 we have the first tier hosting three applications with each node capable of hosting at ost two instances of each application. We use a placeent atrix to show where application instances run. Each eleent I,n of this atrix represents the nuber of instances of application running on node n. Web-application iddleware environents typically use layer 7 routers to deliver web requests to applications. The routers ap web requests to web applications and send the web requests to a specific instance of the target web application. The layer 7 routers use load balancing techniques and affinity rules to select target instances. The router ay also ipleent adission and flow control. A router equipped with this functionality can control the aount of backend resources used by the different types of web requests. However, its ability to increase the aount of resources used by a given type of web requests is liited by the nuber of application instances targeted by this type of web requests as well as by the capacity of the nodes where the application instances run. For exaple, the placeent presented in Fig. 1 allows the request router to handle a big aount of web request load for application A but uch saller load for applications B and C. When application C experiences a spike of web requests while the workload of application A decreases, the web requests destined for application C cannot use ore capacity in Tier 1 than that of the node on which application C is running. Depending on the control logic of the request router, application C will experience either high service tie in Tier 1, long waiting tie in the routing layer, or high request rejection rate. At the sae tie, spare capacity in the syste will reain on the reaining nodes in Tier 1. To adapt to such workload changes, we ust odify the application placeent dynaically in reaction to the changing load. In the considered scenario, a desired syste response to the workload shift would be to start new instances of application C on nodes 2-1 and 3-1 and stop instances of application A on these nodes (to ake roo for the new instances). When we deal with a large nuber of applications with ultiple resource requireents and a large nuber of nodes with heterogeneous capacities, deciding on the best application placeent for a given workload is non-trivial. In this paper we forulate the placeent proble as an instance of a two-diensional packing proble and we study several algoriths for solving it. We also describe a placeent controller that runs on our iddleware and dynaically changes the size of application clusters to respond to dynaic load variations. 2.1 Application placeent architecture An application instance ay run on any node that atches application requireents. In J2EE doain, these properties concern features that cannot be changed using J2EE runtie environent, such as network connectivity, bandwidth, the presence of non-j2ee software libraries, etc. In our syste, an application is deployed to all nodes that atch its properties. Application deployent involves creating persistent configuration of an application server that hosts the application, which includes the definition and configuration of J2EE resources and paraeters. We configure application placeent by starting and stopping the application servers as needed. When an application instance is stopped on a node it does not consue any of the node s runtie resources.

3 Using server start and stop operations to control the nuber and placeent of application instances provides a relatively light-weight resource allocation echanis because it does not require a potentially tie-consuing application deployent, server configuration, distribution of configuration files, etc. A possible disadvantage of such an approach is its soewhat liited flexibility as it does not allow arbitrary cobinations of applications to be deployed and executed on application servers. The controller described in this paper requires the anaged syste to provide certain functionality. When this functionality is present, the controller is also suitable for the anageent of other than Web applications. Siilarly, it could use other control echaniss than application instance starts and stops. For exaple, in the presence of virtualization technology [8, 9], we could start and stop virtual partitions as well. The list of the required functions includes the following ites. laceent sensor we require that for each application there exist a echanis that allows us to obtain the up-to-date inforation about the location of its instances. laceent effecter for each application, we need a echanis that allows us to start or stop its instance on a given node. Request router the request dispatching echanis ust transparently adjust to placeent changes. Application deand estiators the syste ust provide online estiates of the current application deand for all anaged resources. 2.2 Characterizing application requireents In the anageent syste described in this section, the size and placeent of application clusters is deterined autoatically based on application requireents and node capacities. Application requireents are characterized as load-independent and load-dependent ones. Load-independent requireents describe resources needed to run an application on an application server, which are used regardless of the offered load. Exaples of such requireents are eory, counication channels, and storage. In this paper, we focus on a single resource eory. Although, J2EE applications also use other resources in a load-independent anner, we have observed that it is eory that is the ost frequent bottleneck resource in J2EE applications. We have classified eory as a load-independent resource since a significant aount of it is consued by a running application instance even if it receives no load. In addition, eory consuption is frequently related to prior application usage rather than to its current load. For exaple, even in the presence of low load, eory consuption ay be high as a result of caching. Thus, although the actual eory consuption also depends on the application load, characterizing eory usage while taking the load-dependent factor into account is extreely difficult. It is ore reasonable to estiate the upper liit on the overall eory utilization and treat it as a load-independent eory requireent. Our syste includes a coponent that dynaically coputes this upper liit based on a tie-series of eory usage by an application. This coponent is not described in this paper. Consequently, in this paper we assue that an estiate of eory requireents is available. Load-dependent requireents correspond to resources whose consuption depends on the offered load. Exaples of such requireents are current or projected request rate, CU cycles/sec, disk activity, and nuber of execution threads. In this paper, we focus on CU as a resource that is consued in a load-dependent anner as it is the ost frequent bottleneck resource in J2EE applications. CU deand is expressed as the nuber of CU cycles per second that ust be available to an application. Coputing this estiate is a non-trivial proble, which is not discussed in this paper. However, our syste coputes such an estiate online. It includes a coponent, an application profiler [6] that helps us in this process. The application profiler coputes the nuber of CU cycles consued by a request of an application. This per-request CU deand of an application is then ultiplied by the required request throughput for the application, whose estiation depends on adission or flow control echanis used by the request router. If no adission or flow control is ipleented by the router, then throughput prediction ay be based on tie-series of prior throughput observed at application servers. If either adission or flow control is ipleented by the request router, then the prediction is ade based on prior workload characteristics observed at the request router while taking into account the control logic of the request router. In fact, a request router provided by our syste ipleents SLA driven flow control [10]. We use workload observations at the router and its optiization algorith to estiate application throughput that ust be delivered by the syste to axiize the overall syste utility score. Correspondingly, node capacity is characterized using load-independent and load-dependent values that describe the aount of resources used to host an application on a node and the available capacity to process requests of applications. Siilar to application requireents, we focus on the size of eory and the speed of CU as load-independent and load-dependent capacity easures. 2.3 Syste architecture The dynaic control of the size and placeent of dynaic clusters is realized by the syste depicted in Figure 2. Request Router App C-1 Node 1-1 App B-1 Node 2-1 App B-1 Node 3-1 application and resource easureents application-server status events stop start laceent Sensor laceent Effecter current placeent Application Deand Estiators application workload prediction desired placeent start/stop coands laceent Controller laceent Executor configuration changes current placeent approval for supervised ode user Figure 2: Architecture of the placeent anageent coponent At the heart of the syste there is laceent Controller which coputes a new application placeent based on load-independent and load-dependent resource deand estiates for all applications. These estiates are obtained online using Application Deand Estiators. In each control cycle, the laceent Controller coputes a new placeent atrix with the objective to satisfy the entire load-dependent deand of applications without exceeding load-dependent

4 and load-independent resource capacities of the anaged nodes. In addition, the laceent Controller works with the following considerations: Each start and stop operation is potentially costly in ters of resource consuption and the ipact on the running applications. Depending on the processor speed it ay take between tens of seconds to several inutes. The starting or stopping process also consues a significant fraction of CU while it occurs. Therefore, the laceent Controller ais at iniizing the nuber of placeent changes copared to the placeent currently in effect. Oftenties, ore than one placeent exists with the sae aount of satisfied deand and the nuber of changes copared to the previous placeent. Yet, these placeents ay differ with respect to how effectively they utilize available resources. laceents that tend to pack a lot of load on a few nodes adversely affect perforance and high-availability of applications. Therefore, laceent Controller chooses configurations that allow load to be as evenly distributed across all nodes as possible. A user ay want to prevent soe applications fro being anaged by the laceent Controller by putting the in unanaged ode. Applications in unanaged ode cannot be started or stopped, but their load ust be taken into account when coputing placeent for all other applications. Nodes ay differ with respect to their ability to host a particular application. laceent Controller ust observe allocation restrictions introduced as a result of this heterogeneity. Once placeent is coputed by the laceent Controller, it is effected by the laceent Executor, which ipleents the placeent by perforing individual start and stop operations on the application servers. As each start and stop operation disrupts the syste, the laceent Executor orders the individual placeent changes in a anner that iniizes the disruption to the running applications. laceent Executor ensures that: No ore than one application instance is being started or stopped on any node at a tie. Load-independent capacity constraint is observed during placeent changes. If it is necessary to teporarily violate a node s load-independent capacity constraint, the utilized nodeindependent capacity exceeds the available capacity by no ore than one instance. At least one instance of each application is running at all ties. The laceent Executor delays the start and stop actions that would violate the above policies given actions that are already in progress. It onitors the progress of the initiated start and stop actions and reevaluates the actions that have been delayed. As soon as it is allowed by the policies, the laceent Executor initiates the delayed actions. The laceent Executor ipleents placeent changes using laceent Effecter. It onitors the status of the initiated changes and keeps track of placeent changes effected by other anagers or a user using the laceent Sensor. The syste we describe above is highly dynaic allowing various types of configuration changes, such as adding or reoving a node, creating and destroying an application cluster, or changing node or cluster properties to be accoodated at runtie. In addition, all coponents are highly available. This paper does not attept to present all coponents of the syste described in this section, although all of the have been ipleented as a part of our project. We focus on the functionality of laceent Controller and in the following sections we discuss it in detail. 3. LACEMENT CONTROLLER The laceent Controller drives a control loop within which a new placeent is coputed and ipleented. The execution diagra of the control loop is shown in Figure 3. The rest of this section describes the optiization proble solved by the laceent Controller. wait ΔT wait δt yes Input: current placeent atrix I * load-dependent deand vector ω load-independent resource requireents γ load-dependent capacity vector Ω load-independent capacity vector Г... cansolve(i *, ω, γ, Ω, Г) no I coputenewlaceent(i *, ω, γ, Ω, Г) ipleent I Figure 3: laceent control loop laceent Executor Application Deand Estiators Configuration DB laceent Executor 3.1 roble Stateent We forulate the placeent proble as follows. We are given a set of nodes, N, and write n for a eber of that set. We are also given a set of applications M, and write for a eber of that set. With each node n we associate its load-independent and loaddependent capacities, Γ n and Ω n, which correspond to the node s eory and CU power, respectively. With each application, we associate its load-independent and load dependent deand values, γ and ω, which correspond to the application s eory and CU requireents, respectively. We assue that γ is the aount of eory consued by an instance of application, which is independent of offered load. The CU-speed requireents of the application, ω, depend on offered load. With each application we also associate a boolean flag anaged indicating whether instances of the application can be started or stopped by the placeent controller. Also, with each application and each node we associate the allocation restriction flag, R,n indicating whether an instance of the application ay be started on node n. The placeent proble is to find atrix I, where I,n is 1 if an instance of application is running on node n and 0 otherwise, such that (1) allocation restrictions are observed, (2) load-independent capacity liits are observed, and (3) it is possible to allocate load of all applications to instances without exceeding load-dependent capacity liits. If a previous placeent atrix I,n exists, we also want to iniize the nuber of changes between the old and the new atrix. If it is not possible to satisfy the entire deand of applications, we ai at axiizing the aount of satisfied deand. Finally, aong all placeents that axiize the aount of

5 satisfied deand, we want to find one that allows the load allocated to nodes to be balanced. Since we are dealing with ultiple optiization objectives, we prioritize the in the foral stateent of the proble as follows. Let L be a atrix of real valued load placeent and ρ = s.t: n L,n. n Ωn (i) ax n L,n (ii) in (iii) in n I,n I,n (1) n ( L,n Ω n ρ) 2 X n I,nγ Γ n (2) X n L,n Ω n (3) X L,n ω (4) n ni,n = 0 L,n = 0 (5) nr,n = false I,n = 0 (6) anaged = false ni,n = I,n (7) The optiization proble solved by the laceent Controller is a variant of the Class Constrained Multiple-Knapsack roble [11] (CCMK) and differs fro prior forulations of this proble ainly in that it also iniizes placeent changes. Since this proble is N-hard, the laceent Controller adopts a heuristic, which is presented in Section 3.2. The laceent Controller avoids solving the optiization proble if the current placeent can serve the new deand. This is achieved by first testing whether there exists a load-distribution atrix for placeent atrix I that satisfies constraints expressed by Eqns. (2)-(5) (function cansolve in Figure 3). This test fors a failiar linear-prograing proble, for which a nuber of efficient solutions ay be found in literature [12, 13]. If the current placeent can serve the new deand, after waiting δt, the laceent Controller proceeds to the next control cycle. Otherwise, the new placeent is coputed and ipleented. After placeent is changed, the laceent Controller does not resue the new cycle for T > δt. The increased tieout provides sufficient tie for the syste to adjust to the new placeent and reduces the nuber of placeent changes. Both δt and T are configurable paraeters with 1 inute and 15 inutes as their default values. 3.2 laceent Algorith In this section we present an outline of a heuristic algorith used by the laceent Controller to solve the placeent proble. The basic algorith consists of three parts: the residual placeent, increental placeent, and rebalancing placeent. The residual placeent is used at the syste start-up, when there is no prior placeent, and as a subroutine of the increental placeent. The rebalancing placeent is invoked at the end of increental placeent, once the axiizing placeent is found, to odify it in a way that allows for a better distribution of load-dependent deand Residual placeent Residual placeent is based on the following intuitive rule. If we look at allocated eory as the cost of allocating an application s CU deand, it is wise to first place an application with the highest eory requireent copared to its CU deand. This way, we can axiize the chances that this application will be placed on the fewest possible nuber of nodes, and thus the cost of satisfying its CU deand will be the lowest. When choosing a node on which to place an application, it is reasonable to first search for a node with density siilar to the density of the application. It is not wise to load applications with high density on a low density server, since we would be likely to reach the processing capacity constraint and leave a lot of eory unused on that server. Siilarly, if low density applications are loaded on high density servers, we would be likely to reach the server s eory constraint without using uch of the processing capacity. In the residual placeent, nodes are ordered by the increasing value of their densities, Ω n/γ n, and applications are ordered by increasing densities ω /γ n. The algorith starts with the lowest density application and looks for the lowest density node that can fit the application, i.e., a node n such that γ Γ n and R,n is true. If the node satisfies the entire load-dependent capacity of, i.e., Ω n ω then the application is reoved fro the application list and the node is added back to the ordered node list after its load-dependent capacity is decreased by ω. Otherwise, the entire node s available load-dependent capacity is allocated to the application, the node is reoved fro the node list, and the algorith searches for another node on which to place the residual deand of the application. The algorith proceeds until the deand of all applications is assigned to nodes, or no ore assignents are possible given nodes capacities. The coputational coplexity of residual placeent is O( N M ) Increental placeent The increental placeent cobines the residual placeent with the axiu flow coputation to solve the placeent proble while iniizing the nuber of placeent changes. It derives the new placeent I fro the previous placeent I increentally as follows. In a given iteration of the algorith, we first check of the current placeent, I can satisfy the deand. This is done by solving the axiu bipartite flow proble represented by Eqns. (1- i) and (3)-(5). If a solution exists, i.e., ax n L,n = ω, the algorith copletes. Otherwise, we are left with soe residual deand for each application that could not be satisfied given the current placeent, ω n L,n. For soe applications this residual deand ay be zero. We are also left with soe residual load-dependent and load-independent capacity on each server, Ω n L,n and Γn I,nγ. We apply the algorith fro Section to find an allocation of the residuals. If the residual placeent succeeds in finding a feasible allocation, we set I to the resulting allocation and exit. Otherwise, we reove the assignent with the lowest density, L,n/γ fro I and proceed to the next iteration. Observe, that the nuber of iterations perfored by the increental algoriths depends on how hard it is to find the satisfying placeent. The proble is hard to solve if the total deand of applications copared to the total available capacity is high, or the total eory requireent of applications approaches total eory available on nodes. The ore difficult the proble is to solve, the longer it takes for the algorith to coplete. The upper bound on the nuber of iterations is the nuber of assignents in the current I,n. placeent, i.e., n When applications in unanaged ode are present, in the first iteration of the algorith we adopt a slightly odified procedure. We first find the axiu load allocation for applications in un-

6 anaged ode. Then we subtract the capacity utilized by the fro the total available capacity. If any residual deand of these applications reains, it is ignored; the unanaged applications are reoved fro further consideration and the algorith proceeds according to the above procedure Rebalancing placeent The last phase of the placeent algorith ais at odifying the solution proposed by the increental algorith such that a better load balancing across nodes is achieved. This phase begins with old and new placeent atrices I and I and load distribution atrix L. We first copute the aount of deand satisfied by L for all applications: ω + = n L,n. Then we try to find another load distribution atrix L + that satisfies the sae deand for all dynaic clusters and that perfectly balances the load assigned to nodes. We find L + by solving the following optiization proble: s.t: in X n X L +,n ρω n, where ρ = ω+ n Ωn (8) X n L +,n Ω n (9) X L +,n = ω + (10) n ni,n = 0 L +,n = 0 (11) The above proble ay be easily transfored into a in-cost flow proble in a bipartite flow network for which a nuber of polynoial tie algoriths exist [13]. Note that if there exists a load-distribution that achieves the perfect balance of node utilization (utilization of all nodes equal to ρ), then the above optiization will find such a load-distribution. When perfect loaddistribution cannot be found, the load-distribution found by the optiization proble includes soe nodes loaded above ρ as a utilization threshold and soe nodes loaded below ρ. The point is to change assignents in such a way so as to allow shifting soe load fro overloaded nodes to underloaded nodes. Only assignents in I I ay be oved. Assignents that overlap with prior placeent cannot be oved as this could increase placeent churn. The rebalancing placeent proceeds fro the ost overloaded node and attepts to ove soe of its instances to one or ore underloaded nodes. We always choose an instance whose reoval brings the node utilization closest to ρ. This procedure continues until all nodes are balanced or no ore useful changes can be done. 3.3 Algorith variants When the algorith described in the previous section is applied to the syste in which load-dependent deand of applications is high copared to the total available load-dependent capacity, its execution tie increases, and the ability to find the optial placeent decreases. We have observed that the sae algorith applied to a deand that is reduced to a certain percentage of the total available capacity not only executes faster, but is also ore likely to produce placeent that satisfies the entire real deand. Another factor that ipacts the effectiveness of the algorith is the content of the previous placeent atrix; the sae algorith applied to different prior placeent atrices produces different results. Taking these observations into account, we have ipleented three siple variants of the placeent algorith. Basic algorith (BA) - the algorith as described in the previous section Load-reduction algorith (LRA) - basic algorith executed with odified input; whenever total application deand exceeds 90% of the total capacity we proportionately reduce the deand of each application so as to bring the total deand down to 90% of the total available capacity Multiple-runs algorith (MRA) - if placeent atrix produced by the basic algorith does not satisfy the entire deand, we execute the basic algorith one ore tie using the output of the first execution as prior-placeent atrix. We repeat this process until no ore iproveent in the aount of satisfied deand is observed but no ore that 10 ties. 4. SIMULATION RESULTS In this section we study the proposed placeent algoriths via siulation. We investigate the algoriths in three dientions: the effectiveness in satisfying the deand, ability to achieve balanced load distribution, and coputational coplexity. 4.1 Effectiveness of the algoriths In the siulation study, we vary the size of the proble (by varying the nuber of nodes and dynaic clusters) and the hardness of the placeent proble that is being solved Defining proble hardness The hardness of the placeent proble is affected by three factors: eory load factor, CU load factor, and workload variability factor. When we ignore iniu/axiu instances policies, vertical stacking, and allocation restrictions, we can define the hardness factors as follows. Let γ and Γ be the average load-independent deand and capacity values, respectively. The expected axiu nuber of applications that ay be hosted on a group of N nodes is Γ M γ N. The eory load factor is defined as MLF = γ N Γ, where M is the nuber of applications. The CU load factor is defined as CLF = Ω = n Ωn. ω Ω, where The placeent proble is harder to solve if the current loaddependent deand distribution differs significantly fro the distribution at the tie of last placeent coputation. Let ω(t) be a rando variable of load-dependent deand of an application at tie t. Suppose that we know the axiu value of CLF. The axiu change of load-dependent deand distribution occurs, for exaple, when in each control cycle the entire deand in the syste, CLF Ω, is contributed by just one, each tie different application. The average axiu load-dependent deand change is then CLF Ω M. Given the actual probability distribution function of ω(t), we easure workload variability by dividing the expected per-application absolute difference between load-dependent deand values in the current and previous cycles by the average axiu difference, i.e., using workload variability factor WVF = E( ω(t + 1) ω(t) )/ CLF Ω M 1. When WVF increases, the placeent algorith results in ore placeent changes. When WVF = 0, after the initial placeent is found no placeent changes are ever needed.

7 4.1.2 Varying the proble hardness In the experiental design, we vary the nuber of nodes and dynaic clusters. Load independent capacity and deand values are uniforly distributed over sets {1, 2, 3, 4} and {0.4, 0.8, 1.2, 1.6}, respectively with average values of Γ = 2.5 and γ = 1, respectively. Load-dependent capacity is the sae for all nodes and is equal to 100. To achieve the eory load factor of MLF given the nuber of nodes N, for each trial we vary MLF between 40% and 100%, and set M to 2.5 MLF N. To control the CU load factor CLF we generate load-dependent deand randoly and noralize it such that its total aount is equal to CLFΩ. We vary CLF between 90% and 99%. We focus on only one way to generate load-dependent deand. In our experients, ω(t) is uniforly distributed over the entire range independently of ω(t 1). This technique yields the workload variability factor WVF = 2 and is harsh 3 on the algorith. In reality, ω(t) is expected not to differ very uch fro ω(t 1) over short tie intervals Evaluation criteria We copare the algorith variants based on the following criteria: (1) the percentage of satisfied deand, (2) the nuber of placeent changes, and (3) execution tie. These easureents are collected using 100 experients. In each experient we randoly generate syste configuration (i.e., load-independent deand and capacity values), and then perfor a sequence of 11 placeent coputations including one initial placeent. In each coputation, a new load-dependent deand vector is generated. For each coputed placeent, the percentage of satisfied deand is calculated by first finding the load-distribution atrix L that axiizes the aount of satisfied deand given the coputed placeent, as defined through Eqns. (3)-(5). Then we calculate n L,n. ω Results In Figs. 4, 5, and 6, we present experiental results for the fraction of deand satisfied, nuber of placeent changes, and execution tie, respectively. The results are presented with 95% confidence intervals. When eory load factor is high, e.g., MLF=100%, the algorith is eory-constrained and regardless of the CU load factor it usually fails to place all applications and consequently does not eet CU deand. As shown in Fig. 4, in a very eoryand CU constrained scenario of MLF=100% and CU=99%, the Multiple-runs Algorith (MRA) and Load-reduction Algorith (LA) appear to perfor soewhat better than the Basic Algorith (BA), however, all algoriths fail to eet the entire load-dependent deand. Also, they generate roughly the sae (huge) nuber of placeent changes (Fig. 5). Fro the perspective of execution tie, extreely resource-constrained probles are particularly harsh on the Multiple-runs Algorith because they trigger ultiple algorith iterations thereby elevating the placeent coputation tie (Fig. 6). However, when the eory constraint is relaxed to MLF=60%, the placeent proble becoes easier to solve and the differences between algoriths ore obvious. As shown in Fig. 4, MRA satisfies ore deand than LRA and BA. This result is achieved with the sae nuber of placeent changes as with BA. LRA perfors worse than MRA in ters of satisfying the deand but it proves better in reducing the placeent churn (Figs. 4 and 5). The iproved effectiveness of MRA coes at the price of ultiplied execution tie copared to LRA and BA. Our experients have revealed that it not necessary to reduce MLF to as low as 60%. We have observed siilar results, though not as proinent, at MLF=80% as well. When eory and CU constraints are further reduced (to say MLF=60% and CLF=90%), the placeent proble becoes easy to solve and all algoriths perfor equally well. Figs. 4-6 show the results obtained with MLF=40% and CLF=90%. Fraction of deand satisfied Nuber of nodes (Alg, MLF, CLF) (BA, 100%, 99%) (LRA, 100%, 99%) (MRA, 100%, 99%) (BA, 60%, 99%) (LRA, 60%, 99%) (MRA, 60%, 99%) (BA, 40%, 90%) (LRA, 40%, 90%) (MRA, 40%, 90%) Figure 4: Fraction of deand satisfied by algoriths Nuber of placeent changes (Alg, MLF, CLF) (BA, 100%, 99%) (LRA, 100%, 99%) (MRA, 100%, 99%) (BA, 60%, 99%) (LRA, 60%, 99%) (MRA, 60%, 99%) (BA, 40%, 90%) (LRA, 40%, 90%) (MRA, 40%, 90%) Nuber of nodes Figure 5: Nuber of placeent changes 4.2 Effectiveness of load distribution In this section we evaluate the effectiveness of rebalancing placeent applied to MRA. For this purpose, we copare two versions of this algorith: with and without the rebalancing placeent phase. We copare the algoriths while looking at load distribution in the initial placeent, which starts with the epty set of placeent assignents, and at the average of ten subsequent placeents, which start with a non-epty set of prior placeent assignents. Distinguishing these two cases is iportant as in the initial placeent we

8 laceent tie [sec] (Alg, MLF, CLF) (BA, 100%, 99%) (LRA, 100%, 99%) (MRA, 100%, 99%) (BA, 60%, 99%) (LRA, 60%, 99%) (MRA, 60%, 99%) (BA, 40%, 90%) (LRA, 40%, 90%) (MRA, 40%, 90%) Gini index initial, no rebalancing initial, with rebalancing average, no rebalancing average, with rebalancing Nuber of nodes CLF Figure 6: laceent coputation tie typically have greater freedo in replacing instances than in subsequent placeents, where only a subset of all instances is eligible for being oved Evaluation criterion To assess the effectiveness of rebalancing placeent we ust consider that load distribution is perfored by a router. It is the effectiveness of the router that ultiately affects the evenness of the load distribution. In this section, we assue a router that can perfectly balance load within liits iposed by the placeent in effect. To obtain load distribution that is achieved by such an ideal router, we need to solve the optiization proble expressed by Eqns. (1-iii) and (3)-(5). This non-linear proble ay be quite easily solved as a sequence of at ost N probles defined by Eqns. (8)-(11), which are straightforward to linearize. Once the ost balanced distribution is known, we ust easure the aount of inequality in the distribution. As a good candidate of such a easure, we consider the Gini index [14] (Gini), which is defined as a ratio of the area between the line of perfect (unifor) distribution (45 degree line) and the Lorenz curve of the actual distribution to the total area below the 45-degree line. This coefficient is ost frequently used as a easure of incoe inequality. The Gini index of 0 indicates perfect equality. The Gini index approaches one to represent extree ibalance Experient design We evaluate the effectiveness of rebalancing placeent in a network coposed of 10 nodes with load-independent deand uniforly distributed over a set {1, 2, 3, 4}.. We copute placeent for a syste with 10, 30, and 50 applications with identical eory requireents of 0.4. This gives us MLF values of 16%, 48%, and 80%, respectively. We vary CLF fro 5% to 95% Results Figs. 7-9 show initial and average Gini indices for MLF values of 16%, 48%, and 80%, respectively. Our study reveals that the potential iproveent in load balance depends on the nuber of degrees of freedo the rebalancing placeent has while oving instances. For low values of MLF, we deal with only a sall nuber of instances that can be oved with relatively high fraction of total deand allocated to each of the. For high values of MLF, we deal with any instances with poten- Figure 7: Gini index of initial placeent and the average Gini index for MLF=16% Gini index CLF initial, no rebalancing initial, with rebalancing average, no rebalancing average, with rebalancing Figure 8: Gini index of initial placeent and the average Gini index for MLF=48% tially sall fraction of total load allocated to the but we have little extra space to ove these instances to. In addition, high MLF iplies that load allocated to each instance is also high. Oftenties, we can only iprove balance by creating an additional instance for an application and distributing the application load across the increased nuber of instances. However, doing so would result in increasing the disruption to the syste and is not considered by the rebalancing algorith even when applied to initial placeent. Our study also shows that initial placeent typically has a uch better potential for distributing load than subsequent placeents. This observation concerns both the algorith with rebalancing and the one without it. This difference results fro the way we prioritized the objectives: we effect the fewest nuber of changes to the previous placeent that are necessary to satisfy the entire deand even though the resultant load distribution ay be unbalanced. Our study also concerns the analysis of rebalancing cost, which ay be quantified as an increase in placeent tie. However, our study shows that this cost is negligible. In a syste of 10 nodes and 30 applications, the rebalancing phase takes less than 10 sec and is invoked only when placeent changes are suggested by the

9 Gini index initial, no rebalancing initial, with rebalancing average, no rebalancing average, with rebalancing Figure 9: Gini index of initial placeent and the average Gini index for MLF=80% algorith. In configurations where placeent changes are frequent (for exaple with high CLF), the rebalancing tie is only a sall fraction of the overall execution tie. laceent rebalancing affects not only the placeent tie, but also the nuber of placeent changes. Even though our rebalancing algorith does not directly introduce any additional placeent changes, it odifies the input to the subsequent control cycle. This odified input ay lead to a placeent with a higher or lower nuber of placeent changes than if no rebalancing was perfored. However, in our study we could not find any statistically significant difference in the average nuber of placeent changes between the two algoriths. 5. RELATED WORK The proble of dynaically allocating server resources to applications has been extensively studied. A popular approach to this proble relies on dynaic server provisioning in which full server achines are allocated to applications (and provisioned for the) as needed [15]. This approach does not allow applications to overlap on nodes (unless several applications are grouped and anaged as one). The solution proposed in this paper allows ultiple applications to share the sae server thereby achieving finer granularity of allocation and better utilization of resources copared to dedicated-server approaches. Chandra et al. [16] have argued that such fine-grained resource allocation at sall tie-scales can lead to substantial ultiplexing gains. The proble of application placeent in a shared cluster of servers has been investigated by Urgaonkar at el. [1]. In their forulation, instances of an application that execute on different hardware servers, which are called capsules, are anaged independently of one another. Capsule resource requireents are estiated independently based on prior resource usage by a capsule. Then capsules are packed on servers in such a way so as not to exceed each server s capacity. Independent packing of capsules is a rather inflexible echanis of application placeent as it does not allow the nuber of capsules serving a given application to be changed or the load to be shifted fro one capsule to another. Another issue with the approach presented in [1] is that capsule requireents are characterized using a single value that can represent either CU or network bandwidth requireents. The approach introduced in this paper allows applications to be placed based also on their eory CLF requireents. Finally, the technique in [1] ignores previous capsule placeent while finding the new placeent. It is therefore unsuitable for frequent online placeent odifications, which is a goal of the technique described in this paper. In the work of Ardagna et al. [4], the placeent proble ais at iniizing the costs of running the server achines: each achine has a cost, and it is paid if any tier of any application is placed there. This objective is different fro ours. Unlike Ardagna et al., we ust be concerned with the possibility that soe deands ay not be et and therefore try to axiize the aount of deand that can be satisfied. On the other hand, our approach does not consider the cost of running the server achines. There are ore differences between our proble stateents: they accept co-residency prohibitions of a certain for; our forulation does not have this feature; our forulation includes a desire to iniize changes fro an existing placeent, while their forulation has no such concern; they are concerned with a larger resource allocation proble that also includes routing and scheduling, while we solve placeent proble independently. Wolf and Yu in [2] propose an algorith that aps Web sites to cluster servers allowing an overlap between Web-site clusters. The algorith proposed in [2] attepts to iniize the nuber of placeent changes by exploring new configurations starting fro those that differ no ore than one Web-site-to-server assignent fro the current placeent and liiting the total nuber of placeent changes to a configured nuber. Unlike the technique proposed in this paper, Wolf et al. [2] focus on a single-resource capacity constraints. Moreover, their optiization technique is intended to be used infrequently (about once a week) and it ay be executed on a dedicated server. Hence, there is no concern about its coputational overhead. In the proble discussed in this paper, the application placeent is re-evaluated frequently (in the order of inutes) and the coputation is co-located with applications. Therefore, its coputational overhead ust be very sall. Our work can be also copared to the work on Cluster Reserves by Aron et al. [3], where an application and application instance considered in our paper correspond to a service class and a cluster reserve in [3]. The resource allocation proble solved in [3] is to copute the desired percentage of each node s capacity to be allocated to each service class based on overall cluster utilization ratios desired for each service class. This proble differs fro the proble solved in our paper in several aspects. First, we ai at finding the best placeent of application instances on nodes rather than an optial allocation of each node s capacity. Our technique does copute such an allocation as a side effect. However, this allocation is not enforced, eaning that we only copute it to deterine if it is feasible to allocate applications deand to nodes with a given placeent. The actual resource allocation is deterined by the request router at a uch finer tie scale, which iproves the syste responsiveness to workload changes. Second, we ai at satisfying the ost of the offered deand rather than keeping the relative resource allocation as close to the goal as possible. Third, our proble takes into account the liited eory capacity of nodes, which constraints the nuber of application instances that ay be placed on each of the. Finally, the objectives of our optiization proble include iniizing changes fro the previous allocation, which is not the case in [3]. The proble of optially placing replicas of objects on servers, constrained by object and server sizes as well as capacity to satisfy a fluctuating deand for objects, has appeared in a nuber of fields related to distributed coputing. In anaging video-ondeand systes, replicas of ovies are placed on storage devices and streaed by video servers to a dynaic set of clients with a

10 highly skewed ovie selection distribution. The goal is to axiize the nuber of aditted video strea requests. Several ovie placeent and video strea igration policies have been studied. A disk load balancing criterion which cobines a static coponent and a dynaic coponent is described in [17]. The static coponent decides the nuber of copies needed for each ovie by first solving an apportionent proble and then solving the proble of heuristically assigning the copies onto storage groups to liit the nuber of assignent changes. The dynaic coponent solves a discrete class-constrained resource allocation proble for optial load balancing, and then introduces an algorith for dynaically shifting the load aong servers (i.e. igrating existing video streas). A placeent algorith for balancing the load and storage in ultiedia systes is described in [18]. The algorith also iniizes the blocking probability of new requests. In the area of parallel and grid coputing, several object placeent strategies (or, eta-scheduling strategies) have been investigated [19, 20]. Counication overhead aong objects placed on various achines in a heterogeneous distributed coputing environent plays an iportant role in the object placeent strategy. A related proble is that of replica placeent in adaptive content distribution networks [21, 20]. There the proble is to optially replicate objects on nodes with finite storage capacities so that clients fetching objects traverse a iniu average nuber of nodes in such a network. The proble is shown to be N-coplete and several heuristics have been studied, especially distributed algoriths. Siilar probles have been studied in theoretical optiization literature. The special case of our proble with unifor eory requireents was studied in [22, 23] where soe approxiation algoriths were suggested. Related optiization probles include bin packing, ultiple knapsack and ulti-diensional knapsack probles [24]. 6. CONCLUSIONS AND FUTURE WORK In this paper, we propose the design of and evaluate a iddleware clustering technology capable of dynaically allocating resources to web applications through dynaic application instance placeent. We define application instance placeent as the proble of placing application instances on a given set of server achines to satisfy the resource deand of each application cluster. We introduce a placeent controller echanis that dynaically configures the size and placeent of application instances while obeying to user-defined policies. We introduce a placeent algorith that can copute placeent obeying load-dependent and load-independent capacity constraints. The algorith produces a placeent that liits the nuber of placeent changes and allows load to be better balanced across nodes. In the experiental study we evaluate the proposed algorith and its variants with respect to their ability to satisfy the application deand, reduce the nuber of placeent changes, and evenly distribute load. Our work can be iproved in a nuber of ways. First, our algorith does not explicitly odel a cost of a placeent change. We only assue that this cost is high and therefore worth iniizing. A possibly better approach could odel the ipact of a placeent change on the perforance of applications. Instead of iniizing cost, it would attept to axiize the overall syste utility. This new approach would be particularly suitable when application start or stop echanis is a light-weight one, e.g., with the usage of OS-level virtualization technology. Second, we do not prioritize applications. If the entire deand cannot be satisfied, soe applications will be affected either by their increased execution tie, or increased waiting tie, or increased rejection rate. We do not attept to control which applications are affected this way. Third, our rebalancing technique ais at equalizing CU utilization across nodes. However, with heterogeneous nodes, better application perforance ay be achieved if response ties of an application on different nodes are equalized instead. Our technique does not odel or anage response ties as a function of node utilization. 7. REFERENCES [1] B. Urgaonkar,. Shenoy, and T. Roscoe, Resource overbooking and application profiling in shared hosting platfors, in roc. Fifth Syposiu on Operating Systes Design and Ipleentation, Boston, MA, Dec [2] J. Wolf and. Yu, On balancing load in a clustered web far, ACM Transactions on Internet Technology, vol. 1, no. 2, pp , [3] M. Aron,. Druschel, and W. Zwaenpoel, Cluster reserves: A echanis for resource anageent in cluster-based network servers, in roc. ACM SIGMETRICS, Santa Clara, CA, Jun [4] D. Ardagna, M. Trubian, and L. Zhang, SLA based profit optiization in ulti-tier web application systes, in Int l Conference On Service Oriented Coputing, New York, NY, 2004, pp [5] T. Kibrel, M. Steinder, M. Sviridenko, and A. Tantawi, Dynaic application placeent under service and eory constraints, in Int l Workshop on Efficient and Experiental Algoriths, Santorini Island, Greece, May [6] G. acifici, W. Seguller, M. Spreitzer, M. Steinder, A. Tantawi, and A. Youssef, Managing the response tie for ulti-tiered web applications, IBM, Tech. Rep. RC 23651, [7] WebSphere extended Deployent (XD), [8]. Barha, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. ratt, and A. Warfield, Xen and the art of virtualization, in Syposiu on Operating Systes rinciples (SOS), Bolton Landing, NY, [9] VMware, [10] R. Levy, J. Nagarajarao, G. acifici, M. Spreitzer, A. N. Tantawi, and A. Youssef, erforance anageent for cluster based web services, in roc. Int l Syposiu on Integrated Network Manageent, Colorado Springs, USA, March 2003, pp [11] H. Shachnai and T. Tair, Noah s bagels soe cobinatorial aspects, in roc. Int l Conf. Fun with Algoriths, 1998, pp [12] G. Dantzig, Linear rograing and Extensions. rinceton University ress, [13] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin, Network Flows: Theory, Algoriths, and Applications. rentice-hall, [14] C. Gini, Variabilità e utabilità, in Meorie di etodologica statistica, E. izetti and T. Salveini, Eds. Libreria Eredi Virgilio Veschi, [15] K. Appleby, S. Fakhouri, L. Fong, G. Goldszidt, M. Kalantar, S. Krishnakuar, D. azel, J. ershing, and B. Rochwerger, Oceano SLA based anageent of a coputing utility, in roc. Int l Syposiu on Integrated Network Manageent, Seattle, WA, May 2001, pp [16] A. Chandra,. Goyal, and. Shenoy, Quantifying the benefits of resource ultiplexing in on-deand data centers, in First ACM Workshop on Algoriths and Architectures for Self-Managing Systes, San Diego, CA, Jun [17] J. L. Wolf,. S. Yu, and H. Shachinai, Disk load balancing for video-on-deand systes, ACM Multiedia Systes Journal, vol. 5, no. 6, [18] D. N. Serpanos, L. Georgiadis, and T. Bouloutas, Mpacking: A load and storage balancing algorith for distributed ultiedia servers, IEEE Transactions on Circuits and Systes for Video Technology, vol. 8, no. 1, February [19] A. Turgeon, Q. Snell, and M. Cleent, Application placeent using perforance surfaces, in roc. Ninth Int l Syposiu on High-erforance Distributed Coputing, ittsburgh, A, August 2000, pp [20] J. Kangasharju, J. Roberts, and K. W. Ross, Object replication strategies in content distribution networks, in roc. Sixth Int l Workshop on Web Content Caching and Distribution, Boston, MA, [21] S. Buchholz and T. Buchholz, Replica placeent in adaptive content distribution networks, in roc ACM syposiu on Applied Coputing, March [22] H. Shachnai and T. Tair, On two class-constrained versions of the ultiple knapsack proble, Algorithica, vol. 29, [23], Noah bagels - soe cobinatorial aspects, in Int l Conference on FUN with Algoriths, Isola d Elba, June [24] H. Kellerer, U. ferschy, and D. isinger, Knapsack robles. Springer Verlag, 2004.