Multi-Tiered On-Demand Resource Scheduling for VM-Based Data Center

Transcription

1 9th IEEE/ACM International Symposium on Cluster Computing and the Grid Multi-Tiered On-Demand Resource Scheduling for VM-Based Data Center Ying Song 1,2 Hui Wang 1 Yaqiong Li 1,2 Binquan Feng 1,2 Yuzhong Sun 1 1 ey Laboratory of Computer System and Architecture, Institute of Computing Technology, Chinese Academy of Sciences, Beijing; 2 Graduate University of Chinese Academy of Sciences, Beijing, China; {songying, wanghui, liyaq04, fengbinquan}@ncic.ac.cn, yuzhongsun@ict.ac.cn Abstract The trend of using virtualization for server consolidation is more and more popular in enterprise data center. However, ondemand resource allocation among the concurrent hosted services in such a virtualized environment is still a challenge. In order to optimize resource allocation among services in data center, this paper proposes a multi-tiered resource scheduling scheme which automatically provides on-demand capacities to the hosted services via resources flowing among VMs. We model the resource flowing using optimization theory. Based on this model, we present a global resource flowing algorithm in the multi-tiered resource scheduling scheme. This algorithm preferentially ensures performance of some critical services by degrading of others to some extent when resource competition arises. Using our RAINBOW prototype, we evaluate the multitiered resource scheduling scheme with the performance improvements for the most critical services up to 9%~16%, which are 75% of the maximum improvement margin, while performance degradation of others is up to 2%, and leads to 1%~5% improvements in resource utilization than RAINBOW without resource flowing. Compared with the existent scheme, our work leads to 9% less improvements for critical services, while introduces 39% less degradation to low priority services. 1. Introduction Virtualization offers opportunities not only to better isolation and manageability but also to on-demand, finer-grained resource provision. Thus, virtualization technology, such as virtual machine (VM), is ubiquitously used in data center for server consolidation. Improving resource utilization in such a multi-services sharing computing environment is a key technology to save power of data center to some extent [7]. However, nowadays, such enterprise data centers are often underutilized and idle-working even when workloads of some hosted services are high. On one hand, the barrier caused by the computer architecture and the operating system imposes restrictions on the improvement of resource utilization. Even multi-core could not break the barrier of upper bounds of speedup [12]. On the other hand, lack of efficient, on-demand and finegrained resource scheduler also limits the improvement of resource utilization. Based on the resource reallocating scheme provided by VMMs (Virtual Machine Monitor, i.e. Xen [15] and VMware [20]), many researchers [9][17] focus on improving resource utilization as well as guaranteeing quality of the hosted services via on-demand local resource scheduling models or algorithms within a physical server. However, most of them could not be good solutions to tradeoff between resource utilization and QoS. For example, Padala s controller [17] improves resource utilization and performance of some services by hugely reducing performance of others. How to improve resource utilization, as well as guarantee QoS, is a challenge in a VM-based data center. Our previous paper [23] also proposes a set of local resource scheduling algorithms, which improve the resource utilization as well as improve performance of some critical services with small performance degradation of others. Yet, local optimization could not always lead to global optimization [11]. It is necessary to provide a global resource scheduling in a shared computing environment. In this study, we design a multi-tiered resource scheduling scheme to ensure QoS as well as improve the resource utilization in our service computing framework - RAINBOW. We use resource flowing to denote the process in which resources released by some VMs/services are allocated to others. Based on our resource flowing model, we design a global resource flowing algorithm as a complement to the local resource flowing algorithms (proposed in [23]) to optimize the resource allocation in a VM-based data center. This algorithm preferentially ensures the performance of some critical services by degrading of others to some extent when the resource competition arises. We consider CPU and memory flowing which could be generally extended to other resources such as I/O. We implement a Xen-based RAINBOW prototype to evaluate our multitiered resource scheduling scheme on a workload scenario reflecting resource demands of services in a real enterprise environment. The experimental results show that RAINBOW with our resource flowing algorithms improves performance by 9%~16% for those critical services, while introducing up to 2% performance degradation to others, with 1%~5% improvements in resource utilization than RAINBOW without resource flowing. The performance improvements for the most critical service introduced by our work are up to 75% of the maximum improvement margin. Compared with Padala s work [17], our work leads to 9% less improvements for critical services (28% improvements introduced by [17] and 19% improvements caused by our work), while introducing 39% less degradation to others (41% and 2% degradation caused by [17] and our work, respectively). The results indicate that our work improves the resource utilization, and meets QoS goals of services. This paper has the following main contributions. 1) We present a multi-tiered resource scheduling scheme for VMbased data center. 2) We model the resource flowing using optimization theory and resolve it by the Simplex Method. 3) Based on the model, we present a global resource flowing algorithm in our multi-tiered resource scheduling scheme to optimize resource allocation among services. The rest of this paper is organized as follows. Section 2 introduces the motivation. In section 3, we discuss related work. Section 4 introduces our RAINBOW and multi-tiered resource scheduling scheme. Section 5 models the resource flowing. A global resource flowing algorithm is proposed in section 6. Section 7 discusses the implement of our prototype and the experimental results. We conclude in section Motivation In VM-based data center, services run over various capacities (i.e. computing, storage, and communication capacities, which are provided by physical components, such as CPU, memory and network bandwidth), ignoring the positions and architecture of physical components (i.e. multi cores and heterogeneous /09 $ IEEE DOI /CCGRID

2 components). Such ignorance is provided by virtualization, for example, most cloud computing infrastructures (i.e. Amazon EC3) use virtualization to provide isolated capacities to the hosted services. Dynamic load changes, as well as different QoS requirements, of services in their life give rise to diverse time-varying capacity demands. It is necessary to provide ondemand capacities to services via optimizing capacity flowing among those services. Such on-demand capacity flowing is implemented by fine-grained resource (i.e. CPU and memory) flowing among VMs. Figure.1 The evolution of resource management The VM-based resource management differs from previous work in the granularity (from nodes to components) and dimensions (from one to two) illustrated in figure1. Traditional resource management corresponds to the scheduler in figure 1(a), which dispatches jobs/requests onto a set of exclusively servers. In such a case, resources of some servers may be severely wasted even when the queues of jobs/requests on these servers are full, which results from the data dependency and resource competitions among these jobs/requests. As to the VM-based resource management (figure 1(b)), scheduler#1 corresponding to the traditional resource management dispatches jobs/requests onto a set of VMs. It adds a new dimensioned resource scheduler (scheduler#2) to optimize the usage of fine-grained resources via resource flowing among VMs when the resource utilizations are imbalance in these VMs no matter whether the queues of jobs/requests on these VMs are full or not. Most contemporary VMMs (i.e. Xen and VMware) provide partial technical support rather than strategy to the resource flowing. They need a better second dimensioned resource scheduler to optimize the usage of resources, and improve quality of the hosted services. Optimizing resource flowing among VMs is a key technology in such a platform. 3. Related Work Currently, a large body of papers is on managing data center that provides on-demand resources. Several studies provide on-demand resources at the granularity of physical/virtual servers. Oceano [10] dynamically allocates resources for an e-business computing utility. It focuses on sharing at the granularity of whole servers. SoftUDC [13] proposes a software-based utility data center that adopts the strategy of on-the-fly VM migration, which is also implemented by VMware s VMotion [14], to provide automatic load balancing. In [21], a virtual-appliance-based autonomic resource provisioning framework is provided. It dynamically allocates resources to applications via adding/removing VMs on physical servers. All these studies are in contrast to our scheme that controls resource flowing at the granularity of resource components, i.e. CPU time slots. There is a growing body of work on providing on-demand fine-grained resources in a VM-based data center [2][5][6][9] [17][20][23]. In [17], dynamic CPU allocation is done based on the VM utilization and application-level QoS metrics. But it only focuses on CPU reallocation and uses the fixed reallocation threshold according to the experience, while our scheme focuses on both CPU and memory flowing as well as automatically adjusts resource overload thresholds according to the time varying workloads of the hosted services. In [9], a two-level resource management system with local controllers at the VM level and a global controller at the server level is proposed. These local and global controllers only correspond to our locallevel scheduler. Our previous work [23] focuses on both local CPU and memory flowing to achieve better QoS and higher resource utilization using the fixed resource overload threshold according to our experience. In [2]&[6], the authors address dynamic resource allocation in multi-tier virtualized service hosting platforms. All the above works only focus on dynamic resource allocation among VMs within a server ignoring the resource optimization among services in the entire system. In this paper, we not only care about the local scheduling in a single server but also deal with the global scheduling to optimize resource allocation among the hosted services. In [22], the authors optimize global resource allocation for multi-tier services. But this optimization is central controlled, which has the problems of complexity (collecting and computing resource allocation to each VM hosted in every VMM), availability (the single point failure) and non-timeliness (the execution intervals (22min) could not be small because of the scalability). However, the hosted web-based services are interactive with sudden demands on resources. Such slow response on fine-grained resource allocation could not introduce the optimized allocation in realistic web-based workloads. In contrast, our work attempts to address the issue of global resource allocation using a multi-tiered resource scheduling scheme. The local scheduler with the simple function is working in small intervals (1s) in each server, which could fast respond to sudden resource demands by the hosted services. The global scheduler with the simple function is working in 1min/5min intervals as a complement to the local scheduler. All these schedulers work independently. Any scheduler s failure (even the global scheduler) could not lead to the failure of resource allocation in the system. To the best of our knowledge, no other studies proposed the same multi-tiered resource scheduling scheme and algorithms as ours to optimize fine-grained resource allocation in a VMbased data center. 4. Multi-Tiered Resource Scheduling Scheme Based on the service computing framework-rainbow proposed in [23], we present a multi-tiered resource scheduling scheme to optimize resource allocation in a VM-based data center. 4.1 RAINBOW Statement In RAINBOW (illustrated in figure 2), a set of VMs serving a particular service is called a group. The key principle is that VMs belonging to a single group are spread across multiple servers, while each server hosts VMs belonging to different groups. This principle aims to reduce the competitions for resources by the hosted services in a server. 149

3 The global-level scheduler controls the resource flowing among services via adjusting activity of each service. In RAINBOW, multiple copies of each service encapsulated in VMs are split onto multiple servers, namely, the service can use resources in these servers. Adjusting activities of services effects the resource allocation among VMs hosting these services on each physical server, which results in resource flowing among services. Figure.2 Service computing framework - RAINBOW RAINBOW is divided into three layers: the service layer, the virtual resource layer, and the physical resource layer. In the service layer, each service, which is allocated a priority denoting how critical the service is, dispatches workloads to various VMs in its group according to its scheduling algorithms. In order to provide capacities to the hosted services ondemand, we control virtual resource flowing among VMs in the virtual resource layer. Such virtual resource flowing is implement by a set of physical resource flowing ( resource flowing for short) algorithms taking the priorities of the hosted services into account in the physical resource layer. For the purpose of controlling the physical resource flowing, we propose a multitiered resource scheduling scheme. 4.2 Multi-Tiered Resource Scheduling Scheme Figure.3 Three tiered logical flowing Logically, there are three correlated capacity flowing tiers (illustrated in figure 3) in RAINBOW: capacity flowing 1) among services (Tier#1); 2) within a service group (Tier#2); 3) among VMs (Tier#3). These logical tiers correspond to two implemental tiers: resource flowing 1) among VMs within a server (local) and 2) among VMs residing in different servers (global). However, there is no technological support on such global resource flowing. Local optimization could not always lead to global optimization [11]. In RAINBOW, VMs devoting to the same service are located in different physical servers. The local resource flowing in each server leads to independent capacity allocation to these VMs, and could not optimize capacity allocation among the concurrent services. Thus, we provide a multi-tiered resource scheduling scheme illustrated in figure 4 to optimize capacity allocation not only among VMs within a server but also among services. In this scheme, there are three tiers correlative schedulers: the application-level scheduler, the local-level scheduler and the global-level scheduler. The application-level scheduler is implemented by service software to dispatch requests/jobs onto VMs hosting this service. How to design an application-level scheduler is not within the scope of this paper. The local-level scheduler controls the resource flowing among VMs within a server taking the priority, threshold of resource overload (activity we called in this manuscript) of each service and resource utilization of each VM into account, which is introduced in detail in [23]. Figure.4 Three-tiered scheduling scheme In the above schedulers, the key work is to design the resource flowing algorithms. Resource flowing algorithms should solve four problems. 1) Which resources will flow? 2) When will such resources flow? 3) Which VMs will be the source and target of flow? 4) How many resources will flow? This paper focuses on the global resource flowing algorithm. In order to answer these four problems in the global resource flowing algorithm, we model the resource flowing first. 5. The Resource Flowing Model We consider the global resource flowing among services and model it by optimization theory. This model is a general one which can be respectively used by CPU or memory or other resources. Based on this model, in section 6 we present a global resource flowing algorithm which provides on-demand resources to the hosted services. First we introduce the following notations and concepts: R - The total CPU or memory or other resources, which are available to all services, such as 16GB memory. - The number of services hosted in the data center. R out - The resources allocated to all the VMs. C i-min - The minimum threshold of resources allocated to VMs hosting service i, which is used to avoid huge interaction among the services when competition for resources arises. C i- min is set by experience in our experiments, and will be justified in the near future. R it - Resources allocated to VMs hosting service i at time R R it t, which must obey the rules: i= 1 and R it C i-min >0. D it - Resources demanded by VMs hosting service i at time t, which are proportional to the arrival rates of requests. P i - The priority of service i. It indicates how critical the requirement for QoS of this service. If i<j, P i P j. P i is determined by administrator. Q it - The quality of service i at time t, which is the performance metric. The smaller the Q it is, the better QoS the service gains. As we all know, QoS of a service (Q it ), such as response time, is decided by resources demanded by it (D it ) and resources allocated to it (R it ). In other words, Q it is a function of D it and of R it, namely, Q it =f i (R it,d it ). Ф i - The acceptable quality of service i. For example, 3 seconds for response time of web service. In order to fairly weight different services using their QoS, we use QoS-rate (the rate of quality of service i at time t and of the acceptable quality of such service, Q it /Ф i ). 150

4 The goal of resource flowing is to optimize quality of the hosted services taking their priorities into account, giving the limited resources. It is an optimization problem with limiting conditions. Thus, we select the programming model of optimization theory to model the resource flowing. To provide the resource flowing with a utility function [1][23], which maps QoS of the target to a benefit value, we define the utility function UF t as follows. UF t is related to the priorities and QoSrates of the hosted services. Qit fi ( Rit, Dit ) UF t = Pi = Pi (1) i= 1 Φi i= 1 Φi The global resource flowing problem is how to control resource allocation among services with the goal of minimizing the system utility function UF t, giving the limited resources, which is formulated as follows: f i ( Rit, Dit ) minuf t = Pi i=1 Φ i (2) Rit R s. t. i= 1 Rit Ci min ( i = 1,2,, ) Before solving this model, we consider the function Q it =f i (R it,d it ). When R it is no smaller than D it, the Q it is a constant. When R it is smaller than D it, some requests will wait for resources in queue. With plentiful experiments, we extract such functions of several typical enterprise services including web and database (DB for short) services, and of different resources, i.e. CPU and memory. Using httperf [25] as a generator of web workloads, we can measure the relationship among the request number (req), the required CPU/memory resources (D it, 100% CPU denotes one CPU core of 2GHz, mem_n denotes nmb memory) and the allocated CPU/memory resources (illustrated in figures 5&6). response time (s) request number 40% CPU 60% CPU 80% CPU 100% CPU response time (ms) rate(reqs/s) mem_64 mem_256 mem_320 mem_384 mem_512 mem_640 mem_768 mem_896 mem_1024 mem_1536 Figure.5 QoS of web service Figure.6 QoS of web service impacted by CPU impacted by memory Each line in figure 5 denotes service performance (y axis) degradation with the increasing of workloads (x axis) using a fixed CPU resource. In figure 5, we select an acceptable response time (5s) and draw a horizon line with the selected value in the y axis. The junction points of the horizon line and these curve lines correspond to the relationship between the request number and the required CPU with the acceptable response time. Thus, these curve lines show the relationship among the response time, the allocated CPU resources, and the required CPU resources, namely, Q it =f i (R it,d it ). The four lines show the similar trend of alteration. This trend can be divided into four stages: no-alteration, slow-alteration, fast-alteration, and oscillation. At the no-alteration stage, resources provided by system are more than those demanded by workloads. No resource competition arises, which results in no performance impairment. With the increasing of workloads, resources provided by system could not completely satisfy demands of workloads at the slow-alteration stage. At this stage, some resources may be the bottleneck occasionally, which leads to tiny performance impairment. At the fast-alteration stage, competitions for resources arise frequently, so that service performance is greatly impaired. When workloads are beyond limits of the available resources, namely, when workloads are at the oscillation stage, some resources are severely overloaded, which results in the oscillation of service performance. Based on the above analysis, we sum up the function Q it =f i (R it,d it ) of web service using CPU/memory as follows. 0.4 ~ 1.0 D R (3) 0.138D 0.14R R < D 1.5R Q web cpu = 1.55D 2.2R R < D 1.5R 2.5 no sense 1.5R 2.5 < D D R (4) D R R < D R Q web mem = D R R < D R R R < D Figures 7&8 illustrate the experimental results about the relationships of DB service, and of the CPU and memory resources. We use TPC-W [4] as the DB workloads (denoted by EB) generator. Based on these results, we extract functions Q it =f i (R it,d it ) of the service (formula (5)~(6)) using the same method as above. WIPS 100%CPU 200%CPU 300%CPU 400%CPU EBs number Figure.7 QoS of DB service impacted by CPU WIPS D + 22 D R Q cpu = 0.03D 0.04R + 16 R < D R no sense R + < D DB Q EBs 7 R D mem = no sense R < D DB mem_1200 mem_500 mem_ EBs number Figure.8 QoS of DB service impacted by memory / (6),EB=0.4D+160(5) Based on the above analysis, we extract functions f i (R it,d it ), which are denoted by f ij (R it,d it )=a ij D it +b ij R it +g ij, U i(j-1) <D it U ij where f ij (R it,d it ) denotes the j th stage of function f i (R it,d it ), a ij,b ij,g ij refer to the coefficients in function f ij (R it,d it ), and U ij denotes the upper threshold of resources for service i at stage j. The functions of other services could be easily extracted either using the same method as mentioned above or using the following on-line machine learning method. The global scheduler extracts initial and temporary functions at the very beginning using the collected relationship among R it, D it and Q it. Then it randomly collects such information several (i.e. 6) times each hour, and modifies these functions with the increase in the amount of the collected information. This process may last 24 hours or longer. This on-line method takes N (i.e. 6*24) times collecting information and modifying functions, which is linear complexity. Putting these functions into our model, we get the following formulation. m a ij * Dit + bij * Rit + g ij j= 1 minuf t = Pi i= 1 Φ i R R (7) it s. t. i= 1 Rit Ci min ( i = 1,2,, ) where m denotes the maximum number of stages for f i (R it,d it ). We resolve the above model to get the close-to-optimal resource allocation R it at time t. If D R, R it =D it ; else, the process of resolving the model using the Simplex Method is as figure

5 R out = Ci min i= 1 Figure.9 The solution of the resource flowing model 6. The Global Resource Flowing Algorithm The goal of the global resource flowing model is to optimize resource allocation among services. Based on this model, we propose a global resource flowing algorithm (GRFA, illustrated in figure 10) as a complement to the local resource flowing algorithms (LRFaVM for short, proposed in [23]). The local resource flowing algorithms dynamically optimize resource allocation among VMs within a server via adjusting CPU time slots and memory assigned to each VM, according to the resource utilization, QoS and activity of service encapsulated in the VM. The key parameters are activities of the hosted services. They are the bridges between the local and the global resource flowing algorithms. Global resource flowing algorithm optimizes the resource flowing among services via periodically adjusting the activity of each service based on the resolution of the resource flowing model. Figure.10 Global resource flowing algorithm req it denotes the requests number of service i at time t. Using the predicted requests number, GRFA computes the optimal amount of resources which should be allocated to service i at time t (denoted by O it ) according to the resolution of the resource flowing model. Then GRFA compares resources allocated to each service (R it ) with O it to judge whether it needs to adjust the activity of service i or not at time t. In most cases, R it does not equal O it, so that we define a resource threshold (Ψ), which actuates the adjustment of the activities of services, to avoid frequent adjusting the activities of services. Only when the difference between R it and O it exceeds Ψ, GRFA adjusts activity of service i at time t. The efficacy of our algorithm is intimately dependent on how well it can predict the number of arrival requests. While it is certainly tempting to try sophisticated prediction techniques, we take a simple low-overhead last-value-like prediction as reference [19] does, in which the number of arrival requests during the last interval is used as a predictor in the next interval. Differing from the local resource flowing algorithms, the global resource flowing algorithm is invoked every longer intervals (i.e. 1min or even longer). Global flowing algorithm is not the bottleneck even in a large-scale computing environment for its simple functions and low working frequency. 7. Implementation and Performance Evaluation 7.1 The Implementation Details and the Platform The implementation of RAINBOW prototype (illustrated in figure 4) is based on Xen, which is also available to other VMMs. It requires: 1) a monitor module in each VM, which periodically collects the real-time VM utilization and delivers such information to the local scheduler in the resident VMM; 2) an local scheduler module, which is invoked every T i intervals to implement local resource flowing algorithms and reports the resource allocation information to the global scheduler every T j intervals; 3) a global scheduler module, which collects information from every local scheduler about every hosted VM and then returns activity information back to each local scheduler. We use 9 servers in our experiments. In the server pool of our prototype, there are 4 servers each of which has two 2190MHz Dual Core AMD Opteron(tm) processors with 1024B of cache and 4GB of RAM. We use CentOS4.4, one of the Linux distribution, and Xen We use other one server to execute the application-level and the global-level schedulers. The rest 4 servers are used to be clients of services. The systems are connected with a Gigabit Ethernet. 7.2 Benchmark Application Experiments Using our prototype, we conduct a set of experiments to evaluate our multi-tiered resource scheduling scheme. There are three typical enterprise services concurrently running in our environment: the web, DB and office services. Each service encapsulates its copies into four VMs, which are distributed on the four servers. 1) Web service: Apache [18] is used for the web server. LVS [26] dispatches requests among the web VMs using round robin (RR) algorithm. SPECWeb2005 [27] is used to generate e-commerce workloads. The number of requests with good response (respond within 3 seconds) defined by SPECWeb is the performance metric. 2) DB service: MySQL [16] and Apache-tomcat [24] are used for the DB server. LVS is used to dispatch requests using RR algorithm. We use TPC-W [4] as the DB workloads generator and the size of the DB files is 2.7GB. The DB service is evaluated by the average WIPS (the number of Web Interactions Per Second). 3) Office service: It is provided by our office applications virtualization product (illustrated in figure 11). It separates display from the running of applications based on virtual desktop, a trend of enterprise office applications, and it dispatches applications to servers/vms according to utilization of these servers. We use Xnee [3] to emulate the real-world office applications based on the trace collected by [8]. Response time, including starting up time of an application and executing time of an operation, is the key metric to evaluate its performance. Starting up time of an application is sensitive to both CPU and memory. Thus, we capture the starting up time of eight typical applications (Emacs, FTP, gedit, gqview, mozilla, openoffice, acrobat reader, QT designer) for 4 times by the modified VNC. The average starting up time of all the applications is used to be the performance metric to avoid randomicity. 152

6 On each physical server, we create 3 VMs. We initially allocate 700MB memory and 1VCPU to each VM. To ensure that CPU is the bottlenecked resource when services reach their peaks of workloads, we pin all the VCPU of the 3 VMs running on the same server to the same physical core. The rest three cores are used by domain 0. Figure.11 Applications virtualization Based on Xen, our scheme optimizes resource flowing which is not provided by VMMs (i.e. Xen). In order to fairly compare the system performance without manual control, the baseline system RAINBOW without resource flowing ( No- RF for short) provides fixed resources (700MB memory, 1VCPU) to each VM. We initialize the priority P DB :P office :P web as 4:2:1. Initially, we set the activities of these services as follows: 50MB idle memory as the threshold of memory overload, 90% as the threshold of CPU overload (T u ), and 70% as the desired CPU utilization level (T d ) for the individual VMs. All these activities are dynamically adjusted by our global resource flowing algorithm. For the global resource flowing algorithm, we set the thresholds of adjusting activities of VMs (Ψ), which are not the key parameters, to be 20 for CPU (20%CPU) and 100MB for memory via experience of our experiments. We set ΔA to be 5% for CPU and 5MB for memory. Local scheduler collects CPU utilization and idle memory of the hosted VMs as the feedback every second and every 5 seconds, respectively. Global scheduler is invoked and collects resource allocation information every 30 seconds. 7.3 The Experiments Results and Analysis In order to evaluate our multi-tiered resource scheduling scheme, we compare it with No-RF using the same timevarying workloads. LRFaVM refers to RAINBOW adopting local resource flowing algorithms. GRFA uses global resource flowing algorithm based on LRFaVM. Table 1. Multi-tiered resource scheduling vs. No-RF in prototype hosting database, office and web services Cases DB office web CPU utility LRFaVM 6% GRFA 9%(75% of the max margin) Mem utility 12% 1 10% -2% 1% 5% 16% -1% 0.3% 1% Table 2. Local resource flowing vs. No-RF in prototype hosting office, web and HPC services Case Office web HPC CPU utility Mem utility LRFaVM 42% 2% 1% 2% 6% Table 1 illustrates the performance comparisons of different cases in our scheme and No-RF. From this table we can see that LRFaVM provides great improvements (6% and 16%) in performance of the DB and office services, while introducing small degradation (1%) in performance of the web service. Compared with LRFaVM, GRFA further achieves 3% improvements, namely 9% improvements compared with No-RF, in performance of DB service, by reducing the rate of performance improvements in the office service (10% improvements) as well as slightly degrading the performance of web service (2%). Compared with No-RF, the maximum improvement margin for DB service is 12% in the case that all resources shared by all services are exclusively allocated to this single service 1. Thus, the performance improvements for the DB service are up to 75% of this maximum improvement margin. In another experiment [23] in our prototype hosting office, web and HPC services, LRFaVM provides 42% improvements in performance of the most critical service, while introducing no degradation to others (illustrated in table 2). Thus, the improvements provided by resource flowing algorithms are influenced by the hosted services and their workloads. Table 1 also shows that our work causes 1%~5% improvements in resource utilization. These results imply that our work utilizes the hardware more rationally based on the service differentiation. We analyze the reason why GRFA further improves performance of the most critical service (DB) by slightly impairing of others compared with LRFaVM using figures 12~18. GRFA coordinates resource flowing among services taking the resource allocation and service workloads into account, via adjusting activities of these services. Figure 12 illustrates the CPU utilization of each VM in the case of No-RF, reflecting the CPU requirements by workloads of each service. Figures 17 & 18 illustrate the changing of activities of the hosted services, denoted by thresholds of CPU and memory overload, using GRFA. Higher activity of a service means lower threshold of CPU overload or higher threshold of memory overload of the service. Resource allocation among services is in correspondence with activities of the services. Figures 13&15 illustrate CPU allocated to each VM adopting LRFaVM and GRFA respectively. In the first 600 seconds (600s for short), CPU capacities required by DB service are more than those required by other services (figure 12). GRFA adjusts the activities of DB service to be higher than those of others (figure 17). Thus GRFA allocates more CPU cycles to the DB VM than LRFaVM does (figures 13&15). From 3600s to 5300s, CPU capacities required by office service are often more than those required by others (figure 12). In such period, the activities of office service are higher than those of others using GRFA (figure 17). Thus, GRFA allocates more CPU cycles to the office VM than LRFaVM does (figures 13&15). From 2500s to 3600s, from 5300s to 6900s, and after 8000s, CPU capacities required by DB service are more than those required by others (figure 12). The activities of DB service are higher than those of others using GRFA (figure 17). Within such periods, GRFA allocates more CPU cycles to the DB VM than LRFaVM does (figures 13&15). Figures 14&16 illustrate memory allocated to VMs (service_alloc) adopting LRFaVM and GRFA respectively, according to memory used by VMs (service_used). From 500s to 3000s, memory required by DB and office services is less than that required by web service (figure 16). Thus, in figure 18 the activities of DB and office services are very low, while the activities of web service are increasing with the increasing of web workloads. In such period, GRFA allocates more memory to the web VM than LRFaVM does (figures 14&16). From 3000s to 6000s, memory required by office service is more than that required by other services (figure 16). Thus, the activities of office service increase (figure 18), resulting in more memory allocated to the office VM by GRFA than that by LRFaVM (figures 14&16). We select a period signed with blue real rectangle to analyze the process of memory flowing in figure 16. In the magnified part of the process signed with blue dashed rectangle, at the beginning of this part memory used by the office VM is close to memory allocated to it, GRFA controls memory flow from the web VM to the office VM. With the increase of office workloads, GRFA continues controlling 153

7 such flow. At the end of this part, the web workloads exceed the office workloads, GRFA controls memory flow from the office VM to the web VM. The interval of resource flowing depends on workload distributions of the hosted services. It ranges from 1s to 1116s for CPU, and from 5s to 965s for memory in our experiments. Our work leads to the overhead of learning from the feedback and controlling resource flow. Learning from the feedback leads to 0.2% memory (0.2%*4G=8M) and 0% CPU overhead. Controlling resource flow leads to 0%~0.3% CPU and 0% memory overhead per resource flowing. Such overhead is inappreciable to the system. We compare our multi-tiered resource scheduling ( RAIN- BOW for short) with [17] in table 3. We compute the average performance improvements and degradation of services introduced by [17] according to its figures 14&15. Reference [17] provides about 28% improvements in performance of the critical service while introducing about 41% degradation to another service. Although the total improvements provided by [17] (28%) are slightly more than those provided by RAINBOW (19%=9%+10%), the total degradation introduced by [17] (41%) is much more than that introduced by RAINBOW (2%). We can not evaluate such algorithms only by the performance improvements in the service differentiation scheme. The performance degradation should also be considered. Thus the margin between the total performance improvements and the total performance degradation provided by scheduling algorithms ( improv-degrad for short) is used to evaluate them in this paper. Table 3 shows that the improv-degrad value of [17] (-13%) is much smaller than that of our work (17%), which implies that our work is better than [17] in the aspect of assuring QoS to some extent. What are the key differences between [17] and RAINBOW which accounts for RAINBOW's better performance? Both schemes focus on resource scheduling based on the service differentiation using similar service workloads. But [17] only focuses on CPU allocation among VMs within a server and uses fixed overload threshold according to experience, while RAINBOW focuses on both CPU and memory flowing not only among VMs within a server but also among services, as well as automatically adjusts overload threshold according to the time varying workloads of the hosted services. The working intervals of RAINBOW are 1s for CPU and 5s for memory, which are much smaller than that of [17] (10s). This implies that RAINBOW has faster response to the change of resource demands by services. The hosted web-based services are interactive with sudden demands on resources. Such faster response results in better service performance too. Table 3. RAINBOW vs. reference [17] resources working interval threshold improvements degradation improv-degrad Ref[17] CPU 10s Fixed 28% 41% -13%(=28%-41%) RAINBOW CPU&mem 1s(CPU), 5s(memory) Auto adjusted 19% 2% 17% Figure.12 CPU utilization of VMs in No-RF Figure.13 CPU allocated to VMs using LRFaVM Figure.14 Memory allocated to and used by VMs using LRFaVM Figure.15 CPU allocated to VMs using GRFA 154

8 Figure.16 Memory allocated to and used by VMs using GRFA Figure.17 CPU activities of the hosted services using GRFA 8. Conclusions This paper proposes a multi-tiered resource scheduling scheme to optimize resource flowing in the VM-based data center. In this scheme we design an on-demand global scheduling algorithm to control resource flowing among services. We implement a Xen-based computing platform-rainbow to evaluate our multi-tiered resource scheduling scheme. The experimental results show that our local resource flowing algorithms provide great improvements (6% and 16%) in performance of the critical services, while introducing small degradation (1%) in performance of others. Compared with these local resource flowing algorithms, our global resource flowing algorithm further achieves 3% improvements in performance of the critical service, which are 75% of the maximum improvement margin, by slightly reducing the rate of performance improvements in other services, with 1%~5% improvements in resource utilization. Compared with [17] our scheme is better in the aspect of assuring QoS. These results confirm that our multi-tiered resource scheduling scheme gains its goal of optimizing resource flowing among services to improve resource utilization, as well as guarantee QoS based on service differentiation with inappreciable implementation overheads. In the future, we will analyze the potential and overhead caused by each tier of the multi-tiered resource scheduling. Acknowledgments This work was supported in part by the National High-Tech Research and Development Program (863) of China under grants 2007AA01Z119, 2009AA01Z141, and 2009AA01Z151, and the projects of NSFC under grants References [1] D.A.Menascé, etc., Autonomic Virtualized Environments, International Conference on Autonomic and Autonomous Systems, 2006,p.28. [2] G.Jung, etc., Generating Adaptation Policies for Multi-Tier Applications in Consolidated Server Environments, ICAC08, p [3] H.Sandklef, Testing applications with Xnee, Linux Journal, vol.2004, no.117, Jan 2004, p.5. [4] H.W.Cain, R.Rajwar, M.Marden, etc., An architectural evaluation of java TPC-W, HPCA, 2001, p [5] IBM Redbook: Advanced POWER Virtualization on IBM System p5: Introduction and Configuration, Jan Figure.18 Memory activities of the hosted services using GRFA [6] I.Cunha, J.Almeida, etc., Self-adaptive capacity management for multitier virtualized environments, IM 07, p , [7] J.S.Chase, etc., Managing Energy and Server Resources in Hosting Centers, SOSP2001, p [8] J.Wang, Y.Sun, etc., Analysis on Resource Utilization Patterns of Office Computer, The IASTED International Conference on Parallel and Distributed Computing and Systems, 2005, p [9] J.Xu, M.Zhao, etc., On the Use of Fuzzy Modeling in Virtualized Data Center Management, Fourth International Conference on Autonomic Computing (ICAC07), p.25. [10].Appleby, S.Fakhouri, etc., Oceano-SLA based management of a computing utility, Proceedings of the IFIP/IEEE International Symposium on Integrated Network Management, 2001, p [11] L.S. Lasdon, Optimization theory for large systems, Courier Dover Publications, 2002, ISBN ,p.4. [12] M.D.Hill, etc., Amdahl s Law in the Multicore Era, IEEE Computer, July 2008, p [13] M. allahalla, etc., SoftUDC: A Software-Based Data Center for Utility Computing, IEEE Computer Society, Nov 2004, p [14] M.Nelson, etc., Fast Transparent Migration for Virutal Machines,USENIX Annual Technical Conference,2005, p [15] P.Barham, B.Dragovic,.Fraser, etc., Xen and the art of virtualization, SOSP, 2003, p [16] P.Dubois, MySQL, NewRiders, ISBN , Dec [17] P.Padala, X.Zhu, etc., Adaptive Control of Virtualized Resources in Utility Computing Environments, EuroSys 07, p [18] R.T.Fielding, G.aiser, The Apache HTTP Server Project, IEEE Internet Computing, vol.1, no.4, July 1997, p [19] S.Govindan, A.R.Nath, A.Das, Xen and Co.: Communication- aware CPU scheduling for consolidated Xen-based hosting platforms, VEE, 2007, p [20] VMware Infrastructure: Resource Management with VMware DRS. [21] X.Wang, etc., Appliance-based Autonomic Provisioning Framework for Virtualized Outsourcing Data Center, ICAC07, p.29. [22] X.Wang, etc., A Resource Management Framework for Multi-tier Service Delivery in Autonomic Virtualized Environments, IEEE Network Operations and Management Symposium, 2008, p [23] Y.Song, etc., A Service-Oriented Priority-Based Resource Scheduling Scheme for Virtualized Utility Computing, International Conference on High Performance Computing (HiPC), 2008, p [24] [25] [26] [27] 155