USING VIRTUAL MACHINE REPLICATION FOR DYNAMIC CONFIGURATION OF MULTI-TIER INTERNET SERVICES Carlos Oliveira, Vinicius Petrucci, Orlando Loques Universidade Federal Fluminense Niterói, Brazil ABSTRACT In recent years, several Internet services have been developed targeting people that spend most of their time online performing many different activities, such as browsing the Web. Typically, those services in the Web, including e-commerce sites and social networks, adopt a multi-tier design consisting of application and database tiers. In practice, the high workload demands for such Internet services cannot be supported by a single physical machine. To address this issue, we present an implementation of a virtualized cluster architecture that includes mechanisms for managing both application and database tiers. The overall cluster performance is guaranteed by dynamic resource allocation. Specifically, we make use of replication mechanisms in the multiple tiers of the server system. This, in turn, allows a given Internet service workload to be expanded or contracted dynamically across multiple different physical machines. Experimental results using our implemented architecture show that Internet workloads can be effectively replicated and balanced among different servers and the time-varying workloads of different Internet services can be supported accordingly in the server cluster. KEYWORDS Multi-tier Internet applications; Virtual machine replication. 1. INTRODUCTION In recent years, we have seen an increase in demand for web content and services. As a result, several Internet services have been developed targeting those people. Most of those services adopt a multi-tier design, consisting of application and database tiers, as found in server systems such as e-commerce sites and social networks. In order to respond to such high demand, web content and service providers have built large data centers to support many different Internet services. Although there are high peak demands, the average server utilization remains low in a typical server system. For instance, the average resource utilization in traditional data centers range between 5-20% [J. Heo et al., 2009]. This leads to a tremendous waste in terms of capital employed and energy used. To increase resources utilization, server virtualization has been widely adopted in data centers around the world. Basically, virtualization allows physical servers to run multiple independent services, where each service runs on the top of a Virtual Machine (VM). A physical server can run one or more VMs that dynamically share the underlying hardware resources leading to an improved usage of the available resources. In this work we present an implementation of a typical virtualized server architecture where both application and database tiers are implemented using virtual machines (VMs). To manage the Internet services in our architecture, a layer of proxies is interposed between clients and servers to intercept requests generated by the web clients. These proxies are used to permit the load balancing in the different tiers. The overall server performance is guaranteed by a dynamic resource allocation. That is, as we replicate and consolidate VMs, both application and database tiers can be executed by different physical machines.
2. ARCHITECTURE AND IMPLEMENTATION We propose an implementation of a server cluster architecture that takes advantage of Apache proxy [The Apache Software Foundation, 2010a] and MySQL proxy [MySQL, 2010a] capabilities. Our architecture aims to provide mechanisms for replicating multi-tier Internet services in a cluster. This means that application and database tiers may have multiple services running on multiple physical servers. In practice, a running service can be a web server (Apache) or database server (MySQL). In our architecture, we consider that the services within application and database tiers are executed by means of VMs. The underlying idea is that these VMs can be dynamically added, removed or even dimensioned in the system. In other words, the available resources can be allocated/deallocated for both application and database tiers in the system. In the multi-tier architecture, a number of requests are generated by web clients and received by the frontend machine that directs it to the application proxy. Then, the request is sent to an application server. In our implementation, each application server runs on top of a VM containing a copy of a RuBiS PHP version [RUBiS, 2010], which simulates an ebay site. An application server does not directly access a database server. A database proxy is interposed between applications and databases servers running on top of VMs. Each database server contains a copy of a MySQL database that stores tables containing information about users and items available in the web site. This mean that the application server generates dynamic page content and query-language calls are intercepted by the database proxy that redirects it to a database server. The front-end layer runs both application and database proxies, performing low-latency functions such as request forwarding and load balancing. We assume that the physical machine in this layer has enough resources to perform its role and will not fail, although it could be replicated to scale and to achieve fault tolerance. We have implemented two specific modules in the front-end layer. The first, termed Monitor, constantly monitors the CPU utilization of the physical machines. The second, termed Manager, has a list of physical machines and VMs, and uses the information about CPU utilization, collected by the Monitor, to allocate/deallocate resources, i.e., to add or remove VMs, for both the application and database tiers. The Manager uses weights available in the Apache Proxy to balance the amount of work to be done by the servers in the application tier. In our implementation, the Manager acts as follows. When a physical machine achieves a given CPU utilization threshold, the Manager starts a new VM for the application or database, according to the bottlenecked tier. The Manager starts the VM in the physical machine that has the lowest CPU utilization in the cluster. If all active physical machines are overloaded, the Manager adds a new physical machine to the cluster. The deallocation occurs when the applications supported by the cluster can be supported by a smaller number of physical machines, maintaining the CPU utilization bellow a given threshold. At this point, it is also possible to use an optimization technique to adjust the capacity of individual VMs, as experimented with other works, e.g., [V.Petrucci et al., 2011]. 3. REPLICATION MECHANISMS 3.1 Replication in application tier In the application tier, we used the Apache HTTP Server [The Apache Software Foundation, 2010b] integrated with a module termed mod_proxy_balancer [The Apache Software Foundation, 2010b], which is an Apache extension for load balancing. The load-balancer runs in a front-end machine and redirects requests between the pool of VMs that act as back-end servers. Weights are attributed to each back-end server. To do this, Apache provides the variable lbfactor that is a normalized value representing their share of the amount of work to be done. When we need to replicate an application in a second back-end server, we start a VM replica at this server. At the moment this replica is ready for processing clients requests, its status on the load balancer manager is set
up to enable and it is attributed a weight for this VM replica. Note that the weight is incremented in a loop, in small units until it reaches the target weight. 3.2 Replication in database tier The replication in database tier is much more complex than in the application tier. In fact, there are several different database management systems. To simplify our task, we use the Master-Slave Replication [MySQL, 2010c], provided by MySQL. The Master-Slave Replication enables data from one MySQL database server (the master) to be replicated to one or more MySQL database servers (the slaves). Replication is based on the master server keeping track of all changes to its databases in its binary log. The binary log serves as a written record of all events that modify the database from the moment the server was started. Each slave that connects to the master requests a copy of the binary log. The slave also replays the events from the binary log that it receives. This has the effect of repeating the original changes just as they were made on the master. In this environment, all writes and updates must take place on the master server. Reads, however, may take place on one or more slaves. The load balancing in the database layer uses the MySQL Proxy, which is a module provided by MySQL that provides communication between one or more MySQL servers and one or more MySQL clients. Thus, it allows to direct applications requests to the proxy that schedules them between a pool of available application databases. The MySQL Proxy distributes connections from clients to each server. Distribution is handled by a count for the number of connections distributed to each server. New connections are automatically sent to the server with the lowest count. If MySQL Proxy identifies that the slave is lagging behind the master for its replication threads, then the slave is automatically taken out of the list of available servers. Work will therefore be distributed to other MySQL servers within the slave replication group. 4. EXPERIMENTAL EVALUATION To evaluate our architecture implementation, we have conducted an experiment aimed at managing a threetier (front-end/application/database) application using the RuBiS web benchmark [RUBiS, 2010]. The major goal is to use the proposed replication mechanisms to support a multi-tier application while handling a timevarying workload. In order to meet fair response time requirements, we shall perform VM replication before the physical machine saturates, dimensioned for playing safe as 300% of CPU utilization (the maximum value for utilization is 400% because we are using quad-core machines and each core represents 100%). This leaves an amount of 100% CPU capacity available to be used by the VM management domain (Dom0) on the physical servers during the replication activities. All machines share a NFS storage mounted in the front-end to store the VM images. 4.1 Workload generation and QoS measurement RuBiS is the application used in our experiment. Several recent research works took advantage of the RuBiS benchmark, e.g., authors in [Y. Chen et al., 2008] which used the tool to evaluate their proposal. However, one can notice that Rubis is restricted to a semi-static number of clients which can only be increased and decreased following a ramp up/down style during a given experiment. In order to add flexibility, we have extended the RuBiS benchmark tool allowing it to produce arbitrary and more realistic benchmarks for server systems. The QoS (quality-of-service) considered in this work is related to the response time from server side. The response time is defined by the difference between the time a response is generated and the moment the server has accepted the associated request. To obtain the response time for the web applications we have implemented a new Apache module that collects the time information between these two moments using predefined hooks provided by the Apache Module API [Nick Kew, 2007]. To smooth out high short-term
fluctuations in measurements readings, we have integrated a filter procedure in our Apache module based on a single exponential moving average [Engineeting Statistics Handbook, 2010]. 4.2 Management of Internet services We performed an experiment in which we vary the number of accesses to the web site leading to the necessity of expand or contract the service workload across multiple different physical machines to prevent degradation in the QoS perceived by the user. Specifically, in the experiment, we increased the workload in steps, thus resources were allocated to handle the incoming user requests. After the allocation of our four worker machines (Itaipu, Tucurui, Maxwell and Edison each of them an Intel Core i5 2.67GHz, 8GB RAM), the workload decreased in steps and resources were deallocated. Our goal is to use the replication mechanisms to scale the system maintaining the CPU utilization under a given threshold. It is shown, in Figure 1, the allocation of the physical machines, the number of clients and the response time during the experiment. The experiment starts with one load generator machine (Xingo - Intel Core i7 2.67GHz, 8GB RAM), one application VM (running on Itaipu) and one master database VM (running on Tucurui). At time=60s, the workload increased and Itaipu could not handle all requests to the application-tier. Thus, a new worker machine (Maxwell) was added to the cluster and the replication mechanism was used to start a VM replica for the application-tier and balance requests between the VMs running on Itaipu and Maxwell. To handle the increase in the amount of user requests, a slave database was started, at time=200s, on Maxwell since it had free CPU cycles. At this moment, all read requests were sent to the slave database while the write requests were sent to the master database, reducing the CPU utilization on the master database. To increase the number of client requests, at time=250s, we added a second load generator machine (Farad - Intel Core Duo 3.00GHz, 6GB RAM) and a new worker machine (Edison) to the cluster. Edison was used to run an application-tier VM and a slave database. Thus, at this moment, read requests were balanced between slave databases running on Maxwell and Edison. After the workload reach a peak of 52 clients at time around 320s, the workload decreased and resources were deallocated. The experiment finished at time=600s. We observed in this experiment that, using the replication mechanisms for both application and database tiers, a multi-tier Internet application can be supported by different physical servers while handling a timevarying workload. Note that for a large system, a single master can become a bottleneck. In the future, it would be interesting to change the MySQL Master-Slave replication to another database replication policy that supports multiple master databases. Figure 1: Physical machines allocation
5. RELATED WORK Mechanisms for resource allocation in virtualized environments are used by several recent works. [Y. Wang et al., 2010] use live migration between physical machines, which is a limitation when the application cannot be supported by a single machine. Other works replicate only state-less applications [D. Kusic et al., 2009], but most part of current web applications are content-aware applications. Some other proposals adopt replication mechanisms for a single tier of the application, such as [G. Soundararajan et al, 2006], that replicates only the database-tier. Unlike these works, our architecture includes mechanisms for managing both the application and database tiers. 6. CONCLUSION AND FUTURE WORK In this work, we have implemented a virtualized server architecture targeted for multi-tier Internet applications that includes mechanisms for replicating a multi-tier application across different physical servers in a cluster. Using these mechanisms, the workload can be balanced among the different servers in a cluster. For multi-tier applications we argue that it is necessary to apply mechanisms like these to permit the load balancing in the different tiers, in order to guarantee the overall server performance. As observed in [J. Heo et al., 2009], runtime re-allocation of memory among multiple VMs has not been widely studied. They found that as the memory allocation becomes smaller, the total CPU consumption goes up, mainly due to the extra paging activities by the guest OS. Thus, the investigation on how to dynamically allocate memory resources, poses a great challenge for future research. REFERENCES D. Kusic et al., 2009. Power and performance management of virtualized computing environments via lookahead control. Cluster Computing, Vol. 12, No. 1, pp. 1-15. Engineeting Statistics Handbook, 2010. Single Exponential Smoothing. http://www.itl.nist.gov/div898/handbook/pmc/section4/pmc431.htm G. Soundararajan et al, 2006. Database replication policies for dynamic content applications. EuroSys '06: Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006. Leuven, Belgium, pp.89-102. J. Heo et al., 2009. Memory overbooking and dynamic control of Xen virtual machines in consolidated environments. Proceedings of the 11th IFIP/IEEE international conference on Symposium on Integrated Network Management. New York, USA, pp. 630-637. Y. Wang et al., 2010. Power Optimization with Performance Assurance for Multi-tier Applications in Virtualized Data Centers. IEEE International Conference on Parallel Processing. Los Alamitos, CA, USA, pp. 512-519. Y. Chen et al., 2008. Translating Service Level Objectives to lower level policies for multi-tier services. Cluster Computing, Vol. 11, No. 3, pp. 299-311. MySQL, 2010a. MySQL Proxy. http://dev.mysql.com/doc/refman/5.0/en/mysql-proxy.html MySQL, 2010c. MySQL 5.1 Reference Manual: Chapter 16. Replication. http://dev.mysql.com/doc/refman/5.1/en/replication.html Nick Kew, 2007. The Apache Modules Book: Application Development with Apache. Prentice Hall, New Jersey, USA. RUBiS, 2010. RUBiS: Rice University Bidding System. http://rubis.ow2.org/ The Apache Software Foundation, 2010a. Apache Module mod_proxy_balancer. http://httpd.apache.org/docs/2.1/mod/mod_proxy_balancer.html The Apache Software Foundation, 2010b. Apache {HTTP} server version 2.2. http://httpd.apache.org/docs/2.2/ V.Petrucci et al., 2011. Optimized Management of Power and Performance for Virtualized Heterogeneous Server Clusters. 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid. California, USA.