Scalable Load Balancing for Large-scale Web Server Clusters

Journal of Computational Information Systems 0: 3 (204) 5763 577 Available at http://www.jofcis.com Scalable Load Balancing for Large-scale Web Server Clusters Chuibi HUANG,, Jinlin WANG 2, Gang WU, Jun CHEN 2 Department of Automation, University of Science and Technology of China, Key Laboratory of Network Communication System and Control, Hefei 230000, China 2 Institute of Acoustics, Chinese Academy of Sciences, National Network New Media Engineering Research Center, Beijing 00000, China Abstract Load balancing is a fundamental technology to provide scalability for web server clusters. But in the scenarios where the web server clusters have massive amount of user accesses, a single load balancing server is unable to carry the user visits. Therefore load balancing tasks should be undertaken among a set of load balancing servers. Correspondingly the loads among these servers also need to be balanced. In this paper, we proposed a load balancing scheme that could provide scalability, availability and reliability for large-scale web server clusters. The scheme combines existing routing techniques and splits the function of load balancing into three levels that could be implemented by different devices, thereby enabling the web server cluster scales with user accesses. The experiment results have demonstrated the effectiveness of the program. Keywords: Load Balancing; Scalable; Web Server Clusters Introduction Load balancing is a process of distributing client requests among a plurality of web servers. The goals of load balancing may include improving resource utilization, maximizing throughput, minimizing response time, and preventing the server from becoming overloaded. In addition, balancing the loads among multiple web servers may increase reliability through redundancy. A server that performs the load balancing according to a scheuling algorithm is referred to as a load balancer. The load balancer is provided as a gateway to serveral redundant servers typically located in a single geographical location and referred to as server cluster. In general, DNS servers store the IP address of the load balancer rather than the addresses of the web servers to which the load balancer is directly connected. The load balancer s address is called virtual IP Project supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (No. XDA0600302) and the National High Technology Research and Development Program of China (No. 20AA0A02). Corresponding author. Email address: hcb@mail.ustc.edu.cn (Chuibi HUANG). 553 905 / Copyright 204 Binary Information Press DOI: 0.2733/jcis69 July, 204

5764 C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 address by which it masks the addresses of the web servers. The load balancer receives client requests and sends the requests to a web server based on the scheduling algorithm. Because of the rapid development of e-commerce and social networks, the most popular web server clusters generally have massive amount of user accesses. In these scenarios, it is impossible for a single load balancer to carry such a large amount of user accesses. Therefore load balancing tasks should be undertaked among a set of load balancing servers. Correspondingly the loads among these servers also need to be balanced. In this paper, we proposed a load balancing scheme to solve the above problem. The scheme achives scalability and reliability via a novel division of data plane functionality into three levels. As shown in Fig., at the topmost level, routers distribute Fig. : Levels of data plane the loads at the network layer based on the multi path routing protocol. It means that for a VIP, there are multiple paths, each of which is assigned a weight caculated by the mean-variance model, to the destination. At the second level, there are a scalable set of second-layer servers for load balancing, called second-level load balancer. These load balancers are responsible for maintaining connection state in memory and doing transport layer load distribution to thirdlayer load balancer. At the bottom layer, stateful network address translation (NAT) fuctionality is provided by the third-level load balancer. The main advantages of this scheme are: Spliting the function of load balancing into three level that are implemented by different devices, thereby enabling the web server clusters scale with user accesses. Providing reliability and increasing the link bandwidth by making use of multi path protocol. Enabling the oubound load balancing traffic to bypass the second-layer load balancer and go direct to the gateway routers, thereby eliminating throughput bottleneck and reducing latency. The organization of the paper is as follows. In the next section, we review the related work of our research. In Section 3, we explain our model and scalable load balancing scheme. Experiment results are presented in Section 4. Finally, we conclude our paper in Section 5.

C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 5765 2 Related Work Load balancing is a key technology to provide scalability and availiability for web server clusters. The load balancing solutions can be classifield as content-blind load balancing and content-aware load balancing. The content-blind balancer is unawre of the application infromation contained in incoming request. There are several load balancing scheduling policies that are normally used by content-blind load balancers: Round Robin (RR), Weighted Round Robin (WRR), Least Connection (LC), Weighted Least-Connection (WLC), Least Loaded (LL) and Random Server Selection []. A content-aware load balancer works at application layer. This means that the load blancer could receive the http request and analyse its content. This makes the content-aware load balancing more specific to applications so that the load balancer can provide differentiated services. However it is more complex for the the content-aware load blancer to make decisions. Aron et al. [2] proposed TCP connection binding to permits a content-based distribution at the granularity of individual requests because the persistent connections between the load balancer and the back-end servers do not depend on the incoming traffic. Luo et al. [3] proposed a method to improve the distribution of user visit and the reliability of the TCP connection binding mechanism. Maltz and Bhagwa et al. [4] proposed TCP splicing to splice the TCP client connection and the TCP server connection. Therefore all the work can be carried out by sending the data at the network layer. Several locality-aware solutions have been proposed to utilize the cache locality of the Web servers by sending the requests that ask for a specific Web page to a web server that is likely to cache it in memory [5 8]. Non locality-aware solutions pay more attention to other factors, for instance, the resource utilisation [9, 0], the QoS in the web service [, 2, 4], the combination of dynamic and static load balancing strategy [7], or the inclusion of an admission control in the load balancing solution [9]. Patel et al. [5] make use of distributed technology to provide scalability for data center. Casalicchio et al. [3] proposed Client-Aware Policy (CAP) to obtains good performance when providing dynamic Web content. From the literature, few exiting researches focus on the the scenarios where the web server clusters have massive amount of user access and a single load balancing server is unable to carry the user visits. 3 Model and Scheme 3. Mean-variance model The mean-variance model solves the problem of optimal portfolio selection for an individual. Portfolio theory [6] explores how risk-averse participants construct portfolios in order to minimize the market risk for a given level of expected return. Suppose there are N risky assets, whose rates of returns are given by the random variables R, R 2,, R N. Let w = (w,, w N ) T, w n denotes the proportion of wealth invested in asset n, with N n= w n =. The rate of return of the portfolio is R P = w n R n. () n=

5766 C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 The expected return and market risk are defined as: σ 2 P = var(r P ) = µ P = E[R P ] = E[w n R n ] = n= w n µ n. (2) n= w i w j cov(r i, R j ) = w i σ ij w j. (3) where µ n is the expected return of assect n, and σ ij is the covariance of R i and R j. The meanvariance model gets the optimal weight vector w by solving the following constrained optimal problem: minimize w i w j σ ij. (4) 2 subject to To solve the above optimized problem, we form the lagrangian: w i Ri = µ P. (5) w i =. (6) L = 2 w i w j σ ij λ ( w i ) λ 2 ( w i Ri µ P ), (7) where λ and λ 2 are lagrangian multipliers. We then get the optimal weight vector w by differentiating L with respect to w i and the Lagrangian mul-tipliers, and setting the derivative to zero: L = σ ij w j λ λ 2 Ri = 0, i =, 2,, N. (8) w i L λ = L λ 2 = 3.2 Scheme and architecture w i = 0. (9) w i Ri µ P = 0. (0) Our scheme combines existing routing techniques and splits the function of load balancing into three level, accordingly, the system arthitecture consists of three main components- Load Balancing Manager (LBM), Second-level Load Blancer (SLB), Third-level Load Blancer (TLB). In order to better descibe the details of these componnents, we first discourse the load balancing configuration and the overall packet flow.

C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 5767 3.2. Configuration A virtual IP (VIP) to external services are required for each TLB. The VIP configuration process is as follows: Step A TLB sends a configuration message containning its DIP to LBM requesting an externally routable VIP. Step 2 LBM allocates a VIP and configures each SLB with this allocation. Step 3 Packets destined to the VIP are forwarded to the DIP by SLB. 3.2.2 Inbound data flows Fig. 2: Processing inbound packets When a VIP is configured, there is multiple paths to the destination. Each path is assigned a weight which can be caculated by the mean-variance model previously described. We can provide scalability by dynamically adding TLB and Paths. Suppose there are N paths, whose latencies are given by L, L 2,, L N. Let w = (w,, w N ) T, w n denotes the weight assigned to the path n, with N n= w n =. According to the mean-variance model, we can get the optimal weight vector w by solving the following constrained optimal problems: minimize 2 w i w j σ ij. () subject to w i Li = µ. (2) w i =, (3) where µ is the expected latency and σ ij is the convariance of L i and L j. We could use lagrangian equations (Eq. (7), (8), (9), (0)) to get the optimal weight vector w. The inbound packet processing can be illustrated in Fig. 2: () The routers distribute packets destined for the VIP among all the SLB nodes based on the weight of each path.

5768 C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 (2) The SLB map the VIP to a DIP, then adopte the IP protocol to encapsulate the IP packets setting the DIP as the destination address of the IP header. (3) The SLB send the encapsulated packet to TLB. (4) The TLB extract the encapsulated packets and perform Network address translation (NAT) according to its own load balancing algorithm. (5) The TLB sends the packet to the web server. (6) The Web server sends the reply to the TLB. (7) The TLB does a reverse network address translation. (8) The TLB sends the packet directly to the routers without going through the SLB. 3.2.3 Outbound data flows Fig. 3: Processing outbound packets In our scheme, Source Net Address Translation (SNAT) is used to making the outgoing data go directly to routers without going through TLB. This greatly reduces the loads on TLB. From Fig. 3, we can see how outbound data flows are handled: () A web server sends a outbound packet to the TLB. (2) If the packet is the first oubound packet received by the TLB, the TLB will sends a message to LBM requesting an VIP. Otherwise, we go directly to Step (5). (3) LBM allocates a VIP and notifies the SLB this allocation. (4) The allocation is sent to the TLB. (5) The TLB uses the VIP to rewrite the outbound packet. Then the outbound packet is transported directly to the router. (6) The return packet is distribute to one SLB by the router based on the path weight. (7) According to the allocation previously assigned by the LBM, The SLB map the VIP to DIP. Then it encapulates the packet with the DIP and send it to the TLB. (8) The TLB sends the packet directly to the Web Server.

C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 5769 4 Experiments jitter 7 x 0 3 6 5 4 3 2 0 2 0 2 3 4 5 6 7 8 packet start time(sec) 9 0 Fig. 4: Jitter versus packet start time in onepath system jitter 7 x 0 3 6 5 4 3 2 0 2 0 2 3 4 5 6 7 8 packet start time(sec) 9 0 Fig. 5: Jitter versus packet start time in multipath system In this section, the web server clusters and all three components from Fig. are realized by simulation. We used two objective performance parameters to evaluate the performance of our scheme. Jitter is the variation in latency as measured in the variability over time of the packet latency across a network. High jitter can result in both increased latency and packet loss. Packet Loss Rate (PLR) could represent the state of network congestion. The two parameters can be caculated as follows: Jitter = (T recv(j) T recv (i)) (T send (j) T send (i)) Seq(j) Seq(i) where T recv(i) and T send(i) respectively denote the time in which the packet i was received and sent, and Seq(i) is the sequence number of packet i. (4) P LR = N lost N sent (5) where N lost is the total number of lost packets, and N sent is the total number of packets sent. For the sake of brevity, the web cluster system adopting our scheme is refered to as multipath system. Correspondingly, onepath system refers to the system in which there is only one path to the destination for each VIP. We have conducted a series of experimental comparisions between multipath system and onepath system. Fig. 4 shows the jitter varing with the packet start time in onepath system, when the user traffic is emulated. We can see that the jitter changes dramatically during the simulation period, which implies that the network has entered the congested state. Due to there is only one path to the destination, the huge network traffic in this path will lead to network congestion. Fig. 5 shows the jitter varing with the packet start time in multipath system, when the same user traffic is emulated. We can see that the jitter does not change much during the simulation period. Since there are multiple path, the huge traffic can be distributed among these path. Threfore, the bandwidth for the destination is increased and the latency becomes stable. As shown in Fig. 6, the packet loss rate of onepath system grows faster than multipath system, when the traffic grows. It means that the multipath system has higher avalability and scalability

5770 C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 than onepath system. Since there are multiple path to the destination, even if only one path is not congested, the packet will not be lost. Therfore, the multipath system can carry huge traffic with low packet loss rate. 0.9 0.8 0.7 Packet Loss Rate 0.6 0.5 0.4 0.3 0.2 0. One Path Multi Path 0 0 0 20 30 40 50 60 70 80 90 00 0 20 30 Traffic Load(M) Fig. 6: Packet loss rate versus traffic 5 Conclusion In this paper, we proposed a scalable load balancing scheme and applied it to the web server clusters. By spliting the functionality of load balancing into three level, we could scalably distribute the heavy packet processing tasks to the load balancers. The Mean-variance model is used to assigned each path a optimal weight in our multipath system. The experiments show that our scheme does provide good scalability and availability for web server clusters. In future, we can consider how to combine with Qos when distributing the network traffic. References [] K. Gilly, C. Juiz, R. Puigjaner, An up-to-date survey in web load balancing, World Wide Web, 20, Vol. 4 (2), pp. 05-3. [2] M. Aron, P. Druschel, W. Zwaenepoel, Efficient support for P-HTTP in cluster-based web servers. In: Proc. The Annual Conference on USENIX Annual Technical Conference, 999. [3] M. Y. Luo, C. S. Yang, System support for scalable, reliable and highly manageable web hosting service. In: Proc. The 3rd conference on USENIX Symposium on Internet Technologies and Systems, 200. [4] D. A. Maltz, P. Bhagwat, TCP Splicing for Application Layer Proxy Performance, Journal of High Speed Networks, 999, Vol. 8 (3), pp. 225-240. [5] M. L. Chiang, Y. C. Lin, L. F. Guo, Design and implementation of an efficient web cluster with content-based request distribution and file caching, Journal of Systems and Software, 2008, Vol. 8 (), pp. 2044-2058. [6] M. L. Chiang, C. H. Wu, Y. J. Liao, Y. F. Chen, New content-aware request distribution policies in web clusters providing multiple services, In: Proc. The ACM Symposium on Applied Computing, 2009, pp. 79-83.

C. Huang et al. /Journal of Computational Information Systems 0: 3 (204) 5763 577 577 [7] Y. F. Sit, C. L. Wang, F. Lau, Cyclone: a high-performance cluster-based web server with socket cloning, Cluster Computing, 2004, Vol. 7 (), pp. 2-37. [8] H. H. Liu, M. L. Chiang, M. C. Wu, Efficient support for content-aware request distribution and persistent connection inweb clusters, Software: Practice and Experience, 2007, Vol. 37 (), pp. 25-24. [9] K. Gilly, C. Juiz, N. Thomas, Adaptive admission control algorithm in a QoS-aware web system, Information Sciences, 202, pp. 58-77. [0] J. Yao, J. J. Ding, L. N. Bhuyan, Intelligent message scheduling in application oriented networking systems, In: Proc. IEEE International Conference on Communications, 2008, pp. 564-5645. [] O. K. MinHwan, M. S. Park, Distributing requests by (around k)-bounded load-balancing in web server cluster with high scalability, IEICE transactions on information and systems, 2006, Vol. 89 (2), pp. 663-672. [2] C. Li, G. Peng, K. Gopalan, T. cker Chiueh, Performance guarantee for cluster-based internet services, In: Proc. The 23rd International Conference on Distributed Computing Systems, 2002, pp. 327-332. [3] E. Casalicchio, M. Colajanni, A client-aware dispatching algorithm for web clusters providing multiple services, In: Proc. The 0th International Conference on WorldWide Web, 200, pp. 535-544. [4] Z. Shan, C. Lin, D. C. Marinescu, Y. Yang, Modeling and performance analysis of QoS-aware load balancing of web-server clusters, Computer Networks, 2002, Vol. 40 (2), pp. 235-256. [5] P. Patel, D. Bansal, L. Yuan, A. Murthy, Ananta: cloud scale load balancing, In: Proc. the ACM SIGCOMM 203 conference on SIGCOMM, 203, pp. 207-28. [6] R. E. Bailey, The economics of financial markets, Cambridge University Press, 2005. [7] W. Yao, H. W. Pan, C. M. Gao, A Method Based on the Combination of Dynamic and Static Load Balancing Strategy in Distributed Rendering Systems, Journal of Computational Information Systems, 204, Vol. 0 (4), pp. 759-766.