Comparing Load Balancing for Server Selection Using Cloud Services By Daksh Gupta A Project Report Submitted In Partial Fulfillment of the Requirements of the Degree of Masters of Science In Computer Science Supervised By Professor Xumin Liu Department Of Computer Science B. Thomas Golisano College of Computing and Information Sciences Rochester Institute Of Technology Rochester, New York November 11, 2013 1
The project Comparing Load Balancing for Server Selection Using Cloud Services by Daksh Gupta has been examined and approved by the following Examining Committee. Dr. Xumin Liu Professor Project Committee Chair Dr. Rajendra Raj Professor Project Reader Dr. Stanislaw Radziszowski Professor Project Observer 2
Dedication To my family for their continuous love and support; and to all my professors for their continuous guidance. 3
Acknowledgements I am grateful to all the Professors who have guided me throughout my pursue for Master s Degree in Computer Science here at RIT, but a special mention for Professor Xumin Liu for providing her guidance, and help throughout the progress of my project work. 4
ABSTRACT Comparing Load Balancing For Server Selection Using Cloud Services Daksh Gupta Supervised By: Professor Xumin Liu Load Balancing is a technique in which the requests send to the web servers are distributed in such a way that maximum resource utilization is achieved with minimum response time and servers/instances are less overloaded too. The rate at which applications at the enterprise level are expanding, it challenges the infrastructure how to balance the load among the web servers present within the enterprise. These enterprise applications faces a roadblock of determining the least loaded and good performing servers from the pool of available servers for the request sent from the client server. So in order to remain competitive in the present scenario, enterprises are required to have effective load balancing within their environment architecture. On top of it, the main criterion of how to choose architecture for the environment depends on the cost, reliability, and management in a short time frame. To survive in the competitive market, the wisest solution is to use cloud based service for developing infrastructure. This project will help in evaluating which load balancing algorithm should be used, using cloud based service in the industry with a thought on overall performance of infrastructure/system. This report focus on improving and comparing the proposed load- balancing algorithm against load balancing algorithm already implemented, and determining proposed algorithm is better suited. The algorithms will be implemented and compared based on the load, response time, utilization etc. Performance will be evaluated against load, response time, utilization, bps, tps etc. Virtual instances of web servers will be created on cloud services, which will act as web servers, and web services will be developed which will access them. Load balancing algorithms will be developed, which will also be deployed on cloud, and would route the request. The advantages of proposed load balancing algorithm will be highlighted. 5
CONTENTS Chapter 1... 9 1. Introduction... 9 1.1 Overview... 9 1.2 Background and Definitions... 9 1.3 Goals and Motivation... 12 1.4 Related Work... 13 1.5 Hypothesis... 14 1.6 Problem and Proposed Solution... 14 1.6.1 Problem Statement... 14 1.6.2 Proposed Solution... 15 1.6.3 Road Map... 16 Chapter 2... 17 2. Algorithms... 17 2.1 Dynamic Load Balancer with Least Connections and Fastest Response Time... 17 2.1.1 Algorithm... 17 2.1.2 Description... 19 2.2 Dynamic Load Balancer with Least Connections and Fastest Response Time with weight table... 20 2.2.1 Algorithm... 20 2.2.2 Description... 23 Chapter 3... 24 3. Design... 24 3.1 Architecture Diagram... 24 3.2 Integration Diagram of Load Balancing Algorithms... 26 3.3 Service Level Diag. of Load balancing Algorithm... 27 3.4 Evaluation... 28 3.5 Analysis... 28 Chapter 4... 29 4. Implementation and Requirements... 29 6
4.1 Implementation... 29 4.1.1 Implementation Strategy... 29 4.1.1.2 Service Detailed Implementation... 29 4.1.2 Objective... 30 4.1.3 SOAP Implementation... 31 4.2 Requirements... 31 4.2.1 Hardware... 31 4.2.2 Software... 31 4.2.3 Metrics... 31 4.2.3.1 Round Trip Time... 32 4.2.3.2 Memory Utilization... 32 4.2.3.3 Cost... 32 4.2.3.4 Scalability... 32 4.2.3.5 TPS... 32 4.2.3.6 BPS... 32 4.2.3.7 Simple Strategy... 33 4.2.3.8 Variance Strategy... 33 4.2.3.9 Thread Strategy... 33 Chapter 5... 34 5. Analysis... 34 5.1 Basic Set Up... 34 5.2 Performance Test... 36 5.2.1 Simple Strategy... 36 5.2.2 Thread Strategy... 38 5.2.3 Variance Strategy (variance 0.5)... 39 5.2.4 Testing Using HP Load Runner Tool... 41 Chapter 6... 43 6. Conclusion... 43 6.1. Current Status... 43 6.2 Future Work... 44 6.3 Lessons Learned... 44 BIBLIOGRAPHY... 45 7
Index for Tables Table 1: Simple Strategy results comparison... 36 Table 2: Thread Strategy results comparison... 38 Table 3: Variance Strategy results comparison... 39 Table 4: HP Load Runner Results Comparison. 41 Index for Figures Figure 1: Sample SOAP Request... 11 Figure 2: Low Level Architecture... 24 Figure 3: High Level Architecture... 25 Figure 4: Integration Diagram of Load Balancing... 26 Figure 5: Service Diagram of Load Balancing... 27 Figure 6: Soap Request... 33 Figure 7: Dynamic Load Balancer with Least Connections and Fastest Response Time WSDL... 34 Figure 8: Dynamic Load Balancer with Least Connections and Fastest Response Time with Weight table WSDL... 35 Figure 9: Average Time (Strategy Mode)... 37 Figure 10: Maximum Time (Strategy Mode)... 37 Figure 11: TPS (Strategy Mode)... 37 Figure 12: Average Time (Thread Mode)... 38 Figure 13: Maximum Time (Thread Mode)... 38 Figure 14: TPS (Thread Mode)... 39 Figure 15: Average Time (Variance Mode)... 40 Figure 16: Maximum Times (Variance Mode)... 40 Figure 17: TPS (Variance Mode)... 40 Figure 18: 99%tile Response Time (HP LoadRunner)... 41 Figure 19: TPS (HP LoadRunner)... 42 8
Chapter 1 1. Introduction 1.1 Overview Number of users using Internet is growing at an alarming rate so the need of balancing the traffic on the enterprise applications in order to provide high performance and high availability [8]. There are numbers of load balancing algorithms present, but each and every algorithm face some problem or other. The main goal of load balancing is to achieve route the requests among the web servers with a minimum response time. And another issue that enterprise faces is the high cost, managing of web servers built where load balancer can be deployed. Cloud based infrastructure/service can be used as one of solutions where web servers can be built as it has various advantages like cost effective, easily managed, virtual instances, and is scalable too. There are various load balancing techniques available in cloud- based services like Predictive load balancing, random biased, join idle queue etc. [12]. This project will help in evaluating which load balancing algorithm should be used on top of using cloud based service in the industry with a thought on overall performance of infrastructure/system. This report focus on improving and comparing the proposed load balancing algorithm against load balancing algorithm [1] already implemented, and determining proposed algorithm is more well suited. This report describes comparing load balancing techniques on the cloud services, which will act as web servers by using SOAP requests on web services as data set. Web services [13] will be implemented which will each point to one of the load balancing technique, and load will be increased on it. The main objective of load balancing algorithm will be based on distributing or route the data depending upon the closest or best performing web servers. The algorithms will be implemented and compared based on the load, response time, utilization etc. Performance will be evaluated against load, response time, utilization etc. [17, 18]. Virtual instances of web servers will be created on cloud services, which will act as web servers, and web services will be developed which will access them. Load balancing algorithms will be developed, which will also be deployed on cloud, and would route the request. The advantages of proposed load balancing algorithm will be highlighted. 1.2 Background and Definitions Number of users using Internet is growing at an alarming rate so the need of balancing the traffic on the enterprise applications is critical to provide higher tps, lower response time, and less number of outliers [5]. This is being achieved through 9
load balancing, which means to route the requests among web servers. There are number of load balancing algorithms present, but each of them has some issues or disadvantages related to it. Standard definitions: Load Balancing: It means balancing the increased client requests on the distributed instance deployed on the server so as to avoid increased delay in response time due to the increased load. It is mainly important for the high level applications where number of request sent by the user can range from thousands to millions i.e. it s very difficult to predict the number of requests. They need to have many web servers for load balancing. They can be categorized under either static algorithms or dynamic algorithms. Round robin, random etc. are categorized under the static algorithms, whereas least connection, observed, etc. are categorized under dynamic algorithms. [22] Random: The random scheduling, requests received from the client are being assigned to any random server from a given list. As the requests are being assigned randomly, there is no mechanism of sharing the load, which leads to overloaded servers. This algorithm leads to underutilization and overutilization of servers. [22] Round Robin: The round robin algorithm routes the new request to the instance or server, followed in queue. It distributes the requests evenly among all the instances so that load is shared evenly. This algorithm is better than the random as requests are diving evenly among the servers. But the main disadvantage of this algorithm is it works better only of all the servers have same configurations. [22] Weighted Round Robin: This algorithm is an enhancement of Round robin where each instance gets assigned with a load depending on processing capability, which is, determined on how that instance is behaving. It removes the deficiency of round robin, but it does not consider the processing time each server is taking in responding the response. [22] To remove, these deficiencies, dynamic load balancing algorithms were used. [3] Dynamic Load Balancing: Dynamic load balancing monitors the state of the server continuously and assigns request to the server, which has less load. There are various dynamic load- balancing algorithm like least connection, fastest, adaptive load balancing etc. Fastest Load Balancing: This method routes the request from the client to that server which has fastest response time [6]. The disadvantage or shortcoming of this algorithm is it s not 10
possible for every server to respond the response in seconds, which will lead to congestion in the network. Least Connection Load Balancing: This method derives the meaning from its name only. The server passes a new request from the client to that web server which has very least connection at that point of time [3, 22]. This technique works best where all the applications running web server have same infrastructure. The advantage of this algorithm leads to disadvantage of it. If there are two applications having different infrastructure i.e. HTML application and other uses J2EE or xml, it will lead to bottleneck of connections, as all connections will require have different round trip time as its dependent on the server from where the request is originated. [22] Observed Load Balancing: This technique makes use of fastest, least connection. Web servers are categorized based on least and response time [3, 22]. The server with less requests and higher throughput time gets the new request. But the disadvantage faced by it is it does not have any weights associated the server, due to which the server starts getting overloaded with requests. Web Service: Web service is a medium that permits communication between the applications independent of the platform as well as the language used to program it. It consists of functions, which are accessed by other applications using XML. This interface mechanism means SOA (Service Orientation Architecture). [23] SOAP: The Simple Object Access Protocol (SOAP) provides a transport layer between the applications interacting with the web service using soap wsdl in distributed network. SOAP message consists of XML, which consist of the function that application will be requesting from the web service. The web service end point and port is being provided to application for it to communicate with web service. SOAP xml is generated through the WSDL provided by the web service. WSDL consists of functions that web- service provides, and can be accessed through the network. [17] Figure 1: Sample SOAP Request 11
The above figure consists of sample soap request that application uses to interact with web service. JAX- WS: It s an API used in Java, which help in developing the web services, it s a segment of Java EE. It s an open source project. [16] Cloud Computing: Cloud computing is a service provided through the Internet. This service helps the enterprise to obtain software resources like apache, WAS, java, etc. as well as hardware resources i.e. getting firewall set up for the server etc. These days many enterprises or industries are going towards cloud computing, which is not only cheap, but also scalable. One more advantage of using these services is industry are developing an attitude of paying whenever the service is being requested rather than paying a lump amount for everything. Amazon Web Services (AWS) It s a pack of web services provided by Amazon.com so that they can be used for cloud computing. They have many advantages like scalable, reliable, cost effective etc. [24] Amazon Elastic Cloud Compute It is one of the web services, which comes as a part of AWS. It s a web service that helps in increasing the capacity on the server or helps in creating lot of virtual servers, which can be used by clients. [24] Billing is done on the basis of how much the virtual servers are used. 1.3 Goals and Motivation Different enterprises have different requirements, depending upon their needs and budgets. Some enterprises would like to achieve high performance from and availability of their systems without any concerns about the cost incurred, while smaller enterprises with limited budgets would like to get the most out of their systems. Number of users using Internet is growing at an alarming rate so the need of balancing the traffic on the enterprise applications in order to provide high performance and high availability. There are numbers of load balancing algorithms present, but each and every algorithm face some problem or other. The main goal of load balancing is to achieve route the requests among the web servers with a minimum response time. This report focus on improving and comparing the proposed load- balancing algorithm against load balancing algorithm [1] already implemented on cloud servers, and determining proposed algorithm is better suited. Web services will be implemented which will each point to one of the load balancing technique, and load will be increased on it. The load balancer algorithms will be developed as web service so that the client can use it as an end point. The 12
web service will act as global traffic manager (GTM), which will be called by the client, and algorithm will act as local traffic manager (LTM) routing the requests to the appropriate cloud servers to get the result back to client in minimum time. The algorithms will be implemented and compared based on the load, response time, utilization etc. Performance will be evaluated against load, response time, utilization etc. Virtual instances of web servers will be created on cloud services, which will act as web servers, and web services will be developed which will access them. Load balancing algorithms will be developed, which will also be deployed on cloud, and would route the request. 1.4 Related Work DONAR [2] (Decentralized Server Selection) was being developed as a distributed system that provides make sure to resolve the name conflicts and also choosing the appropriate server for the client request which has the less load. This algorithm was being developed to direct incoming request from the client to appropriate web servers in order to balance the load between the web servers. The way this algorithm works is mapping node gets the request from the client, which then sends the request to server. Mapping node has an optimizer, and is decentralized, which listens to other nodes, and collects information, and then sends the request from the client to unoccupied or least used resources/server. High Level architectures applications [1] face performance issues due to the load not getting distributed evenly among the web servers. One of the solutions designed for it was use of dynamic load balancing [1]. This paper designs an architecture where the distributed algorithm uses a load balancer which observe the distributed load centrally, and use various distributed load reallocation policies [6] which reduces the imbalance of load as the load is distributed in a distributed manner thereby it removes the single point of failure, delays, and bottlenecks [1]. This paper [3] discusses the use of adaptive load balancing in an environment. It tries to explain various load balancing algorithms. Round robin algorithm tries to route the requests equally among the instances/instances, it does not care how many connections instance already has previously. Load Connection table is used to check whether server is loaded, normal, under loaded or idle. This table helps in determining to which server the request should be routed too. It also takes into account response time that each request make, in order to find server is behaving the best, for the client to route the request too. [3] Weighted Average Load Balancing technique presented a load balancing technique using preemptive scheduling by Qin [4]. Whenever a new request is received from the client, the algorithm has to do one of the following tasks. It either has to add to the present queue or interrupting current task in execution. Whenever job gets assigned to a node, it checks whether the node has become over loaded or not. If the 13
node gets overloaded, algorithm tries to find a new node so that it transfers the most useful jobs from the overloaded node. Presently there are various techniques of load balancing [6] which falls under the category of either inter web servers (there is only one web servers, and algorithms for load balancing are architected to balance load within that) or intra web servers (i.e. it consists of more than one web servers, load balancing algorithms are designed to distribute load among them). This paper uses an algorithm whose aim was to reduce the imbalance in the web server. The way algorithm (VectorDot) [6] used to work is it used to find nodes in the web servers where imbalance has occurred, and then try to shift the load to available free nodes. Imbalance is found out when the threshold on particular node has increased its limit. 1.5 Hypothesis There are many papers focused on how to balance the load on high- level applications using dynamic load balancing. My aim is to compare two load- balancing algorithms on compared based on the load, response time, utilization etc. Performance will be evaluated against load, response time, utilization, bps, tps etc. I will describe the use of cloud services, Web Services, XML and improved algorithm, which will try to find server, which has less load. I will be working through my research with the initial approach of implementing the algorithms. In next phase, I will be comparing them against the parameters defined in the metrics, and in the last phase will increase the load on cloud by creating a lot of virtual instances or by using load technique in SOAP UI. 1.6 Problem and Proposed Solution 1.6.1 Problem Statement Different enterprises have different requirements, depending upon their needs and budgets. Some enterprises would like to achieve high performance from and availability of their systems without any concerns about the cost incurred, while smaller enterprises with limited budgets would like to get the most out of their systems. Number of users using Internet is growing at an alarming rate so the need of balancing the traffic on the enterprise applications in order to provide high performance and high availability. There are numbers of load balancing algorithms present, but each and every algorithm face some problem or other. The main goal of load balancing is to achieve route the requests among the web servers with minimum response time. This report focus on improving and comparing the proposed load- balancing algorithm against load balancing algorithm [1] already implemented on cloud servers, and determining proposed algorithm is better suited. 14
1.6.2 Proposed Solution The main goal of load balancing is to achieve route the requests among the web servers with minimum response time. This report describes comparing load balancing techniques on the cloud services. I will be using Web services, XML and cloud to design and implement, and then will provide a comparison metrics between the algorithms in terms of query response time, bps, tps, CPU percentage etc. in against the load. The following steps will be followed in order to achieve this process: Step1: JAX- WS web service will be developed which will return response requested by client. SOAP UI tool will behave as the client for the web service, where we will send soap requests and receiving soap responses. Step2: Load balancing algorithm will be developed using Java. There are various types of Load balancing algorithms existing, but I will be using the Dynamic Load balancer [1] and will try to remove its shortcomings with by adding additional logic by introducing server weights logic to it. Dynamic Load Balancer With Least Connections and Fastest Response Time: This paper [1] describes the use of dynamic load balancing. The paper describes the importance of load balancing in system- oriented architecture. The algorithm makes use of combining the logic of least connections present, and fastest response time. Combining them helps in reducing the load. Monitoring agents are used to look at the current activities, and load. They try to classify the instances as over- loaded, balanced and under loaded. Depending upon the instance nature, that instance is being called, which returns the results. As the connections keeps on increasing, this algorithm suffers from in longer run as either new instances need to be created or all the instances are almost on the same side of balanced, or over loaded so it the efficiency reduces. Dynamic Load Balancer with Least Connections and Fastest Response Time With Weight table: This algorithm integrates the concept of dynamic load balancing [1] with weight table assigned to each instance [4]. Each instance has enhanced logic of having a dynamic number assigned for maximum connections defined in the weight table. Whenever the connections limit reaches the threshold, it resets, and starts from fresh. Step3: Amazon Web Services will be used to create EC2 instances. EC2 instances will be created where the load balancer developed in step2 and the web service will be 15
deployed in step1. Load balancer algorithm will determine which EC2 instance of web service should be called for sending the request, and getting the response. Step 4: Soap UI tool will be used to increase the load on the EC2 instances, and will be using soap load UI tool to compare the results from the two- load balancer depending on request/response sent. The results will be compared on the basis of performance, bps, tps, round trip time, and CPU utilization. [14]. The dynamics of creating instances, algorithm, and running tasks will be a challenge. 1.6.3 Road Map To understand the project, the project report is distributed in different sections. Chapter 1 gave a summary about introduction, background, related work, problem statement, and proposed solution. Chapter 2 will provide insight on how the algorithms work, and description. Chapter 3 and 4 will provide description about design, approach, and implementation of project. Chapter 5 describes the experimental results, and with Chapter 6 discuss about the current status, future, and conclusion. 16
Chapter 2 2. Algorithms Load Balancing: It means how to fine- tune the traffic among the distributed application so as to avoid the increase in response time when the load has increased on the server. It is mainly important for the high level applications where number of request sent by the user can range from thousands to millions i.e. it s very difficult to predict the number of requests. They need to have many web servers for load balancing. They can be categorized under either static algorithms or dynamic algorithms. Round robin, random etc. are categorized under the static algorithms, whereas least connection, observed, etc. are categorized under dynamic algorithms. [3, 22] 2.1 Dynamic Load Balancer with Least Connections and Fastest Response Time 2.1.1 Algorithm LOADBALANCER (request) Input: request Output: response Declare ec2 Instance [][], url, url selectbestserverinstance(request, ec2instances) soapreplyanswer getreply (url, request) return soapreplyanswer addconnection (connection,ec2instance) Input: connection[][], ec2instance[] Output: connection[][] i 0 j 0 for ec2 1 to ec2instance.length do j 0 connection[i, j] ec2[i] j j+1 connection[i, j] 0 i i +1 return connection selectbestserver(request, ec2instances) 17
Input: request,ec2instances [][] Output: ec2instance Declare connection [][], responsetime[][], connection addconnection(request, connection,ec2instance) Declare leastconnectionurl, leastresponsetimeurl, secondminconnections i 0 j 1 for connection 1 to connection.length do if connection[i, j] = 0 then connection[i, j] 1 return connection[i, j] else i i +1 Sort the ec2instances of connection in increasing order of connections leastconnectionurl connection[0,0] Sort the ec2instances of responsetime in increasing order of average response time leastresponsetimeurl response[0,0] i 0 if leastconnectionurl = leastresponsetimeurl then return leastconnectionurl else for connection 1 to connection.length do if connection[i,0] = leastresponsetimeurl then numberofconnections connections[i,1 else i i +1 secondminimumconnections connection[1][1] if numberofconnections = secondminimumconnections then return numberofconnections else return responsetimeurl getreply (url, request) Input: url, request Output: reply starttime 0, i 0, j 0 endtime 0 averagetime 0, responsetime 0 starttime getcurrenttime soapreply answerfromwebservice(url,request) 18
endtime getcurrenttime responsetime endtime starttime for response 1 to response.length do averagetime += response[i,1] i i +1 responsetime[url][1] averagetime return soapreply ColumnComparator() Input: Output: sorted Array declare columntosort; ColumnComparator(int columntosort) this.columntosort columntosort compare(object o1, Object o2) String[] row1 (String[]) o1; String[] row2 (String[]) o2; return row1[columntosort].compareto(row2[columntosort]); 2.1.2 Description This balancer tries to find the minimum loaded server based on the connections and response time. After finding the server, it sends that request to it. The algorithm tries to find server by looking at the number of connections and response time. There are many conditions checked in the algorithm to find out which balancer is performing best at that moment. Monitoring agents are used to look at the connections each server has and response time associated with it and after performing the logistics, best server instance is being sent the request. [1] 19
2.2 Dynamic Load Balancer with Least Connections and Fastest Response Time with Weight table 2.2.1 Algorithm LOADBALANCER (request) Input: request Output: response Declare ec2 Instance [][], url, url selectbestserverinstance(connection, numberarray, responsetime) soapreplyanswer getreply (url, request) return soapreplyanswer addconnection (connection,ec2instance) Input: connection[][], ec2instance[] Output: connection[][] i 0 j 0 for ec2 1 to ec2instance.length do j 0 connection[i, j] ec2[i] j j+1 connection[i, j] 0 i i +1 return connection removeconnection (connection) Input: connection[][] Output: connection[][] connection null connection new Connection return connection addrandomnmbrtoserver (numberarray,ec2instance) Input: numberarray[][], ec2instance[] Output: numberarray[][] i 0 j 0 for ec2 1 to ec2instance.length do j 0 randomint randomgenerator.nextint(10) numberarray[i,j] ec2instance[i] i i +1 20
numberarray [i,j] randomint return numberarray removenumbers (numberarray) Input: numberarray[][] Output: numberarray[][] numberarray null numberarray new numberarray return numberarray checkconnectionsagainst WeightOnEachInstance(connection,numberArray) Input: numberarray[][], connection[][] Output:connection i 0, j 1,k 1 checknumberweight false for connection i to connection.length for number k to numberarray.length do if connection[i,1] = numberarray[k,1] then checknumberweight true else checknumberweight false weightnotusedurl connection[i,0] i i +1 k k +1 if checknumberweight true then removenumbers (numberarray); removeconnection(connections); addrandomnmbrtoserver (numberarray, ec2instance); addconnections(connection,ec2instance) selectbestserver(request, ec2instances) Input: request,ec2instances [][] Output: ec2instance Declare connection[][], ec2 Instance[], url, responsetime[][], numberarray[][] connection addconnection(connection,ec2instance) numberarray[] addrandomnmbrtoserver(numberarray,ec2instance) Declare leastconnectionurl, leastresponsetimeurl, secondminconnections, weightnotusedurl, lessnumberurlconnection 21
i 0 j 1 connection checkconnectionsagainstweightoneachinstance(connection,numberarray) for connection 1 to connection.length do if connection[i, j] = 0 then connection[i, j] 1 return connection[i, j] else i i +1 Sort the ec2instances of connection in increasing order of connections leastconnectionurl connection[0,0] Sort the ec2instances of responsetime in increasing order of avg rspnse time leastresponsetimeurl response[0,0] i 0 for connection i to connection.length for number k to numberarray.length do if connection[i,1]!= numberarray[k,1] then lessnumberurlconnection[i] = connection [i,1] if leastconnectionurl = leastresponsetimeurl && weightnotusedurl = leastresponsetimeurl then return leastresponsetimeurl else for connection 1 to lessnumberurlconnection.length do if connection[i,0] = leastresponsetimeurl then numberofconnections connections[i,1] else i i +1 secondminimumconnections connection[1][1] if numberofconnections = secondminimumconnections then return numberofconnections else return responsetimeurl getreply (url, request) Input: url, request Output: reply starttime 0, i 0, j 0 22
endtime 0 averagetime 0, responsetime 0 starttime getcurrenttime soapreply answerfromwebservice(url,request) endtime getcurrenttime responsetime endtime starttime for response 1 to response.length do averagetime += response[i,1] i i +1 responsetime[url][1] averagetime return soapreply ColumnComparator() Input: Output: sorted Array declare columntosort; ColumnComparator(int columntosort) this.columntosort columntosort compare(object o1, Object o2) String[] row1 (String[]) o1; String[] row2 (String[]) o2; return row1[columntosort].compareto(row2[columntosort]); 2.2.2 Description After receiving the request from client, it refers to the connection instance to find the least loaded instance. Response time of least loaded instance is found out, and compared in response- time table and checks whether request should be sent to least loaded server or minimum response- time server. The connection instances are compared against the weight table to see whether connections have reached the number of pre- defined connection that server should have or not. The connection table, response table gets updated. The monitoring service looks to see is there a change in load level on the servers in addition to connections and response time associated. If there is change in weight level, the weight table gets updated with new values. The central server contains weight table has all the value from individual weight tables defined on respective servers. [1,4] 23
Chapter 3 3. Design 3.1 Architecture Diagram Client: They are the users who will request news feed from the web service. They act as requestor for the service, and only authorized users are allowed to use this application. They pass the input to the web service, and expect fast output. Web Service: A web service will be developed, which will find out which server is best available for the client to receive output. A load balancing web service will be created, which after finding the server will send the input/request from the client to another web service hosted on that server to get required output before forwarding it to the user. Web Servers created will be EC2 instances, and web service will be developed using JAX- WS, and with tomcat as the server hosted on it. Server1 Client Client Central Server Server2 Client Response Table Server3 : Describes a weight table, which has weights, assigned to each server. : Dynamic Load Balancing Diagram based on connections and response table (Existing) : Dynamic Load balancing based on connections, response time and Number of weights (Proposed) Figure 2: Low Level Architecture 24
Figure 3: High Level Architecture 25
3.2 Integration Diagram of Load Balancing Algorithms Ec2 Inst Ec2 Inst Ec2 Inst Ec2 Inst Request Request, Ec2Instances Connection, Ec2Instances S : Dynamic Load Balancing Diagram based on connections and response table (Existing) : Dynamic Load balancing based on connections, response time and Number of Weights (Proposed) Figure 4: Integration Diagram of Load Balancing 26
3.3 Service Level Diag. of Load balancing Algorithm All EC2 instances are already created LOAD BALANCER (request) CLIENT Client Makes a Request Central Server Receives the Request request addconnection(conctn,ec2ins) Initialize connection Array by adding all ec2instames with 0 Initialize response Array by adding all ec2instames with 0 response time initially addrandomnumbertoserve r(weightarray, ec2instnce) Initialize weight Array by adding all ec2instames with random weights initially. This will serve as load table Check the select best server function to find best server Find least connection connect server Client receives Response Find the least response time server : Load Balancer : Add Connection Function : Add Response Function : Add Weights Function : Select Best Server Function :Get Reply : checkconnectionsagainst WeightOnEachInstance response Compare least connection server with least response time server After getting server instance, it s compared against the weights to check whether the number of connections for that server has reached that weight assigned. If it has reached, we will take that server out of pool, and take next least loaded connection. Otherwise return the Number of connections is compared against weight assigned for each ec2 instance.. If they are equal, resets all connections to zero and reset the weights. Otherwise return the server, which is least, overloaded with connection with respect to weight assigned. Ec2 Instances/Servers where WS is deployed If both provide same server, return the server, otherwise compare response time of lowest 2 servers. If the difference is minimal, return one with lowest connection points, otherwise return the server with lowest response time. Server gets returned. Connection Array and response tome array are modified for that ec2 instance., And sends request to WS Get the response back from web service : Dynamic Load Balancing Diagram based on connections and response table (Existing) : Dynamic Load balancing based on connections, response time and Number of Weights (Proposed) Figure 5: Service Diagram of Load Balancing 27
3.4 Evaluation After setting up the project, experiments will be performed to evaluate the performance of the algorithm present, and proposed. Performance criteria will be measured on the basis of adding load to the web service by increasing number of threads, and will measure throughput per second. Soap UI tool will be used to increase the load on the web service. 3.5 Analysis The results produced from the performance will help us to analyze which algorithm is behaving better on basis of increasing the thread count, i.e. increasing number of transactions hitting against the web server. Various strategies will be used, and soap- ui tool will be used to increase the transaction per second, and response time will be monitored for it. Analysis will be based on 99%tile of all the response time, along with maximum time, minimum time and average time too. Standard deviation will also be measured for both the algorithms. Various types of strategies like variance, thread, simple etc. will be used. 28
Chapter 4 4. Implementation and Requirements 4.1 Implementation 4.1.1 Implementation Strategy The project implemented will be based on three tier architecture i.e. client, web service server, and server level. Client will be implemented which will act as a web service client which access the services hosted on the server level. The middle layer will also be a web service, which will interact with the service on server level. Middle layer will be basically a mapping algorithm, which finds out which service server to ping in order to return the answer to the client in least response time. Load balancing algorithms will be implemented on middle layer i.e. they act as mapping algorithm between clients and server. 4.1.1.2 Service Detailed Implementation The client sends request, which is being retrieved by the central server, which decides which server it should send the request to get the response back in minimum time. It consists of 3 sections: Client: Requests the server for the response for the request sent. The request will be in form of SOAP requests, and will get SOAP response. Central Server: Dynamic Load Balancer with Least Connections and Fastest Response Time: This balancer tries to find the minimum loaded server based on the connections and response time. After finding the server, it sends that request to it. The algorithm tries to find server by looking at the number of connections and response time. Monitoring agents are used to look at the connections each server has and response time associated with it and after performing the logistics, best server instance is being sent the request. [1] Dynamic Load Balancer With Least Connections and Fastest Response Time With Weights Assigned: Each cloud tomcat server is assigned with weight through the algorithm. The weights signify total number of connections each tomcat server should handle before stopping accepting new requests of client connection. For example: if a server is assigned with a weight of 30, it can handle 30 connections. The way weights help in balancing the algorithm is it prevents the server to get overloaded with connection requests as it reset the weights on all the server after all servers have 29
Server: reached its assigned weight. Resetting of weights resets the load weight table, as well as the connections table i.e. it helps in overcoming the overloading of objects as they got re- initialized. The client sends the request to load balancer; it refers to the connection instance to find the least weighted instance. Response time of least connection instance is found out, and compared in response- time table and checks whether request should be sent to least connection server or minimum response- time server. The connection instances are compared against the weight table [4] to see whether connections have reached the number of pre- defined connection that server should have or not, and if it has, no new request will be connected against that particular server. The connection table, response table gets updated. If all the servers have reached respective weights assigned, the weight, and the connection get reset, and it start from the scratch, i.e. all the objects used in algorithm again initialize, thereby reducing the complexity of trying to find least connection which is used to face in algorithm [4]. The monitoring service looks to see is there a change in load level on the servers in addition to connections and response time associated. If there is change in load level, the weight table gets updated with new values. The central server weight table has all the value from individual weight tables defined on respective servers. The main objective of this section is to process soap request. Web Service will be hosted on these servers. After processing the soap request sent by the mapping algorithm through client, it sends back the response to the client, which he has requested for. 4.1.2 Objective The main objective of the project is to improve the balancing algorithm. In this project, I will try to improve the algorithm Dynamic Load Balancer with Least Connections and Fastest Response Time mentioned in [1]. The main disadvantage of the current implemented algorithm is as the connections keeps on increasing, this algorithm suffers from in longer run as either new instances need to be created or all the instances are almost on the same side of balanced, or over loaded so it the efficiency reduces. In order to remove the shortcoming discussed above, the new proposed algorithm will integrate the concept of dynamic load balancing [1] with weight table assigned to each instance [4]. Each instance has enhanced logic of having a dynamic number assigned for maximum connections defined in the weight table. Whenever the connections limit reaches the threshold, it resets, and starts from fresh, thereby increasing the efficiency. The new proposed algorithm is termed as Dynamic Load Balancer with Least Connections and Fastest Response Time with Weight table. 30
4.1.3 SOAP Implementation The project will be using SOAP messages for request/response data between client and the server. The request will be send to the load balancer algorithm running on tomcat, which will find out which will try to find out which server is behaving best at the moment, and will forward the request to that server, which will respond back with the output. In total 3 web services will be developed. The first two web services developed are for the load balancer algorithm, and third web service is for the program, which will return the output back to client. Both load balancer web service will be calling third program web service for the output responded back to the client. 4.2 Requirements 4.2.1 Hardware The system hardware used for the implementation will be my local machine, which has windows and Linux both configured. The hardware details are as follow: Windows 7, i5 processor Linux Env 4Gb RAM, 64 bit Operating System Amazon Cloud 4.2.2 Software Tomcat 6 Java 1.6 Web Service will be built using JAX- WS 3.0 API on Java1.6. SoapUI client will be used to access the web service. Eclipse IDE will be used for development purposes. Load balancing algorithms will be developed using Java 1.6. Amazon EC2 virtual instances will be created [12]. Jars will be downloaded. UTF- 8 Encoding will be used. Spring 3.0 HP Load Runner Tool 4.2.3 Metrics The core of any comparison study is metrics. Several studies have used different metrics (such as performance, storage etc.) to evaluate and compare different services [7] [8] [9]. The project will consider metrics, which are relevant for this 31
comparison. The various parameters that can be considered are the performance, response time, scalability, throughput, and resource utilization. Analysis will be based on 99%tile of all the response time, along with maximum time, minimum time and average time too. Standard deviation will also be measured for both the algorithms. Various types of strategies like variance, thread, simple etc. will be used. 4.2.3.1 Round Trip Time The end- to- end response time will be compared between the two load balancers by sending requests through the web service. For good load balancer, end- to- end response time should be as low as possible. The Response time will be measured against the load sent on two load balancers. 4.2.3.2 Memory Utilization Memory utilization will be compared for the load balancers against the load. Memory utilization can be measured through JVM. To determine whether load balancer is better, it should be consuming less memory, and should do garbage collection at timely manner to remove all unused object. It should not do memory leak. 4.2.3.3 Cost Cost for utilizing different cloud services will be estimated as its one of the main deciding factor for the enterprises. The cost serves as one of the main deciding factor because enterprise has budget constraints, and they have to look at the best available option, which will be beneficial for the industry. 4.2.3.4 Scalability This is an ability of an algorithm to perform load balancing with any finite number of nodes. The scalability will be tested against the load balancer algorithm by increasing the number of nodes, or number of transactions per second to check what the upper limit is till the system starts degrading. The higher the scalable factor, better the performance is. 4.2.3.5 TPS It stands for transaction made per second. For example: a test case that runs for 10 seconds and handled 100 requests will get tps of 10. Algorithm, which provides higher tps, is considered better. [18] 4.2.3.6 BPS It stands for bytes per second. For example: test case running for 10 seconds and handled 100000 bytes will get BPS of 10000. Algorithm, which provides higher bps, is considered better. [18] 32
These parameters will be compared with the data set that will be passed to the URL s where the load balancer is installed which will further send the requests to the web service and get the data back. Data set will be in form of soap requests. We will increase the number of thread count of the soap request in order to overload the server. The data set will be of the format: Figure 6: Soap Request The SOAP request will be sent through SOAPUI, and we will be using SOAP UI Load Test tool [15] to get the metrics we have to compare. The metrics will be compared against the parameters bps, tps, avg time (rtt), memory utilization. Different Load strategies will be used for each parameter for evaluating the response from the load balancer. The strategies are simple, variance, burst, thread, grid [15]. 4.2.3.7 Simple Strategy This strategy runs by adding definite amount of thread with a definite delay between each thread run for server. For example for a test to run with test with 5 threads with 5 seconds delay, we initialize thread array to 5, delay to 5000ms. [18] 4.2.3.8 Variance Strategy This strategy fluctuate the thread amount over a course of duration in a sawtooth manor; we set interval to the desired value and the variance value which determines how much thread amount should decrease or increase during the test duration. [18] 4.2.3.9 Thread Strategy This test increases the amount of thread from one level to another over the run test duration. The main purpose of this strategy is to find at which thread count the test can attain the highest transaction per second, and it can also be used to find the level of thread count where the performance starts degrading. [18] 33
Chapter 5 5. Analysis 5.1 Basic Set Up To check how the algorithms are behaving, and to see whether the proposed algorithm is performing better than the original algorithm, performance tests were being carried out to check the performance. The tests were performed with different parameters to find the performance. We use soap Ui tool, and add the wsdl of the load balancer algorithm to it. Every request will hit the load balancer algorithm, which has the various EC2 instances on which a web service is deployed which provide the response that client is requesting for. The main aim of this performance test is to test the middle layer. Middle layer consists of load balancer, which routes the request. The efficiency will be determined by comparing tps, bps, memory usage by JVM (will use top command) etc. We will compare efficiency of proposed algorithm, and will show that using proposed algorithm Dynamic Load Balancer with Least Connections and Fastest Response Time with weight table is better, and efficient. WSDL generated are shown below: Figure 7: Dynamic Load Balancer with Least Connections and Fastest Response Time WSDL 34
Figure 8: Dynamic Load Balancer with Least Connections and Fastest Response Time with Weight table WSDL The architecture design used in this project is three- tier architecture i.e. client, mapping server, and response server. The client sends request, which is being retrieved by the central server, which decides which server it should send the request to get the response back in minimum time. The main objective of this project is to develop the mapping algorithm for the central server. There will be two mapping algorithm developed, one with dynamic load balancer with least connections and fastest response time, and another with dynamic load balancer with least connection, fastest response time with weights. The way the dynamic load balancer with least connection and fastest response time works [1] is the balancer tries to find minimum loaded server based on connection and response time. After finding the server, it sends that request to it, and gets the response, and sends it back to client. The way the proposed algorithm (Dynamic Load Balancer with Least Connections and Fastest Response Time with Weights Assigned) [1,4] works is each cloud tomcat server in the algorithm are assigned with equal weights. The weights signify total number of connections each tomcat server should handle before stopping accepting new requests of client connection. After receiving the request from client, it refers to the connection instance to find the least weighted instance. Response time of least connection instance is found out, and compared in response- time table and checks whether request should be sent to least connection server or minimum response- time server. The connection instances are compared against the weight table to see whether connections have reached the number of pre- defined connection that server should have or not, and if it has, no new request will be connected against that particular server. The connection table, response table gets updated. If all the servers have reached respective weights assigned, the weight, and the connection 35
gets reset, and it start from the scratch, i.e. all the objects used in algorithm again initialize. 5.2 Performance Test The load balancer web service is deployed on tomcat 6.0 using java 1.6. Soap Ui tool is used to send the request to the load balancer web service, and load strategy is used to carry out the performance test. Various strategies were used to test against the load balancers (original and proposed), to find which load balancing algorithm was behaving better. We will be running three different load tests to compare our performance test results. The load tests that will be run will be simple, thread and variance mode. The thread level will be adjusted in each run of the load test along with the time too. The Parameters that will be compared will be minimum time, average time, and tps (transactions per second). The main focus of each test will be to see whether the new proposed load balancing algorithm have better tps (transactions per second) rate than original. Higher the tps better will be the algorithm performance. 5.2.1 Simple Strategy This strategy runs by adding definite amount of thread with a definite delay between each thread run for server. For example: for a test to run with test with 5 threads with 5 seconds delay, we initialize thread array to 5, delay to 5000ms. [18] Below are the results obtained after running tests against algorithm. Average Time (ms) Maximum Time (ms) TPS Thread Count Time (Sec) Original Algorithm Proposed Algorithm Original Algorithm Proposed Algorithm Original Algorithm Proposed Algorithm 5 60 232 231 456 450 5.01 5.05 10 80 234 230 337 350 10.06 10.13 15 100 233 232 309 325 15.01 15.13 20 120 234 231 357 340 20.19 20.3 25 150 235 233 392 355 25.3 25.4 Table 1: Simple Strategy results comparison The table was plotted with graphs with left axis contains average time (ms) v/s right axis as thread count. 36
236 234 232 230 Original Algorithm Proposed Algorithm 228 0 5 10 15 20 25 30 Figure 9: Average Time (Strategy Mode) By running the tests for Strategy mode against both algorithms, it was seen average response time of proposed algorithm performs better than the algorithm [3]. Response time described here is from client end. Lower response time is one of the features that are most wanted, so by analyzing the result from the graph, it can be said that proposed algorithm behaves better. 500 400 300 200 100 0 0 5 10 15 20 25 30 Figure 10: Maximum Time (Strategy Mode) Original Algorithm Proposed Algorithm This test is to find the outliers and to find whether the outlier s time is reduced or not. From the graph, we can analyze that by increasing the thread count; the maximum time keeps on decreasing, thereby concluding the maximum time reduces with increase in thread when compared proposed v/s original algorithm [3]. 40 20 0 0 5 10 15 20 25 30 Figure 11: TPS (Strategy Mode) Original Algorithm Proposed Algorithm The above test compares tps for the proposed algorithm against original algorithm. For this test, there is not much change in tps, but if we it in graph proposed algorithm performs marginally better, and it is linearly increasing with increase in number of threads passed against each algorithm. The above graphs compares performance of algorithms ran against simple strategy mode. And proposed algorithm has performed better in average time, max time and little better for transaction response time (it can be checked from table entries). 37
Thread Count The next set of performance test ran with thread strategy against both algorithms. 5.2.2 Thread Strategy Time (Sec) Original Algorithm Average Time (ms) Proposed Algorithm Original Algorithm Maximum Time (ms) Proposed Algorithm Original Algorithm TPS Proposed Algorithm 10 60 233.04 232.04 404 281 40.66 39.74 20 80 232.5 232.14 337 353 75.8 76 30 100 233.14 238.68 371 347 101.96 111 40 120 233.62 233.8 358 377 131.35 148 50 150 236.08 236.5 380 388 158.63 170 Table 2: Thread Strategy results comparison The table was plotted with graphs with left axis contains average time (ms) v/s right axis as thread count. This test increases the amount of thread from one level to another over the run test duration. The main purpose of this strategy is to find at which thread count the test can attain the highest transaction per second. I ran the test against thread count till 50. 240 238 236 234 232 230 0 10 20 30 40 50 60 Figure 12: Average Time (Thread Mode) Original Algorithm Proposed Algorithm Average time is termed as response time for the client, and from this test it is concluded original algorithm performs marginally better than proposed algorithm, but in real time response time almost looks equal if removed one outlier. 500 400 300 200 100 0 0 10 20 30 40 50 60 Figure 13: Maximum Time (Thread Mode) Original Algorithm Proposed Algorithm 38
This test is to find the outliers and to find whether the outlier s time is reduced or not. From the graph, we can analyze that by increasing the thread count; the maximum time keeps on decreasing, thereby concluding the maximum time reduces with increase in thread when compared with original algorithm [3]. 200 150 100 50 0 0 10 20 30 40 50 60 Figure 14: TPS (Thread Mode) Original Algorithm Proposed Algorithm Thread Count The above test shows proposed algorithm behaves much better than the original algorithm [3]. It can be seen with the increase in number of threads the tps of both algorithms is increasing considerably but the rate at which tps proposed algorithm increased is more than if compared against original algorithm [3]. Higher the tps better is the performance of algorithm, and also with increase in number of threads tps is increasing which also make it more scalable. This tests helps in summarizing that proposed algorithm is much more scalable than original algorithm [3]. The next set of performance test ran with variance strategy against both algorithms. 5.2.3 Variance Strategy (variance 0.5) Time (Sec) Original Algorithm Average Time (ms) Proposed Algorithm Original Algorithm Maximum Time (ms) Proposed Algorithm Original Algorithm TPS Proposed Algorithm 10 60 230.04 232.04 280 290 36.74 36.42 20 80 236.5 234.91 283 280 92.8 92 30 100 234.14 238.68 391 347 69.2 69.45 40 120 235.2 233.8 398 377 84.13 124.01 50 150 239.2 236 422 388 18.93 29.32 Table 3: Variance Strategy results comparison The table was plotted with graphs with left axis contains average time (ms) v/s right axis as thread count. The test was performed with a variance of 0.5 for all cases. 39
240 235 230 225 0 10 20 30 40 50 60 Figure 15: Average Time (Variance Mode) Original Algorithm Proposed Algorithm By running the tests for variance mode against both algorithms, it was seen average response time performs much better for proposed algorithm than the original algorithm [3] with increase in thread count. Response time described here is from client end. Lower response time is one of the features that are most wanted, so by analyzing the result from the graph, it can be said that proposed algorithm behaves better when the number of thread increases which is a nice feature. 600 400 200 0 0 10 20 30 40 50 60 Figure 16: Maximum Times (Variance Mode) Original Algorithm Proposed Algorithm This test is to find the outliers and to find whether the outlier s time is reduced or not. From the graph, we can analyze that by increasing the thread count; the maximum time keeps on decreasing, thereby concluding the maximum time reduces with increase in thread when compared with original algorithm [3]. This test show proposed algorithm behaved better than original [3]. 150 100 50 0 0 10 20 30 40 50 60 Figure 17: TPS (Variance Mode) Original Algorithm Proposed Algorithm Transactions per second (tps) for proposed algorithm are much better than the original algorithm. Higher the tps better is the performance. By increasing the number of threads, tps increases at a much higher rate for proposed algorithm. Performance degrades for both algorithms at thread count of 50. 40
5.2.4 Testing Using HP Load Runner Tool I download HP Load runner trial version [21] to test the performance of algorithms in terms of 99%tile of response time and TPS. 99%tile response time will indicate which algorithm performs always better in terms of response time. VUsers Time (min) 99%tile Response Time (ms) Original Algorithm Proposed Algorithm Original Algorithm TPS Proposed Algorithm 20 5 259 259 76.6 79.05 30 5 261 258 115.27 117.1 40 5 268 263 151.3 157.4 50 5 273 265 182.5 192.3 60 5 293 275 202.4 222.6 70 5 269 267 262.1 275.2 Table 4: HP Load Runner Results Comparison The table was plotted with graphs with left axis contains average time (ms) v/s right axis as VUsers. 300 290 280 270 260 250 0 20 40 60 80 Original Algorithm Proposed Algorithm Figure 18: 99%tile Response Time (HP LoadRunner) The graph was plotted for the 99%tile response time for both algorithms, and it is evident from the graph that proposed algorithm performed much better as compared. 99% of the time, it responded faster by 5-8 ms. Lower the response time, better is the performance considered. 41
300 250 200 150 100 50 0 0 20 40 60 80 Figure 19: TPS (HP LoadRunner) Original Algorithm Proposed Algorithm The graph indicates that tps rate is better for proposed algorithm against original algorithm [3]. The better tps rate means algorithm can process more transactions per second, and is scalable. From the above two graphs, its concluded that proposed algorithm has better tps, and better 99% tile response time, which states proposed algorithm is performing better compared to original algorithm for the same load and criteria. From all the performance tests (simple, thread, variance and HP Load Runner) conducted against both original algorithm [3] and proposed algorithm [3,4], it is seen that proposed algorithm has always been better in respect to increase in tps, lower response time, and low maximum value compared to original algorithm. All the three tests (average time, 99%tile, maximum time, and tps) performed better for proposed algorithm. 42
Chapter 6 6. Conclusion The main objective is this report was to improve the load- balancing algorithm [1] as it used to suffer from performance issues because servers used to get overloaded as it had to maintain the objects in memory in order to determine which the best available server for the client. In this report, concepts of server- weights [4] was added to algorithm [1] so that it can perform better in terms of scalability, tps, response time etc. By adding this concept of static server- weights [4], the objective of the project i.e. find the most effective load- balancing algorithm, based on performance, measured in terms of higher TPS and low response time, for the industry as well as more scalable was evaluated. After analyzing the performance results, its seen that Dynamic Load Balancer with Least Connections and Fastest Response Time with Weight table (proposed Algorithm) [1, 4] perform better than the Dynamic Load Balancer with Least Connections and Fastest Response Time (original algorithm) [1]. The better the performance, more is the transaction being processed in the same time. Load Balancing means routing the requests sent by the client, and getting response in fast manner. The proposed algorithm not only processes more transactions, it also reduces or does not change the average time (round trip time). Not only the proposed algorithm has a better tps, it has better scalability too as seen from the performance results. Performance results also confirmed that 99%tile of all the responses received, the new algorithm performs better. 6.1. Current Status The main objective of this report was to determine which load balancing algorithm should be used which can provide better scalability, higher tps, and better response time. I tried to improve the load balancing technique [1] by adding the concept of weighted servers [4]. Both Load Balancing Techniques were implemented as web service, and comparison of their performance was done. The web service will act as global traffic manager (GTM), which will be called by the client, and algorithm will act as local traffic manager (LTM) routing the requests to the appropriate cloud servers to get the result back to client in minimum time. Algorithms were evaluated based on the performance tests executed on them. Performance tests like simple strategy, thread strategy, variance strategy, and hp load runner tests were executed against load balancing web service for tps, avg response time, maximum time, and 99%tile response time. All the tools that were used for performing performance tests were open source, and free. HP Load runner tool was based on free trial. Based on the analysis of performance results, it was seen that proposed algorithm 43
(Dynamic Load Balancer with Least Connections and Fastest Response Time with Weight table) has performed better than the original algorithm (Dynamic Load Balancer with Least Connections and Fastest Response Time) for criteria of 99%tile response time, tps, max time etc. It has better tps, higher scalability, and response time improves too. 6.2 Future Work The future work for this master s project can be extended to various areas. Some of the few areas are: The proposed load balancing technique uses the concept of static weights. It can be extended to dynamic weights where weights are assigned to the server dynamically based on the server performance. The server, which is performing, better is assigned higher weight as compared to less performing server. This can be achieved by adding monitoring agents on the each server, which monitors the performance. Concept of threads can be used too. Threads will be used in load balancing algorithms where requests sent by client are added in a service pool queue, and thread fetches requests from it, performs the requested operation. It follows the lifecycle of thread. It should be made sure that all the threads are mutually exclusive of each other so that the balancing algorithm does not go in deadlock state. JMS can be used to. Instead of using web service for the load- balancing algorithm, they can be deployed as standalone applications on the servers/instances, which have the concept of JMS, coded in it. JMS is termed as Java Messaging Service. The framework will use JMS as mechanism for the clients to send the messages. Load balancing techniques can be compared against web service v/s JMS. 6.3 Lessons Learned Many things were learnt during implementation of the master s project. Algorithms were developed using JAX- WS, and use SOAP tool for load testing was beneficial. Web service was created from top to bottom approach. I learnt how to write script in order to use HP Load runner tool to call the web service. 44
BIBLIOGRAPHY [1] Azzedine Boukerche, Robson Eduardo De Grande, Dynamic Load Balancing Using Grid Services for HLA- Based Simulations on Large- Scale Distributed Systems, Proceedings of the 2009 13th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications, 2009. [2] Patrick Wendell, Joe Wenjie Jiang, Micheal J. Freedman, and Jennifer Rexford. DONAR : Decentralized Server Selection for Cloud Services. In Proceedings of the ACM SIGCOMM 2010 conference on SIGCOMM 10. ACM, New Yrk, NY, USA, 231-242 [3] Gowtham Kanagaraj, Naveen Shanmugasundaram And Sathish Prakash Adaptive Load Balancing Algorithm Using Service Queue 2nd International Conference on Computer Science and Information Technology (ICCSIT'2012) Singapore April 28-29, 2012 [4] Xiao Qin, Hong Jiang, Yifeng Zhu, David R. Swanson, Boosting Performance for I/OIntensive Workload by Preemptive Job Migrations in a Cluster System, Proceedings of the 15 th Symposium on Computer Architecture and High Performance Computing, 2003. [5] Aameek Singh, Madhukar Korupolu, Dushmanta Mohapatra, ServerStorage Virtualization: Integration and Load Balancing in Web server, Proceedings of the 2008 ACM/IEEE conference on Supercomputing, 2008. [6] Hyotaek Lim Dynamic Load Balancing and Network Monitoring in iata Protocol for Mobile Appliances, Multimedia and Ubiquitous Engineering (MUE), 2010 4th International Conference on 11-13 Aug. 2010 [7] Cardellini, V. Dynamic load balancing on Web- server systems, Internet Computing, IEEE [8] M.Pathan, C. Vecchiola and R.Buyya, Load and proximity aware request- redirection for dynamic load distribution in peering CDNs, in OTM, Nov 2008. [9] Klaithem Al Nuaimi, Nader Mohamed, Mariam Al Nuaimi and Jameela Al- Jaroodi A Survey of Load Balancing in Cloud Computing: Challenges and Algorithm 2012 Second Symposium on Network Cloud Computing and Applications [10] A. Iosup, S. Ostermann, M. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema. Performance analysis of cloud computing services for many- tasks scientific computing. In Parallel and Distributed Systems, IEEE Transactions on, 2011. http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5719609. [11] Allan, D. Farkas, J.; Mansfield, S. Intelligent load balancing for shortest path 45
bridging, 2012 [12] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. A.Patterson, A. Rabkin, I. Stoica, and M. Zaharia. Above the clouds: A berkeley viewof cloud computing. In EECS Department, University of California, Berkeley, 2009. http: //www.eecs.berkeley.edu/pubs/techrpts/2009/eecs- 2009-28.html. [13] T. Harris. Cloud computing services a comparison. In., 2010. http://www.thbs. com/pdfs/comparison%20of%20cloud%20computing%20services.pdf. [14]- http://www.eweek.com/c/a/cloud- Computing/Amazon- Web- Services- Delivers- High- Performance- Computing- Solution- 757277/ [15] How to create Amazon EC2 Instance : http://twiki.org/cgi- bin/view/blog/blogentry201105x2 [16]Building Web services with JAX- WS http://docs.oracle.com/javaee/5/tutorial/doc/bnayl.html [17] SOAP UI Tool: www.w3.org/tr/soap/ [18] Execute Load Test: http://www.soapui.org/load- Testing/loadtest- execution.html [19] M.Pathan, C. Vecchiola and R.Buyya, Load and proximity aware request- redirection for dynamic load distribution in peering CDNs, in OTM, Nov 2008. [20] Amazon Web Service Deliver High Performance: http://www.eweek.com/c/a/cloud- Computing/Amazon- Web- Services- Delivers- High- Performance- Computing- Solution- 757277/ [21] HP Load Runner Tool: http://www8.hp.com/us/en/software- solutions/software.html?compuri=1175451&action=downloadsoftware&jumpid=p s_r11374_us/en/large/eb/loadrnr_googlesemdg&tcmid=tcm:245-865742&originid=8-2ldremr§ionid=alm_ito&k_clickid=ams 125 59697 4cb21060-4918- b928- f3d6-000033d159d7#! [22] Intro to Load Balancing for Developers: https://devcentral.f5.com/articles/intro- to- load- balancing- for- developers- ndash- the- algorithms#.un0jshb29no [23] New To SOA and Web Services: http://www.ibm.com/developerworks/webservices/newto/service.html [24] Amazon Web Services: http://aws.amazon.com/ 46