Classification of Load Balancing in a Distributed System Divya Aaggarwal 1, Vikas Siwach 2 1 M. Tech. Student (CSE) U.I.E.T., MDU, Rohtak (Haryana) 2 Assistant Professor, U.I.E.T., MDU, Rohtak (Haryana) Abstract Load balancing is the process of improving the performance of a parallel and distributed system through a redistribution of load among the processors. The purpose of load balancing in Distributed System is to minimize the response time by transferring the load from overloaded nodes to under loaded-nodes. In this paper, various types of load balancing techniques are discussed that are currently available. The main purpose of this paper is to classify various load balancing algorithms/techniques which helps in future for designing new techniques. Keywords: Distributed System, Load Balancing, Load Balancing s. programming of a distributed system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers. A distributed system can be viewed as a collection of computing and communication resources shared by active users. When the demand for computing power increases the load balancing problem becomes important. 1. Introduction Distributed computing systems have become a natural setting in many environments for business and academia. This is due to the rapid increase in processor and/or memory hungry applications coupled with the advent of low-cost powerful workstations [1].The cost/performance ratio of networks of workstations has been constantly improving. This trend is expected to continue in the near future. The aggregate peak rate of such systems often matches or exceeds the peak rate offered by the fastest parallel computers. Thus, distributed computing systems are a viable and less expensive alternative to parallel computers. However, a serious difficulty in concurrent 100 Figure 1: Distributed System In a typical distributed system setting, tasks arrive at the different nodes in a random fashion. This causes a situation of non-uniform loading across the system nodes to occur. Loading imbalance is observed by the existence of nodes that are highly loaded while others are lightly loaded or even idle. Such situations are harmful to the system performance in terms of the mean response time of tasks and resource utilization [1].
A system [2-3] of distributed computers with tens or hundreds of computers connected by high-speed networks has many advantages over a system that has the same standalone computers. A distributed system provide the resource sharing as one of its major advantages, which provide the better performance and reliability than any other traditional system in the same conditions[4]. optimal resource utilization, throughput, or response time. Using multiple components with load balancing, instead of a single component, may increase reliability through redundancy [29]. Load balancing attempts to maximize system throughput by keeping all processors busy Load balancing is done by migrating tasks from the overloaded nodes to other lightly loaded nodes to improve the overall system performance. In distributed systems more than one processor processing parallel programs. The amount of processing time needed to execute all processes assigned to a processor is called workload of a processor. A system [5] [6] of distributed computers with tens or hundreds of computers connected by high speed networks has many advantages over a system that has the same standalone computers. A distributed system provide the resource sharing as one of its major advantages, which provide the better Load Balancing based on the idea of migration of excess load from heavily loaded node to lightly loaded ones. The problem starts with to determine when to migrate a process or task [30].This solution is typically based on local load situation: for example, a simple procedure may be the comparison of the load between various nodes with those of the neighboring node and a determination of the node to which the task is to be migrated. performance and reliability than any other traditional system in the same conditions. One of the research issues in distributed systems is the development of effective techniques for distributing workload on multiple processors. The main goal is to distribute the jobs among processors to maximize throughput, maintain stability, resource utilization and should be fault tolerant in nature. In section 2 we discussed load balancing and its classification in brief. Categorizations of different load balancing algorithms are enlisted in section3. Jobs in a distributed system can be divided into different classes based on their resource usage characteristics and ownership. For example the jobs that belong to a single user can form a class. Alternatively, we can distinguish different classes of jobs by their execution times [31]. There are three typical approaches to load balancing problem in single class job distributed systems: a) Global approach: In this case there is only one decision maker that optimizes the response time of the entire system over all jobs and the 2. Load Balancing In computer networking, load balancing is a technique to spread work between two or more computers, network links, CPUs, hard drives, or other resources, in order to get 101 operating point is called social optimum. This is the classical approach and has been studied extensively using different techniques such as
nonlinear optimization [7, 8] and polymatroid optimization [9]. b) Cooperative approach: In this case there are several decision makers (e.g. jobs, computers) that cooperate in making the decisions such that each of them will operate at its optimum. Decision makers have complete freedom of preplay communication to make joint agreements about their operating points. This situation can be modeled as a cooperative game and game theory offers a suitable modeling framework [10]. c) Non-cooperative approach: In this case each of infinitely many jobs optimizes its own response time independently of the others and they all eventually reach equilibrium. This situation can be viewed as a non-cooperative game among jobs. The equilibrium is called Wardrop equilibrium [11]. At the Wardrop equilibrium a job cannot receive any further benefit by changing its own decision. If the number of jobs are finite the Wardrop equilibrium reduces to the well-known Nash equilibrium [10]. always executed on the node to which it is assigned that is static load balancing methods are non-preemptive. A general disadvantage of all static schemes is that the final selection of a host for process allocation is made when the process is created and cannot be changed during process execution to make changes in the system load [12]. Major load balancing algorithms are Round Robin [13] and Randomized s[14], Central Manager [15] and Threshold[12,16]. B. Dynamic Load-Balancing It differs from static algorithms in that the workload is distributed among the nodes at runtime. The master assigns new processes to the slaves based on the new information collected [17,18]. Unlike static algorithms, dynamic algorithms allocate processes dynamically when one of the processors becomes under loaded. Instead, they are buffered in the queue on the main host and allocated dynamically upon requests from remote hosts[12]. This method is consisted of Central Queue and Local 2.1 Types of Load Balancing There are two types of load balancing algorithms: A. Static Load-Balancing In this method, the performance of the nodes is determined at the beginning of execution. Then depending upon their performance the workload is distributed in the start by the master node. The slave processors calculate their allocated work and submit their result to the master. A task is 102 Queue [19]. Load balancing algorithms work on the principle that in which situation workload is assigned, during compile time or at runtime. Comparison shows that static load balancing algorithms are more stable compare to dynamic. It is also ease to predict the behavior of static, but at the same time, dynamic distributed algorithms are always considered better than static algorithms [12].
Processing power does not mean only the processing speed of Processor; it includes the overall configuration of node. In applications with constant workloads, static load balancing can be used as a pre-processor to the computation. Other applications, such as adaptive finite element methods, have workloads that are unpredictable or change during the computation; such applications require dynamic load balancers that adjust the decomposition as the computation proceeds. Load balancing algorithms vary in their complexity where complexity is measured by the amount of communication used to approximate the least loaded node. Static algorithms collect no information and make probabilistic balancing decisions, while dynamic algorithms collect varying amounts of state information to make their decisions. The most significant parameter of the system was found to be the cost of transferring a job from one node to another. It is this cost that limits the dynamic Load balancing increase the performance of each host. Small jobs will not suffer from starvation. Since load imbalance leads directly to processor idle times, high efficiency can only be achieved if the computational load is evenly balanced among the processors. Generally a load balancing scheme consists of three phases-1.information collection, 2.Decision-Making and 3.Data migration. 1. Information collection: During this phase the load balancer gathers the information of workload distribution and the state of computing environment and detects whether there is a load imbalance. 2. Decision-Making: This phase focuses on calculating an optimal data distribution. 3. Data migration: This phase transfer the excess amount of workload from overloaded processor to under loaded ones. algorithms, but at the high end of complexity are the dynamic algorithms which do collect varying amounts of information. Potentially the more information an algorithm can collect the better decision it will make. The problem with the complex balancing algorithms is that they cannot keep up with the rapidly changing state information of the system. The best they can do is to make their decisions based on information passed between two nodes i.e. the node itself and the node that sent it the information concerning its queue length. Load balancing is done basically to do following benefits. Load balancing reduces mean job response time under job transfer overhead. 103 3. Categorization of Different Load Balancing s A number of Dynamic load balancing algorithms are existing for different systems; a small description is presented for each algorithm, followed by a detailed classification. 3.1 Decision and Migration based algorithms [25],[26],[8] These algorithms are classified as fallows. 3.1.1. Local Decision and Local Migration (LDLM): In this strategy a processor distributes some of his load to its neighbors after each fixed time interval. This is a LDLM because the
decision to migrate a load unit is done purely local. The receiver of a load is also a direct neighbor. The balancing is initiated by a processing unit which sends a load unit. We implemented this strategy after x iterations of the simulator y load units are sent to random neighbors. 3. 1.2. Direct neighborhood (d-n): If the local load increased by more than Up percent or decrease by more than Down percent, actual value is broadcasted to direct neighbors. if the load of a processing element exceeds that of its least neighbor load by more than d percent, then it sends one unit to that neighbor. 3.1.3. Local Decision and Global Migration (LDGM): In this strategy the load units are migrated over the whole network to a randomly chosen processor. regular communications. In order to propagate the work over the entire system, the window is shifted (slightly moved so that they overlap only a part of the old domain) for the next balancing phase. 3.4 The X-Tiling [13]: Similar to Tiling algorithm but extra links are added to the current topology of the processor to form a hypercube topology. 3.5 The Rake [13]: It uses only regular communications with processors in the neighborhood set. Firstly, the average load is processed and broadcasted. In the first transfer phase, during p iterations, each processor sends to its right neighbor the data over the average workload It uses only regular communications with processors in the neighborhood set. Firstly, the average load is processed. In the second transfer phase, during the extra workloads, each 3.2 The Random [16]: Each time an element (task) is created on a processor, it is sent on a randomly selected node anywhere in the system. For each node, the expectation to receive a part of the load is the same regardless of its place in the system. 3.3 The Tiling (Direct Neighborhood DN) [13]: It divides the system in small and disjointed sub-domains of processors called windows. A perfect load balancing is realized in each window using 104 processor sends to its right the work over the average workload + 1. 3.6 The Pre-Computed Sliding [13]: It is an improvement of the Rake algorithm. Instead of transferring data over the average workload of the system, it computes the minimal number of data exchanges needed to balance the load of the system. Unlike the Rake algorithm, it may send data in two directions. 3.7 The Average Neighbor [17], [20]: The architecture is made of
islands. An island is made of a center processor and all the processors in its neighborhood. It works on the load balancing every node in the island. The partial overlapping allows the load to propagate. 3.8 The Direct Neighborhood Repeated [21]: Once a sender-receiver couple is established, the migrating load can move from the sender to the receiver. In its turn, the receiver can have an even less neighborhood. The receiver is allowed to directly forward the migrating load to the less loaded nodes. Load migration stops when there are no more useful transfers. 3.9 The Central [11], [28]: Firstly, the average workload is computed and broadcasted to every processor in the system. Then, the processors are classified into 3 classes: idle, overloaded, and the others. The algorithm tries to match each overloaded node with an idle peer. Table 1: : Various Load Balancing s S. Description No. 1. Decision and There are three types: Migration a) Local decision and based algorithm local migration b) Direct neighborhood c) Local decision and global migration 2. Random In this, each time an element is created on processor and sent randomly selected node anywhere in the system. 3. Tiling Directed neighborhood A perfect load balancing is realized in each window using regular communications. 4. X-Tiling 5. Rake 6. Pre-Computed Sliding 7. Average Neighbor 8. Direct Neighborhood Repeated 9. Central 4. Conclusion In this extra links are added to the current topology of the processor to form a hypercube topology. It uses only regular communications with processors in the neighborhood set. It computes the minimum no. of data exchanges needed to balance load of the system. It works on the load balancing every node in the island. The partial overlapping allows the load to propagate. In this receiver is allowed to directly forward the migrating load to the less loaded nodes. In this average workload is computed and broadcasted to every processor in the system. The main aim in distributed system is to execute the process at minimum cost i.e. time is most important factor can be considered in cost calculation. Load balancing is the task of distribution of application tasks to different processors in an efficient manner to minimize program execution time. It involves assigning work to each processor proportional to its computing power, hence minimizing the idle time and at the same time overload of 105
work to the processor. The Classification is considered helpful for designers to compare different load-balancing algorithms and design new ones tailored for their needs. In the future, we intend to develop a framework for applications with load balancing that utilizes this Classification and helps the designer to make his own algorithm. References [1] A. E. El-Abd, "Load balancing in distributed computing systems using fuzzy expert systems," presented at International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science (TCSET 2002), Lviv-Slavsko, Ukraine, 2002. [2] S. Malik, "Dynamic Load Balancing in a Network of Workstation," 19 November 2000 2000. [3] D. L. Eager, E. D. Lazowska, and J. Zahorjan, "Adaptive load sharing in homogeneous distributed systems," IEEE Trans. Softw. Eng., vol. 12, pp. 662-675, 1986. [4] S. Sharma, S. Singh, and M. Sharma, "Performance Analysis of Load Balancing s," World Academy of Science, Engineering and Technology, vol. 38, 2008. [5] S. Malik, Dynamic Load Balancing in a Network of Workstation, 95.515 Research Report, 19 November, 2000. [6] Derek L. Eager, Edward D. Lazowska, John Zahorjan, Adaptive load sharing in homogeneous distributed systems, IEEE Transactions on Software Engineering, v.12 n.5, p.662-675, May 1986. [7] X. Tang and S. T. Chanson. Optimizing static job scheduling in a network of heterogeneous computers. In Proc. of the Intl. Conf. on Parallel Processing, pages 373 382, August 2000. [8] A. N. Tantawi and D. Towsley. Optimal static load balancing in distributed computer systems. Journal of the ACM, 32(2):445 465, April 1985. [9] K. W. Ross and D. D. Yao. Optimal load balancing and scheduling in a distributed computer system. Journal of the ACM, 38(3):676 690, July 1991. [10] D. Fudenberg and J. Tirole. Game Theory. The MIT Press, 1994. [11] H. Kameda, J. Li, C. Kim, and Y. Zhang. Optimal Load Balancing in Distributed Computer Systems. Springer Verlag, London, 1997. [12] S. Sharma, S. Singh, and M. Sharma, "Performance Analysis of Load Balancing s," World Academy of Science, Engineering and Technology, vol. 38, 2008. [13] Z. Xu and R. Huang, "Performance Study of Load Balancing s in Distributed Web Server Systems " CS213 Parallel and Distributed Processing Project Report. [14] R. Motwani and P. Raghavan, "Randomized algorithms," ACM Comput. Surv., vol. 28, pp. 33-37, 1996. [15] P. L. McEntire, J. G. O'Reilly, and R. E. Larson, Distributed Computing: Concepts and Implementations. New York: IEEE Press, 1984. [16] W. I. Kim and C. S. Kang, "An adaptive soft handover algorithm for traffic-load shedding in the WCDMA mobile communication system," presented at WCNC'2003, 2003. [17] S. Malik, "Dynamic Load Balancing in a Network of Workstation," 19 November 2000 2000. [18] Y.-T. Wang and A.-R. J. T. Morris, "Load Sharing in Distributed Systems," IEEE Transactions on Computers, vol. 34, pp. 204-217, 1985. 106
[19] W. Leinberger, G. Karypis, and V. Kumar, "Load Balancing Across Near-Homogeneous Multi-Resource Servers," presented at s,cancun, Mexico, 2000. [20] S.Zhou, A Trace Driven Study Of Load balancing, IEEE Trans.On Software Engineering,Vol.14,no.9,1988,pp,1327-1341. [21] F. Ramme, Lastausgleichsverfahren in Vertileilten Systemen, Master Thesis,University of Paderborn.1990. [22] Powley, C., Ferguson, C. and Korf, R. E.: Depth-First Heuristic Search on a SIMD Machine. Artificial Intelligence, vol. 60 (1993) 199-242. [23] Subramanian, R. and Scherson, I.: An Analysis of Diffusive Load Balancing. Proceedings of Sixt h Annual ACM Symposium on Parallel s and Architectures, (June 1994) 220-225. [24] Dekeyser, J. L., Fonlupt, C. and Marquet, P.: Analysis of Synchronous Dynamic Load Balancing algorithms, Parallel Computing: State-of-the Art Perspective (ParCo'95), volume 11 of Advances in Parallel Computing, pages 455--462, Gent, Belgium ( September 1995) [25] Saletore, V. A.: A distributive and adaptive dynamic load balancing scheme for parallel processing of medium- grain tasks. Proceedings of the 5th Distributed Memory Conference (April 1990) 995-999. [26] Willebeek- LeMair, M.H. and Reeves, A.P.: Strategies for dynamic load balancing on highly parallel computers. IEEE Trans. on parallel and distributed systems, vol. 4, No. 9 (Sept. 1993). [27] Corradi, A., Leonardi, L. and Zambonelli, F.: Diffusive load-balancing policies for dynamic applications. IEEE Concurrency Parallel, Distributed and Mobile Computing (January-March 1999) 22-31. [28] Hillis, W.D.: The Connection Machine. MIT press, 1985. [29] Abbas Karimi, Faraneh Zarafshan, Adznan b. Jantan, A New Fuzzy Approach for Dynamic Load Balancing, (IJCSIS) International Journal of Computer Science and Information Security, Vol. 6, No. 1, 2009 [30] Parveen Jain, Daya Gupta, An for Dynamic Load Balancing in Distributed Systems with Multiple Supporting Nodes by Exploiting the Interrupt Service, International Journal of Recent Trends in Engineering, Vol 1, No. 1, May 2009. [31] Daniel Grosu, Anthony T. Chronopoulos Ming-Ying Leung, Load Balancing in Distributed Systems: An Approach Using Cooperative Games., NASA NAG 2-1383 (1999-2001) 107