A Novel Load Balancing Algorithms in Grid Computing

A Novel Load Balancing Algorithms in Grid Computing Shikha Gautam M.Tech. Student Computer Science SITM LKO Abhay Tripathi Assistant Professor Computer Science SITM LKO Abstract: The Grid is emerging as a wide-scale distributed computing infrastructure that promises to support resource sharing and coordinated problem solving in dynamic multi-institutional Virtual Organizations. The idea is similar to the former Meta Computing where the focus was limited to computation resources, whereas Grid computing takes a broader approach. On one hand, Grid computing provides the user with access to locally unavailable resource types. On the other hand, there is the expectation that a large number of resources are available. A computational Grid is the cooperation of distributed computer systems where user jobs can be executed on either local or remote computer systems. The main aim of the paper is to design an architectural framework to propose decentralized, scalable, adaptive, and distributed algorithms for load balancing across resources for data-intensive computations on Grid environments using job migration algorithms which is Load Balancing Algorithm. Keywords- Grid Computing, Load Balancing, Load balancing on Arrival, Migration. 1. Introduction: In a centralized system, the load balancing is done only by a single processor. Such algorithms are bound to be less reliable than decentralized algorithms, where load scheduling is done by many, if not all, processors in the system. However, decentralized algorithms have the problem of communication overheads incurred by frequent information exchange between processors. Grid computing is a distributed infrastructure which allows large scale resource sharing and system integration. Load balancing is the most important factor to improve the efficiency and performance of a multiple nodes in a grid based distributed network. An independent program or a partitioned module of a parallel task or program is called as workload. Grid connected computers are basically a combination of computing resources applied to a common task, usually to a scientific, technical or business problem that requires a major number of processing cycles or the need to process huge amount of data. In a grid some nodes may be heavily loaded while some may be idle. Therefore, load balancing is the problem of distributing workload among physically separated nodes during run time. Workload distribution is carried out in such a way that a set of independent tasks are distributed among all the computing nodes of the grid so that the 81

jobs are uniformly distributed and none of the nodes are overloaded or under loaded. A load balancing algorithm improves the performance of the system. However, the degree of improvement in the performance of the system depends not only on the specific algorithm used but also on the degree of uneven distribution of load over the nodes. Many scheduling and load balancing solutions have been proposed for conventional distributed computing systems. Dynamic decentralized approach is one of the main load balancing strategies which considers the run time environment before distribution of jobs among the nodes in a grid. The dynamic decentralized approach is preferred because of the varying nature of the elements of the grid in capacity or number during run time and also be heterogeneous in nature giving rise to different loading circumstances. In this paper we have present a new load balancing algorithms for the grid based on various different metrics such as load balancing time, average resource utilization rate, communication overhead, reliability, stability, fault tolerance and scalability. 2.1 Load - Balancing Algorithms: A Simple Classification Many different load balancing algorithms are described in the literature. However, most of these descriptions are presented in a mixture of text, drawings and pseudo-code, using inconsistent terminology details. Readers ability to evaluate and compare the various algorithms is severely impaired by the absence of a common reference framework. The concepts used to classify the algorithms are also useful for the methodical design and analysis of new loadbalancing algorithms. This section presents a simple classification of load balancing algorithms most relevant to this research. 2.1.1 Static versus Dynamic: Load balancing could be done statically at compile-time or, dynamically, at run-time. Static load-balancing algorithms assume that a priori information about all of the characteristics of the jobs; the computing nodes and the communication network are known. Load-balancing decisions are made deterministically or probabilistically at compile time, and remain constant during run-time. The static approach is attractive because of its simplicity and the minimized run-time overhead. However, the static approach cannot respond to a dynamic runtime environment, and may lead to load imbalance on some nodes and significantly increase the job response time. Dynamic load-balancing policies attempt to dynamically balance the workload reflecting the current system state, and are therefore thought to be able to further improve system performance. Thus, compared with static ones, dynamic load-balancing policies are thought to be better able to respond to system changes and to avoid states that result in poor performance. 2.1.2 Non-Preemptive Versus Preemptive: Dynamic load-balancing policies may be either non-preemptive or preemptive. A no preemptive load-balancing policy assigns a newly arriving job to what appears at that moment to be the best node. Once the job execution begins, it is not moved, even if its run-time characteristics, or the run-time characteristics of any other jobs, are changed after assigning the job in such a way as to cause the nodes to become much unbalanced. Non-preemptive load-balancing policies can be applied to any distributed system; however, they are particularly suited to loosely coupled systems, which have relatively low-speed internodes communication and tend to consist of 82

performance heterogeneous nodes. By contrast, a preemptive load-balancing policy allows load-balancing whenever the imbalance appears in the workloads among nodes. If a job that should be transferred to a new node is in the course of execution, it will continue at the new node. Migration allows these imbalances to be corrected. However, to migrate a job in execution is much more complex and requires considerable overheads. If the preemptive policies were attempted in a loosely coupled large-scale system, the system performance would probably suffer significantly more, since there would be a large number of messages generated, which would congest the communication system. The preemptive policies are suitable only for tightly coupled distributed systems, in which the processing nodes are homogeneous and are connected by a high-speed low-latency interconnect. Many studies have also shown that job migration is often difficult in practice, the operation is generally expensive in most systems, and there are no significant benefits of such a mechanism over those offered by non-migratory counterparts. Hence, we consider only non- preemptive loadbalancing strategies. 2.1.3 Site-Level versus Grid-Level: When a job arrives at a site, the loadbalancing system of the site will analyze the load situation of every node in the site and will select a node to run the job. Even though the site is heavily loaded, each job must queue in the site and wait to be processed. We classify this kind of loadbalancing as site-level load-balancing, for which the objective is to optimize the system performance in a single site. Many traditional load-balancing algorithms fall in the category of site-level. On the contrary, if a site lacks sufficient resources to complete the newly arriving tasks, or the site is heavily loaded, the load-balancing system of the site will transfer some tasks to other sites, and will increase the system throughput and resource utilization in multiple sites. We call this load-balancing as grid-level load-balancing. The focus of this paper is on grid-level load-balancing. 2.1.4 Centralized versus Distributed: Load-balancing policies can be classified as centralized or distributed. Centralized policies may be considered as a system with only one load-balancing decision maker. Arriving jobs to the system are sent to this load-balancing decision maker, which distributes jobs to different processing nodes. The centralized policies have the advantages of easy information collection about job arrivals and departures, and natural implementation that employs the server-client model of distributed processing. On the other hand, distributed policies delegate job distribution decisions to individual nodes. Usually, each node accepts the local job arrivals and makes decisions to send them to other nodes on the basis of its own partial or global information on the system load distribution. It appears that this policy is closely related to the individually o optimizes its own cost optimal policy, in that each job independently of the others. The distributed load-balancing is widely used to handle imperfect system load information. There are two kinds of hybrid models. One is a combination of fully centralized and distributed algorithms. The other is a hierarchical model, which combines partially centralized and distributed algorithms to overcome some of the limits of fully Centralized algorithms. The first model is applicable only for small-scale distributed systems; the latter still has fault-tolerance problems, due to single point of failure in a set of manager nodes of clusters. The hierarchical model can be simplified as two- 83

level if the set of manager nodes are organized in a fully distributed style. 3. Existing Load Balancing Algorithm: Two classes of well-known dynamic and distributed load-balancing algorithms are presented in this section. The focus is on the load-balancing algorithms utilizing partial information to make decision. Although some algorithms are initially presented for parallel computers, they are applicable in a distributed computing system with more or less deficiencies. Thus, these are also introduced here. Most load-balancing policies execute two activities that require communications: distribute its own load information and collect other nodes information and transfer tasks. If each node is required to interact with other nodes, it will have to use mechanisms such as broadcast, global gathering, long-distance communication which are not scalable and create intolerable overhead or congestion in systems with a large number of nodes. To reduce this overhead, in many policies, a node only exchange information and transfer tasks to its physical and/or logical neighbors. These are usually called neighbor-based load-balancing algorithms. Clustering is another technique to tackle the problem. The nodes can be partitioned into clusters based on network transfer delay where load-balancing operates on two-level: intra-cluster and inter-cluster via cluster managers or brokers. These are usually called cluster-based loadbalancing algorithms. We will give corresponding discussion to these two classes of algorithms below. 3.1 Neighbors-Based Load-Balancing Algorithms: The neighbors-based approach is a dynamic load-balancing technique that allows the nodes to communicate and transfer tasks with their neighbors only. Each node balances the workload with its neighbors so that the whole system will be balanced after a number of iterations. Since this technique does not require a global coordinator, it is inherently local, fault tolerant and scalable. Hence, this approach is a natural choice for load-balancing in a highly dynamic environment. 3.2 Cluster Based Load-Balancing Algorithms: In accordance with the structure of the cluster model, we distinguish between two load balancing levels: Intra-cluster (Interworker nodes) and Inter-clusters (Intra- Grid) load balancing. 3.2.1 Intra-Cluster Load Balancing: Depending on its current load, each cluster manager decides to start a load balancing operation. In this case, the cluster manager tries, in priority, to balance its workload among its worker nodes. At this level, communication costs are not taken into account in the task transfer since the worker nodes of the same cluster are interconnected by a LAN network, of which communication cost is constant. 3.2.2 Inter-Clusters Load Balancing: The load balancing at this level is used if a cluster manager fails to balance its workload among its associated nodes. If we have a case, each overloaded cluster manager transfers tasks from its overloaded worker nodes to under loaded clusters. In contrast to the intra-cluster level, we should consider the communication cost among clusters. Knowing the global state of each cluster, the overloaded cluster manager can distribute its overloaded tasks between under-loaded 84

clusters. The chosen under-loaded clusters are those that need minimal communication cost for transferring tasks from overloaded clusters. A task can be transferred only if the sum of its latency in the source cluster and cost transfer is lower than its latency on the receiver cluster. This assumption will avoid making useless task migration. 4. Algorithm Implementation: 4.1 Load Balancing On Arrival (LBA): In this approach, each processor P i calculates its status parameters, which are the estimated arrival rate, service rate, and load at every status exchange period T s. This information gets exchanged to every buddy processor in the buddy set. On every job arrival, processor P i will calculate the expected finish time of job j on buddy processor P k by estimating the load on P k at time t=cst+ t j c. For this estimation, P i will calculate the expected number of arrivals and departures for buddy processor P k for j time period t=cst+ t c t n-1, where T n-1 is the last status exchange instant. Algorithm: Load balancing on Arrival Input: Inter-arrival Time, Jobs, status exchange period Output: Job Migration STEPS 1. Estimate Arrival rate (λ i ) and Service Rate (μ i ) 2. λ i (T n-1 )=α. λ i (T n-2 )+(1- α).(a i (T s )/T s ) 3. μ i (T n-1 )=β. μ i (T n-2 )+(1-β).(D i (T s )/T s ) //where α and β are arrival rate estimation factor, A i is actual number of jobs arrival, D i is actual number of jobs departure, and T s is status exchange period. 4. Exchange information with all processor 5. Count Arrival of new jobs j in processor i 6. If (processor=idle_state) 7. Start processing job j 8. Else 9. If (T j migration< Migration_Limit) 10.Estimate EFT ij 11. EFT jk =Q i (CST)/ μ i (T n-1 )+ERT ji 12. For all processor, 13. Estimate EFT jk max(l k,i (CST)+EA(t j c )- EDk(t j c ))/μ k (T n-1 )+ERT jk. //Q i is number of jobs waiting in queue //CST: Current System Time j j 14. Substitute t c =t=cst+ t c T n-1 for first term 15. If(EFT jk <EFT ji ) 16. Initiation job migration on processor k 17. Else 18. Put job j on waiting queue of processor i. 19. Else 20. Put job j on waiting queue for processor i. END. 5. Conclusions The proposed system has addressed issues that are imperative to Grid computing environments by introducing job migration algorithms. The proposed algorithms differ within the manner load balancing is disbursed and is shown to be cost effective in minimizing the response time on Grid environments. The algorithm is enhanced for large-scale systems to take into account the job migration value, resource heterogeneity, and network heterogeneity when load balancing is considered. The algorithm is applicable to small-scale systems, performs load balancing by estimating the expected end time of employment on individual processors on every job arrival to estimate system parameters like the job arrival rate, processing rate, and load on the processor and balance the load by migrating jobs to individual processors by considering job transfer value, resource heterogeneity, and network heterogeneity. In this paper, we presented decentralized, scalable, adaptive, and distributed algorithms for load 85

balancing across resources for data-intensive computations on Grid environments. REFERENCES [1] Nadia Ranaldo, Giancarlo Tretola, and Eugenio Zimeo A Scheduler for a Multiparadigm Grid Environment INRIA Sophia-Antipolis - Universit de Nice - CNRS/I3Se 2004, Route des Lucioles, BP 93 FR-06902 Sophia Antipolis, France. [2] Mrs. Sharada Patil, Prof. Dr. Arpita Gopal Comparison of Cluster Scheduling Mechanism using Workload and System Parameters International Journal of Computer Science and Application ISSN: 0974-0767. [3] A. Abed, G. Oz, A. Kostin, Competetion based Load Balancing for Dstributed Systems, Proceedings of the seventh IEEE International Symposium on Computer Networks (ISCN 06). [4] K. Lu, R. Subrata, A. Zomaya, An Efficient Load Balancing Algorithm for Heterogeneous Grid Systems considering Desirability of Grid Sites, 25th IEEE International Conference on performance, Computing and Communication, April 2006. [5] A. Singh; An Efficient Load Balancing Algorithm for Grid Computing using Mobile Agent, International Journal of Engineering Science & Technology (IJEST), June-2011. [6] Ian Foster, Carl Kesselman Steven Tuecke, The Anatomy of the Grid Enabling Scalable Virtual Organizations, Intl J. Supercomputer Applications, 2001. [7] Hans-Ulrich Heiss and Michael Schmitz, Decentralized Dynamic Load Balancing: The Particles Approach. [8] Rajkumar Buyya, Manzur Murshed, GridSim: A toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing, Concurrency and Computation: Practice and Experience, Vol. 14, No. 13, 2002, 1175-1220. [9] Grid Workloads Archive, http://gwa.ewi.tudelft.nl [10] B. Yagoubi and Y. Slimani Task Load Balancing Strategy for Grid Computing Journal of Computer Science 3 (3): 186-194, 2007 ISSN 1546-9239, 2007 Science Publications [11] Belabbas Yagoubi and Meriem Medebber A Load Balancing Model for Grid Environment 1-4244-1364-8/07 2007 IEEE [12] Sandeep Sharma, Sarabjit Singh, and Meenakshi Sharma Performance Analysis of Load Balancing Algorithms World Academy of Science, Engineering and Technology 2008 [13] S.Ayyasamy and S.N. Sivanandam A Cluster Based Replication Architecture for Load Balancing in Peer-to-Peer Content Distribution International Journal of Computer Networks & Communications (IJCNC) Vol.2, No.5, September 2010. [14] Igor Grudenic, "Scheduling Algorithms and Support Tools for Parallel Systems", IEEE, July 2012. [15] JOHANSSON H., STEENSLAND J., A performance characterization of load balancing algorithms for parallel SAMR applications, Tech. Report 2006-047, Department of Information Technology, Uppsala University, 2006. 86