A Neural Network Approach for Dynamic Load Balancing In Homogeneous Distributed Systems

Transcription

1 A Neural Network Approach for Dynamic Balancing In Homogeneous Distributed Systems Aly E. El-Abd Department of Electronics and computer Engineering Arab Academy for Science and Technology P.O. Box 1029, Alexandria, Egypt Mohamed I. El-Bendary Department of Electronics and computer Engineering Arab Academy for Science and Technology P.O. Box 1029, Alexandria, Egypt Abstract A novel neural-based solution to the problem of dynamic load balancing in homogeneous distributed systems is proposed. The winner-take-all (WTA) neural network model is used for implementing the selection and location policies of a typical dynamic load balancing algorithm. Unlike most of the previous literature that assumed independent tasks, which is not always true, tasks with interprocess communication requirements are considered. All delays due to any usage of the communications network resource are taken into account. A simulation study was carried out to verify the effectiveness of the proposed approach, results were compared against the no load balancing case. Although performance improvements are dependent on the system overall load, load intensity per node, and nature of tasks, the results suggest that it is always beneficial to use load balancing than not at all. 1. Introduction The interest in building distributed systems has experienced a dramatic increase. Such increase was primarily driven by the vast advances and constantly declining costs of hardware technology and the various advantages these systems are known to provide such as resource sharing, high performance, and availability. This interest has spurred a large research activity in this area. One of the critical issues to the performance of distributed systems is the issue of balancing the work load of computational tasks among the different nodes comprising the system. imbalance is observed by the existence of nodes that are highly loaded while others are underloaded or even idle, such situations are harmfull to the system performance specially in terms of the average response time of tasks and the resourcestilization. The problem of load balancing has been tackled along several key dimensions. Static load balancing involves assigning tasks to processors at compile time. The problem has been formulated as an optimal assignment problem of tasks to processors [11,14,16,18]. Alternatively, the assignments can be made dynamically [1,4,5,12,18,20]. Dynamic load balancing involves the reallocation of tasks to processors after their initial assignments, which is done by migrating tasks from the overloaded nodes to other lightly loaded nodes so that load is shared evenly among the system nodes to improve the overall system performance. The problem of optimally making such assignment is NP-complete. Dynamic load balancing algorithms are characterized by the following key components: (i) The transfer policy: which decides when a task should be considered for migration, such decision is dependent on the current load level of the node. (ii) The selection policy: decides which task should be a candidate for migration. (iii) The location policy: decides to which host should a candidate task at an overloaded node be migrated. (iv) The information policy: which is the mechanism by which the load state information is exchanged between nodes, this may be done on periodic or event-driven basis. Since the decisions made by these policies are based on load information which is outdated due to the communication delay incurred in sending the load state update messages, we study the proposed policies taking all delay considerations into account. Recently, the location policy has received considerable attention [7,13]. However, very little on the selection policy have been reported in the open literature. Most of the previous work has chosen to trivialize the selection by either selecting the last task in queue or by selecting a random task. Other selection schemes were based on the service times of the tasks in the queue [21]. This was based on the assumption that all tasks are independent with no interprocess communication requirements. Instead, we consider both categories of tasks to exist, those independent and communicating tasks. The main focus of this work is the selection and location policies of dynamic load balancing algorithms.

2 The model we employ in our approach to implementing the selection and location policies is the Winner-Take-All (WTA) neural network model. This model is a single layer model, the connection weights between neurons are fixed so it is a fixed-weight net. The operation of this type of neural networks is based on competition [2], where neurons aggregate their inputs and the winner is the one with the largest net input, the winner neuron typically reinforces its output signal while simultaneously inhibiting the outputs of all other neurons in the layer. The motivation for using the WTA neural network is its simplicity and ease of implementation. The application of neural networks to the problem of dynamic load balancing has been reported in the literature by [8] where they have used a comparator neural network for the automated learning of load indices to be used by dynamic load balancing policies, although their work gives an insight to the area of application their results were under the unrealistic assumptions of no overheads or delays which are affecting factors of the quality of load balancing decisions. They have also integrated the comparator neural network with a population-based learning technique to automate the learning of load balancing policies[9]. The rest of this paper is organized as follows, section 2 describes the system model and discusses the assumptions concerning the load balancing strategy. Section 3 introduces the proposed approach for the selection and location policies. Section 4 discusses the simulation results and performance evaluation. Section 5 concludes this work. 2. System Model The aim of this section is defining the model of the distributed system under study, the host model, the task model, and the assumptions made Distributed System Model Our notion of a homogeneous distributed system is that of a multicomputer system. Communication between the system hosts is done solely by message passing. By homogeneity we mean that the hosts have the same software and hardware architecture, accordingly, any software component is assumed to be free to reside and being serviced on any host. We assume that the processing power may vary among the different hosts Host Model Each host in the system is assumed to have a computing facility (CPU), memory, and a disk. In developing the logical view of each host, we adopt the formulation by [12]. We assume that each host maintains two queues, the first queue is the active processes queue which houses the tasks that have started execution and have not yet finished. The second queue is the waiting processes queue which houses the tasks that have not yet started execution. Unlike most of the previous work, where the local scheduling policy assumed was the first-come-first-served (FCFS), we assume the local scheduling policy is a round-robin. The reasoning for this is that multiprogramming preemptive systems are much more efficient than their non-preemptive counterparts with a FCFS scheduling policy. Also the round-robin policy has an attractive feature over the FCFS policy which is that the waiting time of a task increases linearly with the task length [8]. Each host maintains load tables and routing tables as numerical values, ready to consume whenever the load balancing is invoked. balancing invocation is upon application arrival. Each host has a dedicated computing facility for executing the load balancing functions Task Model Most of the previous work has assumed that the tasks are independent, but this is not the real case all the way. Actually, jobs are often comprised of tasks that have interprocess communication requirements. A case that corresponds to a wide range of scientific problems. We assume that a job is partitioned at compile-time into several tasks with precedence constraints and explicit message-driven code. Initially, tasks are submitted to the waiting processes queue. A task is created and dispatched to the active processes queue if the following two conditions are met. First, the active processes queue is not at maximum capacity. Second, no precedence constraints are violated. An example of a task graph generated by the compiler is given in fig.(1) where ellipses correspond to tasks and arrows indicate precedence relations. The PARADIGM compilation system[3] developed at the university of Illinois is an example of the assumed parallelizing compiler. We define the following quantities used by the proposed selection and location policies.

3 X4 X2 Figure 1. Task graph example. P ij penalty assigned by task i to task j due to its need for communication with task j. S ij Communication Cost between task i and task j. E h (x) Estimated remaining execution time of task x at host h. T hk (x) Estimated cost of transfer of task x from host h to host k. U h Estimated unfinished work at host h. C hk Communication Cost / packet between hosts h and k. The proposed approach requires the existence of a compile and run-time libraries support for the estimation of execution times, remaining times, penalties, communication costs, and the unfinished work. A methodology that estimates the communication costs at compile time is reported in [3] based on the assumption that the underlying operating system supports communication primitives such as send-message and receive-message. The costs calculated are functions of the number of nodes involved in the communication (node-to-node, broadcast, or multicast) and the message length. Each penalty is the aggregated cost of all communication primitives between task i and task j. Hence, the communication cost between task i at host h and task j at host k, S ij, is given by: S ij = C hk *P ij In case of h=k, S ij is equal to zero. The estimated execution time of each task is obtained at compile-time. The estimated remaining execution time and estimated task transfer cost, say x from host h to host k are obtained at run-time. The estimated unfinished work is calculated at the time of load balancing invocation Assumptions X5 X1 X3 X6 Since the migration of a task after it has started execution incurs an enormous overhead that might outweigh any performance gains of load sharing, we assume that only non-preemptive transfers are allowed. (i.e., only tasks that have not yet started execution are considered for remote execution). The information policy adopted is the event-driven policy. A load state update message is sent by a host upon changing its load condition. The load conditions assumed are lightly loaded, normal load, and heavy loaded. We use two load measures, namely, the instantaneous queue length and the estimated unfinished work. The instantaneous queue length is the number of tasks existing in both the active and waiting processes queues. Using the queue length alone as the load measure and balancing parameter is not sufficient for obtaining good quality load balancing decisions, specially when variance in computing power at different hosts and variance of service times of tasks increase. The estimated unfinished work which is defined as the length of time needed for a host to complete all tasks in both of its queues. We adopt the acceptance policy proposed by [1] and recommended by [6] which is the request-reply policy allowing for wrong allocation decisions to be undone by giving the receiver host the right to refuse accepting more load. This also prevents the underloaded nodes from becoming heavily loaded due to excessive task transfers from several highly loaded nodes. The transfer policy assumed is a threshold policy, a host decides to cooperate in load balancing whenever its load index either falls below a certain threshold (becomes underloaded) or gets higher than an upper bound (becomes overloaded). 3. Proposed Selection and Location Policies This section discusses the proposed selection and location policies based on the winner-take-all (WTA) neural network model. Each of the two policies is mapped to a competitive layer neural network Selection Policy The WTA neural network employed has an architecture as depicted in figure (2). Each neuron represents a task of the waiting processes queue or the active processes queue. Each neuron has an associated bias that is equal to the estimated remaining execution time of the task at the local host. An inhibitory connection weight is set from node X i to node X j which is equal to the worst case communication cost per packet from the local host to any other host times the penalty P ij, which is the worst case S ij.. After the selection competitive layer is formed using the local information, the neurons compete for the right to be ON (fire). The winner neuron will denote the task with the largest difference between the estimated execution time and the interprocess communication cost. The reasoning behind this is the heuristic that the task with the highest execution time is more likely to be worth the transfer

4 overhead, and that the task with minimum interprocess communication requirements would yield a least cost transfer. Hence, the winner identifies the candidate task to be migrated. E 1 (x1) E 1 (x3) X1 X3 S 31 Figure 2. Selection policy neural network Location Policy S 21 S 41 S 32 S 43 X2 X4 S 42 E 1 (x2) E 1 (x4) The WTA neural network used for implementing the location policy has an architecture as shown in figure (3). Each neuron represents a host in the system. Each neuron accepts as input the estimated execution time of the selected task at the host represented by the neuron. Each neuron has an associated bias which is equal to the estimated unfinished work at the designated host. Connection weights are only from the neuron designating the local host to the other neurons representing hosts participating in load balancing. Weights values are equal to the estimated cost of transfer of the candidate task from the local host to each other participating host. All biases, inputs and weights are inhibitory since they represent load, execution and transfer overheads respectively. The location policy neural network is constructed based on the output of the selection layer and the current information concerning the load states in terms of the queue lengths of other hosts. In fact, the number of neurons in the location network is the number of the underloaded hosts of the system at the time of load balancing invocation. A host is eligible for being a receiver if its active processes queue length is less than its maximum length, so that a transferred task may start remote execution as soon as it arrives at the receiver. Once the network is constructed, neurons are allowed to compete for the ON activation level. The winner neuron identifies the candidate host to which the combined cost of execution and communication is minimum in accordance with available information about costs and loads. The candidate host is sent a request message for transferring the candidate task. The candidate host accepts the task by sending an accept message if it can still accommodate more load. If the load state of the candidate host has changed to normal or overload, or is currently waiting for tasks from other hosts that will result in load state change, a reject message is sent back to the local host. Both accept and reject messages will have the new load state information of the candidate host attached to them. In case of an accept message received, the task will be transferred immediately. In case of rejection, the load tables are examined, if there are other hosts that are eligible for participation in load balancing the competition phase is entered again. It should be noted that the winner neuron may be the one representing the local host, in such case the job will be executed locally in spite of the fact that there are other lightly loaded nodes in terms of the queue length alone. Additionally, we have incorporated the worst case communication cost between tasks so that a selected task will either be transferred to a host along the worst communication link or will be placed on a host to which less communication cost exists. Both previously stated features contribute in ensuring that good quality allocation decisions are made most of the time and that task transfer will only take place if it is beneficial. U 1 H1 H3 H4 T 43 (X*) U 4 U 3 Figure 3. Location policy neural net. 4. Performance Evaluation A simulation study was carried out in order to assess the performance of the proposed dynamic load balancing approach. We have used the mean response time as a performance criterion for evaluation. The simulation study assumes service times to be exponentially distributed and arrival rates to be poisson distributed. Although this may not hold in a practical sense, it would be sufficient in giving an overview of the performance[6]. In order to validate the simulation process, results from previous literature have been reproduced. 4.1 Qualitative Analysis U 2 T 41 (X*) H2 T 42 (X*) Qualitative analysis is concerned with the load

5 balancing algorithm properties such as generality, stability and scalability. The proposed selection and location policies are general enough to work with various transfer and information policies. A stable load sharing algorithm must prevent task thrashing by making the appropriate dynamic assignments of tasks to hosts so that the offered load gets serviced within a reasonable finite length of time. Scalability of a load sharing algorithm implies independence of physical system characteristics (e.g. system size, physical topology), and being tolerant to changes of physical resource characteristics (e.g. communication bandwidth and processor speed). Stability is a precondition for scalability [6]. The algorithm employs features that contribute to its stability, namely, the use of the request-reply policy which allows avoiding poor allocation decisions due to outdated information. Also, task thrashing is avoided since tasks are allowed to migrate only once as they are transferred to nodes where they can start execution immediately. The algorithm makes use of state information tables about all the nodes in the system, hence the algorithm depends on the number of nodes in the system so it is not scalable in this sense. Modifications to the algorithm are possible in order to account for scalability but they are not examined here. 4.2 Quantitative Analysis Quantitative analysis is concerned with the performance evaluation of the algorithm at different load conditions. We have modeled each host as an M/M/1 queuing system. The active processes queue has a maximum length of six tasks while the waiting processes queue is assumed to be of infinite capacity. A host is considered lightly loaded if its active processes queue has a length less than four. A host is considered in a normal load condition if its active queue has a length greater than four and less than six. A host is considered overloaded if its active queue is full and more than two tasks are getting queued in its waiting processes queue. The communication costs per packet assumed are shown in table (1). The algorithm is tested under different loads imposed on individual nodes but with the same overall load. We test the algorithm using tasks with interprocess communication for both cases of even and uneven load at the same overall load Light. The input data used for the light load testing is shown in tables(2, 3), for both cases of even and uneven loads. The results compared to the no load balancing case are also shown in figure (4). The numeral 1 on the x-axis denote the uneven load case while the numeral 2 denotes the even load case. The algorithm improves the mean response time of tasks substantially in both cases of even and uneven loads. An improvement of 34.38% is achieved in case of even load while in the case of uneven load the performance gain is around 47.73%. The reasoning behind this is that an uneven load pattern permits more task migration activity to take place due to the existence of several underloaded nodes. Response Time Pattern No Balancing Balancing Figure 4. Response time under light load of % Moderate. In the case of moderate load, two tests are carried out since this load condition is of prime interest. The first test is biased towards the light overall load condition while the second is biased towards the heavy overall load condition. In the first test the overall load was 70%. Results are shown in figure (5). The performance gain attained is around 41.36% for the case of the even load pattern while for the uneven load a performance improvement of 32% is achieved. An interesting result is that biasing the system towards the light load while imposing an uneven load pattern affects the performance gain negatively. This result is interpreted by a close examination of tables(4,5). Table(5) shows the uneven load pattern, although the overall load is still at the beginning of the spectrum of moderate loads (70%), the load distribution at the node is characterized by a high variance in the load intensities of nodes. Also the majority of nodes are at heavy load state (90%). This implies that a contention is expected to occur at the node of 10% intensity since it is highly underloaded. But the algorithm employs a request-reply policy which forces the highly loaded nodes to execute their load tasks locally whenever it can not accommodate more load. This increases the waiting time of tasks spent at the overloaded nodes. In contrast let us examine table(4) which shows the even load pattern case. All nodes are at the moderate load state, and the variance in the load intensities is substantially low. Such pattern allows for less but yet more effective task migration activity to occur. The second test shown in tables (6,7) is at the high end of the moderate loads spectrum. Results shown in figure (6) indicate that a performance improvement of 22.43% is achieved in the uneven load case while in the case of even load the improvement percentage was 21.03%. Clearly, in

6 both cases the performance gain is less than the previous tests due to imposing a higher overall load Response Time Response Time 1 Pattern 2 No Balance Balance Figure 5. Response time under moderate load of 70% Pattern No Balancing Balancing Figure 6. Response time under moderate load of 85% Heavy. The heavy load test is of a special interest because it is always important to get an estimate of the system performance under exceptionally heavy load conditions. Table (8) shows the test at an overall load of 95%. Performance is improved at this condition also which proves the robustness of the proposed load balancing scheme. The performance gain attained is around 21.03%. Common to all the tests is that the system overall throughput is kept nearly the same, this implies that the improvement in the response time does not hinder the system throughput which turns the performance gains achieved by this scheme into low-cost gains. Response Time Pattern No Balancing Balancing Figure 7. Response time under heavy load of 95%. Table 1. Communication costs/packet C12 C13 C14 C15 C23 C24 C25 C34 C35 C Table 2. Light load (41.16%-70%) even load pattern 1 Exponential(5s) Poisson(0.1176) 58.8% 2 Exponential(7s) Poisson(0.1) 70% 3 Exponential(6s) Poisson(0.111) 66% 4 Exponential(5s) Poisson(0.0588) 41.16% 5 Exponential(7s) Poisson(0.125) 62.5% % Table 3. Light load (15%-95%) uneven load pattern 1 Exponential(5s) Poisson(0.03) 15% 2 Exponential(7s) Poisson(0.04) 30% 3 Exponential(6s) Poisson(0.132) 79.23% 4 Exponential(5s) Poisson(0.113) 79.23% 5 Exponential(7s) Poisson(0.19) 95% % Table 4. Moderate load (65%-90%)even load pattern 1 Exponential(5s) Poisson(0.14) 70% 2 Exponential(7s) Poisson(0.093) 65% 3 Exponential(6s) Poisson(0.133) 80% 4 Exponential(5s) Poisson(0.14) 70% 5 Exponential(7s) Poisson(0.093) 65% 70% Table 5. Moderate load (10%-90%)uneven load pattern 1 Exponential(5s) Poisson(0.02) 10% 2 Exponential(7s) Poisson(0.128) 90% 3 Exponential(6s) Poisson(0.15) 90% 4 Exponential(5s) Poisson(0.14) 70% 5 Exponential(7s) Poisson(0.128) 90% 70% Table 6. Moderate load (85%-87.5%)even load pattern 1 Exponential(5s) Poisson(0.175) 87.5% 2 Exponential(7s) Poisson(0.121) 85% 3 Exponential(6s) Poisson(0.133) 80% 4 Exponential(5s) Poisson(0.121) 85% 5 Exponential(7s) Poisson(0.175) 87.5% 85%

7 Table 7. Moderate load (65%-90%)uneven load pattern 1 Exponential(5s) Poisson(0.13) 65% 2 Exponential(7s) Poisson(0.129) 90% 3 Exponential(6s) Poisson(0.15) 90% 4 Exponential(5s) Poisson(0.18) 90% 5 Exponential(7s) Poisson(0.129) 90% 85% Table 8. Heavy load (89%-98%)even load pattern 1 Exponential(5s) Poisson(0.19) 95% 2 Exponential(7s) Poisson(0.14) 98% 3 Exponential(6s) Poisson(0.148) 89% 4 Exponential(5s) Poisson(0.19) 95% 5 Exponential(7s) Poisson(0.14) 98% 95% 5. Conclusion and Future Work A novel solution to the problem of dynamic load balancing in a homogeneous distributed system based on the Winner-Take-All (WTA) neural network model was presented. Two different neural networks were introduced for implementing the selection and location policies of a typical dynamic load balancing algorithm. Both cases of independent tasks and tasks with interprocess communication requirements were considered during load balancing decision making. Also for achieving good quality allocation decisions, all communication costs associated with any use of the communications network resource were accounted for. The mean response time performance measure was found to improve for all cases but with different percentages. Several considerations are now under study. Modifications to the algorithm in order to be scalable, extending the approach to the heterogeneous case, and applying other neural network models specially those with learning capabilities, remain for future work. We hope that our work stimulates more research in this area in the near future. 6. References [1] D. Eager, E. Lazowska, and J. Zoharjan, Adaptive Sharing in Homogeneous Distributed Systems, IEEE Trans. Software Engineering, Vol.12, 1986, PP [3] M. Gupta and P. Banerjee, Compile-Time Estimation of Communication Costs on Multicomputers, Proc. Int l Parallel Processing Symposium, Beverly Hills, CA, March [4] A. Hac and X. Jin, Dynamic Balancing in a Distributed system using a sender-initiated Algorithm, Proc. 13 th Conf. Local Computer Network, [5] C.H. Hsu and J.W.-S. Liu, Dynamic Balancing Algorithms in Homogeneous Distributed Systems, Proc. 6 th Int l Conf. Distributed Computing Systems, [6] O. Kremien and J. Kramer, Methodical Analysis of Adaptive Sharing Algorithms, IEEE Trans. Parallel and Distributed Systems, Vol. 3, No. 6, 1992, PP [7] P. Krueger and N.G. Shivaratri, Adaptive Location Policies for Global Scheduling, IEEE Trans. Software Engineering, Vol. 20, No. 6, 1994, PP [8] P. Mehra and B. Wah, Automated Learning of Work Measures for Balancing on a Distributed System, Proc. Int l Conf. Parallel Processing, CRC Press, August, [9] P. Mehra and B. Wah, Population-Based Learning of Balancing Policies for a Distributed Computer System, Proc. Computing in Aerospace 9 Conf., AIAA, October, [10] R. Mirchandaney, D. Towsley, and J.A. Stankovic, Analysis of the Effects of Delays on Sharing, IEEE Trans. Computer, Vol.38, No.11, 1989, PP [11] G.S. Rao, H.S. Stone, and T.C. Hu, Assignment of tasks in a distributed processing system with limited memory, IEEE Trans. Computer, Vol. c-28, April 1979, PP [12] J.C. Ryou and J.S.K. Wong, A Task Migration Algorithm for Balancing in a Distributed System, Proc. 22 nd Annual Hawaii Int l Conf. System Sciences, Vol. 2, Software Track [13] K.G. Shin and Y. Chang, A Coordinated Location Policy for Sharing in Hypercube-Connected Multicomputers, IEEE Trans. Computers, Vol. 44, No.5, 1995, PP [14] K.G. Shin and M-S Chen, On the Number of Acceptable Task Assignments in Distributed Computing Systems, IEEE Trans. Computers, Vol. 39, No. 1, 1990, PP [15] J.A. Stankovic, Simulations of Three Adaptive, Decentralized controlled, Job Scheduling Algorithms, Computer Networks vol. 8, PP , August [16] H.S. Stone, S.H. Bokhari, Control of distributed processes, IEEE Computer, Vol. 11, 1978, PP [2] L. Fausett, Fundamentals of Neural Networks Architectures, Algorithms, and Applications, Prentice-Hall, inc., 1994.

8 [17]. S. Subramaniam and D.L. Eager, Affinity Scheduling of Unbalanced workloads, Proc. IEEE Supercomputing Conf., [18] A. Tantawi and D. Towsley, Optimal Static Balancing in Distributed Computer Systems, Journal ACM, Vol.32, April 1985, PP [19] Y. Wang and R.J.T. Morris, sharing in distributed Systems, IEEE Trans. Computer, Vol. C-34, No.3, 1985, PP [20] M.J. Zaki, W. Li, and S. Parthasarathy, Customized Dynamic Balancing for a Network of Workstations, Computer Science Dept., The University of Rochester, Technical Report 602, December [21] S. Zhou, A Trace-driven Simulations Study of Dynamic Balancing, IEEE Trans. Software Engineering, Vol.14, No.9, 1988, PP