DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS

DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS

DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS by Belal Ahmad Ibraheem Nwiran Dr. Ali Shatnawi Thesis submitted in partial fulfillment of the requirements for the degree of M.Sc. in Computer Engineering At The Faculty of Graduate Studies Jordan University of Science and Technology August, 2009

DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS by Belal Ahmad Ibraheem Nwiran Signature of Author Committee Member Dr. Ali Shatnawi (Chairman) Dr. Raed Bani Hani (Member) Dr. Sameh Ghwanmeh (External Examiner) Signature and Date August, 2009

DEDICATION To My Parents, Wife, and my friend Mohammad i

ACKNOWLEDGMENTS To my supervisor, teachers, and the supervisory committee ii

TABLE OF CONTENTS Title DEDICATION ACKNOELEDGEMENTS TABLE OF CONTENTS LIST OF FIGURES LIST OF TABLES LIST OF APPENDICES ABSTRACT Chapter One: Introduction 1.1 Graph Theory 1.1.1 Graph Definition 1.1.2 Graph Theory Applications 1.2 Load Balancing Problem 1.3 Load Balancing Classification 1.4 Dynamic Load Balancing Classification 1.5 Example Systems 1.5.1 NOW MOSIX 1.5.2 Condor 1.5.3 Sprite Chapter Two: Related Work 2.1 Load Balancing Algorithm for Heterogeneous Grid Systems 2.2 Dynamic Load-Balancing Using Genetic Algorithms Page i ii iii v ix xi xii 1 1 1 4 4 7 9 11 11 13 14 17 17 19 iii

2.3 Dynamic Rate of Change Load Balancing (RoC-LB) Algorithm 2.4 Loadserver Algorithm 2.5 The Generalized Dimension Exchange Method 2.6 An Optimal Dynamic Load Balancing Algorithm 2.7 Clustering-Based Dynamic Load Balancing Algorithm 2.8 Parallel Sorting with Dynamic Redistribution 2.9 Symmetrically initiated adaptive algorithm 2.10 Flexible load sharing Chapter Three: Approach 3.1 Problem Statement 3.2 Algorithm 3.2.1 Star Center Algorithm 3.2.2 Algorithm Elements Chapter Four: Experiments and Results 4.1 Experiments with Static Loads Generation 4.1.1 Experiments with static load generation and a fixed number of node 4.1.2 Experiments with static load generation and different numbers of nodes 4.2 Experiments with Dynamic Loads Generation 4.2.1 Experiments with dynamic load generation and a fixed number of node 4.2.2 Experiments with dynamic load generation and different numbers of nodes Chapter Five: Conclusion References Appendix A Arabic Abstract 22 23 24 25 26 30 32 33 37 37 39 39 40 54 56 57 70 76 76 91 99 100 103 105 iv

LIST OF FIGURES Figure Description Page 1. Graph example 3 2. The adjacency matrix of the graph shown in figure 1 4 3. Casavant and Kuhl s Load Balancing Classification 7 4. Osman and Ammar s Dynamic Load Balancing Classification 9 5. Performance of Sender-Initiated vs. Receiver-Initiated algorithms 10 6. Genetic Algorithm string example of scheduling sample 20 7. Overview of a web cluster 27 8. Simple system graph 38 9. The main elements of the SCLB algorithm 40 10. Low state algorithm of the Star Center algorithm 47 11. High state algorithm of the Star Center algorithm 48 12. Normal State algorithm of the Star Center algorithm 50 13. The Load Balancer algorithm of the Star Center algorithm 51 14. 5-nodes Open Ring graph 55 15. 12-nodes Ring graph 55 16. (4 4) Mesh graph 56 17. (4 4) Torus graph 56 18. 4 4 4 Hypercube graph 56 19. Average initial load distribution in all experiments in section 4.1 57 20. SCLB algorithm relative performance for experiment 1 60 21. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 1 60 v

22. SCLB algorithm relative performance for experiment 2 61 23. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 2 61 24. SCLB algorithm relative performance for experiment 3 63 25. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 3 63 26. SCLB algorithm relative performance for experiment 4 65 27. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 4 65 28. SCLB algorithm relative performance for experiment 5 67 29. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 5 67 30. SCLB algorithm relative performance for experiment 6 69 31. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 6 32. The execution time using the SCLB and HOLB algorithms for each graph in experiment 7 33. The execution time using the SCLB and HOLB algorithms for each graph in experiment 8 34. The execution time using the SCLB and HOLB algorithms for each graph in experiment 9 35. The execution time using the SCLB and HOLB algorithms for each graph in experiment 10 36. The execution time using the SCLB and HOLB algorithms for each graph in experiment 11 37. The execution time using the SCLB and HOLB algorithms for each graph in experiment 12 38. Loads applied using the uniform load generation policy with min = 0, max = 50, and α = 0 69 71 72 73 74 75 76 78 vi

39. Loads applied using the fixed load generation policy with fl = 50 78 40. Loads applied using the exponential load generation policy with beta = 10.5 79 41. SCLB algorithm relative performance for experiment 13 80 42. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 13 81 43. SCLB algorithm relative performance for experiment 14 82 44. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 14 83 45. SCLB algorithm relative performance for experiment 15 84 46. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 15 85 47. SCLB algorithm relative performance for experiment 16 86 48. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 16 87 49. SCLB algorithm relative performance for experiment 17 88 50. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 17 89 51. SCLB algorithm relative performance for experiment 18 90 52. The average difference between the processed load using the SCLB and HOLB algorithms by each node in experiment 18 53. The execution time using the SCLB and HOLB algorithms for each graph in experiment 19 54. The execution time using the SCLB and HOLB algorithms for each graph in experiment 20 55. The execution time using the SCLB and HOLB algorithms for each graph in experiment 21 56. The execution time using the SCLB and HOLB algorithms for each graph in experiment 22 91 92 93 94 95 vii

57. The execution time using the SCLB and HOLB algorithms for each graph in experiment 23 58. The execution time using the SCLB and HOLB algorithms for each graph in experiment 24 96 97 viii

LIST OF TABLES Table Description Page 1. Adjacency list of the graph shown in figure 1 4 2. Local loads V.s. time units of the system graph shown in figure 8 3. The average initial load applied to each node in a simulated graph in each experiment (round) 38 57 4. Graph characteristics for experiment 1 58 5. Results of experiment 1 58 6. Graph characteristics for experiment 2 60 7. Results of experiment 2 60 8. Graph characteristics for experiment 3 62 9. Results of experiment 3 62 10. Graph characteristics for experiment 4 64 11. Results of experiment 4 64 12. Graph characteristics for experiment 5 66 13. Results of experiment 5 66 14. Graph characteristics for experiment 6 68 15. Results of experiment 6 68 16. Results of experiment 7 70 17. Results of experiment 8 71 18. Results of experiment 9 72 19. Results of experiment 10 73 20. Results of experiment 11 74 21. Results of experiment 12 75 ix

22. The average load processed by each node in a simulated graph in each round 23. Load generation policies used in the experiments conducted in section 4.2 77 77 24. Graph characteristics of experiment 13 79 25. The results of experiment 13 80 26. Graph characteristics of experiment 14 81 27. The results of experiment 14 82 28. Graph characteristics of experiment 15 83 29. The results of experiment 15 84 30. Graph characteristics of experiment 16 85 31. The results of experiment 16 86 32. Graph characteristics of experiment 17 87 33. The results of experiment 17 88 34. Graph characteristics of experiment 18 89 35. The results of experiment 18 90 36. Results of experiment 19 92 37. Results of experiment 20 93 38. Results of experiment 21 94 39. Results of experiment 22 95 40. Results of experiment 23 96 41. Results of experiment 24 97 x

LIST OF APPENDICES Appendix Description Page A. Graph algorithms used in the study 103 xi

ABSTRACT DYNAMIC GRAPH ANALYSIS FOR LOAD BALANCING APPLICATIONS by Belal Ahmad Ibraheem Nwiran This research investigates dynamic graphs for load balancing applications. It suggests a new distributed dynamic load balancing algorithm. This algorithm depends on exchanging messages between the nodes of a network of processing elements (i.e. network of workstations, network routers, processors in multi-processor architectures etc.) to transfer loads (to be alternatively called time units) and load-requests. Each node takes its decisions individually depending on its state and the messages it receives. The objective of this algorithm is to optimize some performance criteria such as the total response time. In an application that consists of tasks, the process can be thought of as load balancing by migrating tasks from a loaded node to a less loaded one. xii

21. T.L. Casavant, J.G. Kuhl. A taxonomy of scheduling in general-purpose distributed computing systems. IEEE Transactions on Software Engineering Feb 1988; 14(2): 141-154. 22. Michael Chau, Hsinchun Chen. Comparison of Three Vertical Search Spiders. IEEE Computer May 2003; 36(5): 56-62. 23. Wikipedia, Graph Theory, November, 2001. http://en.wikipedia.org/wiki/graph_theory. March, 2009. 24. A. Barak, O. Laden, and Y. Yarom. The NOW Mosix and its Preemptive Process Migration Scheme. Bull. IEEE Technical Committee on Operating Systems and Application Environments 1995; 7(2): 5-11. 25. M. Litzkow, M. Livny and M. Mutka. Condor A Hunter of Idle Workstations. The 8 th International Conference on Distributed Computing Systems; 1988. 26. F. Douglis and J. Ousterhout. Transparent Process Migration Design Alternatives and the Sprite Implementation. Software Practice and Experience Aug 1991; 21(8): 757-785. 27. Niranjan G. Shivaratri, Phillip Krueger, Mukesh Singhal. Load Distributing for Locally Distributed Systems. Computer Dec 1992; 25(12): 33-44. 28. C.-Z. Xu, B. Monien, R. L uling and F. C. M. Lau. Nearest neighbor algorithms for load balancing in parallel computers. Concurrency: Pract. Exp. 1995; 7(7): 707 736. 29. A. Grama, A. Gupta, G. Karypis, and V. Kumar. Introduction to Parallel Computing. 2 nd ed. Addison Wesley; 2003. 30. J. Watts and S. Taylor. A Practical Approach to Dynamic Load Balancing [dissertation]. Syracuse University; 1997. 31. Michael J. Litzkow. Remote unix - turning idle workstations into cycle servers. Proceedings of Summer USENIX Conference; 1987. 32. F. Magoules, J. Pan, K. Tan, and A. Kumar. Introduction to Grid Computing. CRC; 2009. 102

Appendix A This appendix describes the graph algorithms that we used in our study. 1. Transitive Closure The transitive closure of a binary relation R is a relation R* defined as follows: x R* y if and only if there is a sequence: x 0 = x, x 1, x 2,, x n = y, where n > 0 and x 0 R x 1, x 1 R x 2,, x n-1 R x n. A binary relation can be represented by a directed graph. Suppose that G is the directed graph represented by a relation R. The directed graph G* representing the transitive closure R* of R is called the transitive closure of G. Reachability matrix of G is a matrix whose elements represent the reachability between two vertices v i and v j. v i is reachable to v j if there is a sequence of directed edges between them. The adjacency matrix of G* is the same as the reachability matrix of G. The following algorithm is developed by Warshall [16] to find the transitive closure of a graph G: Let M be the adjacency matrix of G for( i = 1; i < n; i++) { for (j = 1; j < n; j++) { if (M[i][j] == 1) { for (k = 1; k < n; k++) { M[j][k] = M[j][k] M[i][k]; } } } } where n is the order of the graph G. 103

2. Dijkstra s algorithm The Dijkstra s algorithm [13] is used to find the shortest path from a specified vertex to all other vertices in a graph. The following assumptions and notations will be used in algorithm: 1. Let G = (V, E) be the graph in which the algorithm will be applied on. It is assumed that all lengths are not negative. 2. u is the vertex from which the shortest path to all other vertices will be found. 3. D(v) is the distance between u and v. 4. N` is a subset of V that contains all the vertices {v} which the shortest path between u and v is definitely known. The following pseudo code is the Dijkstra s algorithm. Initialization: N` = {u}; foreach v in V { if v is adjacent to u { D(v) = d(u, v); Else D(v) = ; } } do { find w V - N` such that D(w) is minimum; add w to N`; foreach v V - N` that is adjacent to w { D(v) = min(d(v), D(w) + d(w, v)); } } while(n` V); 104