An Efficient Distributed Load Balancing For DHT-Based P2P Systems

Transcription

1 An Efficient Distributed Load Balancing For DHT-Based P2P Systems Chahita Taank 1, Rajesh Bharati 2 1 PG Student, 2 Professor, Computer Engineering Dept DYPIET, PUNE. Abstract- In a distributed system environment it is likely that some nodes are heavily loaded while others are lightly loaded or even idle. It is desirable that the work-load is fully distributed among all nodes so as to utilize the processing time and optimize the whole performance. Many algorithms have been proposed for load balancing issue in P2P systems. However, all these solutions either ignore the heterogeneity nature of the system, or reassign loads among nodes. Load balancing is the process of distributing client request over the set of servers and is a key element of obtaining good performance in distributed application. This algorithm presents an efficient load balancing with partition created by user and accordingly balances the load from one to another peer. They typically use DHT based indexing such as chords, peers participating are heterogeneous, and they migrate three kind of data text document, image and sound, peers can balance their loads proportional to their capacities. Present algorithm does not rely on any tree networks and independent of the geometry, leading to design more appropriate for large scale, dynamic environment. Finally comparing the algorithm with existing and earlier decentralized load balance algorithms designed for structured P2P networks. Keywords-- DHT, Load Balancing, Peer to Peer system, Partition. I. INTRODUCTION Designing an efficient load balancing algorithm got a lot of interest from many researchers for a long period of times. A job scheduling algorithm should achieve fair load balancing between all nodes in the P2P distributed systems. P2P systems have now become an emerging application in the distributed computing technology. Compared with traditional client-server architecture, P2P systems greatly improved the resources utilization by making use of large unused resources across the network such as processing power and enormous storage. In the literature, there are many load-balancing algorithms, especially for process migration. Each mechanism has pros and cons. Generally, the load-balancing mechanisms are divided into two main categories: central-based algorithms and decentralized algorithms. But we will focus on decentralized structured peer to peer system. Load balancing is one of the main issues in structured Peer-to- Pear (P2P) systems. An improvement in the load balancing algorithm can significantly enhance the resources utilization and fairness in P2P systems. Load balancing across multiple nodes has been widely studied in P2P systems. Some proposals exist for load balancing in P2P systems in which each node maintains servers and allows the address space to be partitioned equally but the degree of each node is log 2 n which is not desirable. Most of the structured P2P systems that have been proposed use a Distributed Hash Table (DHT) as an allocation mechanism. The goal of a DHT is to support three operations: join, leave and data update. Joins and leaves are initiated by nodes when they enter and leave the service. An update is a targeted request initiated at a node, and is forwarded toward its target by a series of update-step requests between nodes. In a DHT, each node and key has a unique ID, and each key is mapped to a node according to the DHT definition. The ID space of each DHT is partitioned among the nodes, and each node is responsible for those keys who s IDs are located in its space range. As peers participating in a DHT are heterogeneous, introduces the idea of server which have partition to deal with the heterogeneity of peers. Participating peers in a DHT can host inside one server different numbers of partition and a unique key is generated and there it uses look up sevice to store data and data type(node pad, image or pdf), thus taking advantage of peer heterogeneity. In this paper, the reallocation of a server from a source peer to a destination peer can be simply done by simulating the leave and join operations offered by a typical DHT. Each peer deal with the load balancing problem by minimizing the following: [LBFi - A] and system wide performance metric movement cost as much as possible. v Vi Ci - A Let V = set of server in system, N = set of participating peers, Each peer i N, Vi = set of servers allocated to peer i, = load value of server v Vi, Ci = capacity value of peer i and A = expected load value per unit capacity. 546

2 And Load Imbalance factor of peer i Ci Movement cost v s v V i N v Vi II. Ci == MC RELATED WORK Earlier studies (e.g., [6], [7]) have proposed load balancing algorithms, targeting at static, small-scale, and/or homogeneous environments. Due to space limitation, we provide a concise review of the load balancing techniques designed for DHTs in this section. As the previous study [9] has provided a survey on the load sharing algorithms in traditional high-performance computing systems, we refer interested readers to [10] for an interesting survey on the load balancing algorithms in DHTs the emerging Internet based autonomous network infrastructures. Consider an object set O and a peer set N. Conventional DHTs (e.g., Chord [1] and Pastry [2]) assume that the loads of objects in O are identical. The load of a peer can thus be estimated as the number of objects hosted by the peer. Assuming that the load of each object o Є O is lo =1, earlier studies [1], [11] show that the load imbalance factor in a typical DHT can be up to O (log n), where n = N is the total number of nodes participating in the system. To be more precise, the maximum load of a peer can be O (log n), times the average. Later, several proposals (e.g., [11]) have reduced the load imbalance factor to a constant. In contrast to [11], the proposals (e.g., [3]) exploit the heterogeneity of peers, such that the number of objects allocated to a peer is proportional to the peer s capacity. Unlike [1], [3], [11] our work presented in this paper does not assume that objects in the system contribute the identical load to peers. That is, our proposal does not aim to balance loads among peers in terms of numbers of objects allocated to peers. Instead of simply assuming lo =1for all o Є O and evenly partitioning the key space into each peer, Chord [1] suggests the notion of virtual servers. 547 Different virtual servers in a system manage disjoint key subspaces, with a virtual server serving as an elementary entity for balancing loads among peers. We note that if a virtual server v manages the key subspace S, then the objects, which may have unequal loads and whose keys are within S, contribute their loads to v. The idea of virtual servers enables a DHT to reallocate the virtual servers, such that the resultant load of a peer is proportional to the peer s capacity. Based on the concept of virtual servers, the many-tomany framework is presented in [12] to cope with the load imbalance in a DHT. In the many-to-many framework, light and heavy nodes register their loads with some dedicated nodes, namely, the directories. The directories compute matches between heavy and light nodes and then, respectively, request the heavy and light nodes to transfer and to receive designated virtual servers. As noted by the authors in [3], the many-to-many framework essentially reduces the load balancing problem to a centralized algorithmic problem. As the entire system heavily depends on the directory nodes, the directory nodes may thus become the performance bottleneck and single point of failure. In contrast, in this paper, we are particularly interested in fully distributed solutions to the load balancing problem. The study most relevant to ours is perhaps the work by Zhu and Hu [4]. Zhu and Hu [4] organize virtual servers in a DHT network into an auxiliary tree-shaped overlay network on top of the DHT. Such tree-shaped overlay is used to compute and disseminate the global aggregate, namely, the average allowing each peer k in the system to be based on A to compute exactly its target load threshold Tk and then to identify whether it is heavy or light. Additionally, the tree overlay facilitates the reallocation of virtual servers. The reallocation is performed in a bottomup fashion toward the root of the tree. As we discussed above, while [4] simulation strives to distribute loads evenly due to the virtual servers, the nodes in the tree experience skew the workload for coordinating the reallocation of virtual servers, thus introducing another load imbalance issue.in contrast to [4], our solution attacking the load balancing problem need not rely on any auxiliary tree networks, which are clearly failure-prone and thus require sophisticated maintenance. In addition, while the solution requires aggregates global information to compute A and then to reassign virtual servers, each peer in our proposal is independently based on partial knowledge of the system to reallocate its virtual servers. Specifically, the nodes in our proposal need not coordinate the reallocation of their virtual servers, leading to a design more appropriate for large-scale, dynamic environments.

3 Shen and Xu [5] also present a fully decentralized method for migrating objects (i.e., data items) stored in a DHT. Akin to the solution in [4], Shen and Xu suggest organizing a DHT into a two-level hierarchical network, where the higher level of the network consists of local clusters. Objects with excess loads are moved in a local cluster to balance the loads of the peers located in the same cluster. For those objects that cannot be moved in a local cluster, they can be transferred to peers in some foreign clusters. Unlike [5], our proposal does not assume any specific geometry of DHTs and is thus applicable to any DHT network. III. ALGORITHM SKETCH In this paper, we assume that the entire hash space provided by a DHT and each server in the DHT has a unique ID selected independently and uniformly at random from the space. Let N be the set of participating peers, and S be the set of servers hosted by the peers in N in the DHT. Denote the set of servers in peer i by Si. Each peer in our proposal estimates the load, which is denoted by Ti, that it should perceive, where A is an estimation for the expected load per unit capacity. If the current total load of i is greater than Ti (i.e., i is overloaded), then i migrates some of its partition which are nothing but a table inside the servers to other peers they can have different kind of data text img or pdf. Otherwise, i is under loaded, which does nothing but waits to receive the migrated servers. For an overloaded peer (e.g., peer i), i picks those servers for migration, such that 1) i becomes under loaded, and 2) the total movement cost, MC, (3) is minimized due to the reallocation. If i is an under loaded peer, then i may be requested to receive a migrated server, and i accepts such a server if the added load due to the server will not overload itself; otherwise, i rejects such server. Algorithm 1 (REALLOCATION), which given as follows illustrates our idea. Switch load (i) do Case 1 load (i) >Ti While load(i) > Ti and Si!= Ui do V arg min { v } i seeks peer j I to accommodate k, if j satisfy that it will not be over loaded after k. distinguish Break; Case 2 Load (i) <= Ti Break; If j accept k then, Si Si {k} Else reject that node Ui Ui U {k} While load (i) < Ti do Receive k to host Si Si U {k} Algorithm 1: REALLOCATION (i) Add in set U to Having Ā and thus Ti = Ā * Ci + ε, peer i can then determine whether it is overloaded or not. As mentioned, if i is overloaded(more than 16 mb), i dispatches some of its servers to other peers to reduce its excess load. In this proposal, i reallocates its excess load to the under loaded peers in I, such that the under loaded peers accept the load proportional to their available capacities. By the available capacity of a peer p, we indicate Tp - Ls, that is, the remaining capacity that p can manage without being overloaded. Hence, our intention is to reallocate a sever, s, to an under loaded peer, j, whose available capacity is proportional to the total available capacity of the system. 548

4 We present a novel scheme for each peer i to approximate Pr (X < x) and Pr(Y < y) by sampling a few number of nodes, denoted by set I, in the system. We will detail the estimation of the two probability distributions Pr(X < x) and Pr(Y < y). IV. EXPERIMENTAL RESULT The results of the load balancing in the peer to peer networks and by migrating servers the peer could balance the load proportional to their capacity; the load balancing is carried out by 5 modules. They are Network Creation Implementation of the Client Insert kind of data Establishment of the Server and insertion of data Migration of the Load Notably, similar to studies in [3], [4], [5], the participating peers in our proposal balance their loads periodically every time period (e.g., T minutes). However, we impose no global synchronization among the peers, each peer schedules its load balancing algorithm every T minutes according to its local clock. Next, Algorithm 1 intends to minimize MC by simply choosing a server v with the minimum load each time until the hosting peer becomes under-loaded. As we can see in Algorithm 1, the challenges of implementing the algorithm are 1) how a peer precisely and timely estimates A and 2) how an overloaded peer seeks the peers to receive its migrated servers for balancing the loads among peers. To deal with these issues, our idea is to represent the capacities of participating peers and the loads of servers as the probability distributions, which are denoted by Pr(X<x) and Pr(Y<y), respectively. Both Pr(X<x) and Pr(Y<y) provide valuable information to help the participating peers estimate A, and the overloaded peers discover the under-loaded peers to share their excess loads[15]. Network Creation In the module we design the windows for the peer page. These windows are used to send a message from one peer to another. We use the Swing package available in Java to design the User Interface. Swing is a widget toolkit for Java. In this module mainly we are focusing the login design page with the Partial knowledge information. In this network select n random nodes. In this situation the main focus is on the DHT table in network. Distribute Hash Table maintain a neighbor information so we can get the partial information about the clients. Like Client name, Server name, Id etc. With this information client can communicate with other clients in the network and share their data. Implementation Of The Client In this module we develop a client home page design. In this first the client is going to login. Client can Browse the file from the system and upload the data. 549

5 Insert Kind Of Data In this module we select kind of data we wish to insert. Each server have some capacity so it will check the memory first and if memory is available it will get the data from the client and store it in the memory, if memory is not available than it will check, is there any under loaded server available if so it will upload the data to that server. Establishment Of The Server And Insertion Of Data In this module we made a limitation that not more than three server can be created in particular peer. The server will handle the request and response to the peer according to its capacity. Server will have some capacity of 16mb. Migration Of The Load The load balancing in the peer network is accomplished in this module, it manages the work load in the network when the excess load occurs in the network and the node is heavily-loaded the loads are transferred between server of another peer. Massage can come on another peer but as we are in heterogeneous distributed network it should not be transparent to user, thus full filling single system image property. Comparative Study 1. Movement Cost Below figure shows the movement costs due to centralized directory and distributed system. Distributed system performs best in terms of the load imbalance factor. 550

6 V. CONCLUSION In this paper, we have presented a novel load balancing algorithm based on DHT. Our proposal is unique in that we represent the system state with probability distributions. Unlike prior solutions [15] that often rely on global knowledge of the system, each peer in our proposal independently estimates the probability distributions for the capacities of participating peers and the loads of servers. With the approximated probability distributions, each peer identifies whether it is under loaded or not and then reallocates its loads if it is overloaded. The simulation results reveal that our proposal performs well and that it is comparable with the centralized directory approach in terms of the movement cost of servers, and for future work we can enhance the system for security analysis in cloud. REFERENCES [1] I. Stoica, R. Morris, D. Liben-Nowell, D.R. Karger, M.F. Kaashoek, F. Dabek, and H. Balakrishnan, Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications, IEEE/ACM Trans. Networking, vol. 11, no. 1, pp , Feb [2] A. Rowstron and P. Druschel, Pastry: Scalable, Distributed Object Location and Routing for Large-Scale Peer-to-Peer Systems, Lecture Notes in Computer Science, pp , Springer, Nov [3] S. Surana, B. Godfrey, K. Lakshminarayanan, R. Karp, and I. Stoica, Load Balancing in Dynamic Structured P2P Systems, Performance Evaluation, vol. 63, no. 6, pp , Mar [4] Y. Zhu and Y. Hu, Efficient, Proximity-Aware Load Balancing for DHT-Based P2P Systems, IEEE Trans. Parallel and Distributed Systems, vol. 16, no. 4, pp , Apr [5] H. Shen and C.- Z. Xu, Locality-Aware and Churn-Resilient Load Balancing Algorithms in Structured P2P Networks, IEEE Trans. Parallel and Distributed Systems, vol. 18, no. 6, pp , June [5] L.M. Ni and K. Hwang, Optimal Load Balancing in a Multiple Processor System with Many Job Classes, IEEE Trans. Software Eng., vol. 11, no. 5, pp , May [6] L.M. Ni, C.-W. Xu, and T.B. Gendreau, A Distributed Drafting Algorithm for Load Balancing, IEEE Trans. Software Eng., vol. 11, no. 10, pp , Oct [7] C. Chen and K.-C. Tsai, The Server Reassignment Problem for Load Balancing in Structured P2P Systems, IEEE Trans. Parallel and Distributed Systems, vol. 12, no. 2, pp , Feb [8] T.L. Casavant and J.G. Kuhl, A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems, IEEE Trans. Software Eng., vol. 14, no. 2, pp , Feb [9] Y. Zhu, Load Balancing in Structured P2P Networks, Handbook of Peer-to-Peer Networking, Springer, July [10] D. Karger and M. Ruhl, Simple Efficient Load Balancing Algorithms for Peer-to-Peer Systems, Proc. 16th ACM Symp. Parallel Algorithms and Architectures (SPAA 04), pp , June [11] A. Rao, K. Lakshminarayanan, S. Surana, R. Karp, and I. Stoica, Load Balancing in Structured P2P Systems, Proc. Second Int l Workshop Peer-to-Peer Systems (IPTPS 02), pp , Feb [12] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman and Co., [13] F. Dabek, M.F. Kaashoek, D. Karger, R. Morris, and I. Stoica, Wide-Area Cooperative Storage with CFS, Proc. 18th ACM Symp. Operating Systems Principles (SOSP 01), pp , Oct [14] Hung-Chang Hsiao, Hao Liao, Ssu-Ta Chen, and Kuo-Chan Huang Load Balance with Imperfect Information in Structured Peer-to- Peer Systems IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, OL. 22, NO. 4, APRIL [15] W.K. Hastings, Monte Carlo Sampling Methods Using Markov Chains and Their Applications, Biometrika, vol. 7, no. 1, pp , Apr [16] S.M. Ross, Markov Chains, Introduction to Probability Models, ninth ed., pp , Academic Press, [17] M. Mitzenmacher and E. Upfal, Coupling of Markov Chains, Probability and Computing: Randomized Algorithms and Probabilistic Analysis, pp , Cambridge Univ. Press, [18] P. Diaconis and D. Stroock, Geometric Bounds for Eigenvalues of Markov Chains, The Annals of Applied Probability, vol. 1, no. 1, pp , Feb [19] A. Sinclair, Improved Bounds for Mixing Rates of Markov Chains and Multicommodity Flow, ombinatorics, Probability and Computing, vol. 1, no. 4, pp , Dec