Ant-based Load Balancing Algorithm in Structured P2P Systems

Ant-based Load Balancing Algorithm in Structured P2P Systems Wei Mi, 2 Chunhong Zhang, 3 Xiaofeng Qiu Beijing University of Posts and Telecommunications, Beijing 876, China, {miwei985, zhangch.bupt., qiuxiaofeng}@gmail.com Abstract Dynamic load balancing is one key adaptation mechanism often deployed in networking and computing systems. umerous proposals exist for load balancing in P2P networks. All of them will enhance the availability of P2P system to some extent. However, their convergence rate is still low, and the load collection overhead and load transfer overhead are high. In this paper, we propose an ant-based load balancing algorithm, and design a candidate nodes discovery mechanism and a target nodes selection mechanism. Two types of agents are cooperating to realize these mechanisms: Search ant and Guide ant. Performance evaluation shows that, to achieve some specific load balancing effect, compared to the classical algorithms, the convergence rate, the load collection overhead and load transfer overhead of our proposed algorithm are significantly reduced; In addition, this paper analysis how system and algorithm parameters affect the performance of proposed algorithm in depth. Keywords: P2P, DHT, Load balancing, Ant colony optimization, Search ant, Guide ant. Introduction Decentralized structured overlays and distributed hash tables (DHT) proffer a unique vision of computing: a collection of computing and communication resources shared by active users. However, nodes are heterogeneous, workload assigned to system may be heavy-tailed, node availability and churn rates may change over time. Load balancing is a key step towards adapting to these characteristics and ensuring the reliability and availability of the systems. There is a large body of literature on load balancing schemes of DHT system. The basic idea of all algorithms is based on the real-time load condition of some part or all DHT system, to design the load re-assign or load transfer algorithm. Therefore, the evaluation index of load balancing algorithms include: load balancing effect, load balancing overhead. Here, load balancing overhead can be divided into load information collection overhead and load transfer overhead, the speed of load information collection and load transfer reflect the convergence rate indirectly. According to the different entity which is responsible for collecting real-time load conditions and generating load re-assign or load transfer algorithm, the DHT load balancing can be divided into two categories: centralized and decentralized dynamic load balancing algorithm. Centralized load balancing algorithm [-6] can be divided into all-centralized and semi-centralized load balancing. In all-centralized load balancing scheme, a central node is responsible for collecting real-time load conditions and generating load re-assign or load transfer algorithm for global system; in the semi-centralized load balancing scheme, multi-central nodes are respectively responsible for collecting real-time load conditions and generating load re-assign or load transfer algorithm for local system. In the centralized model, although the load collection and redistribution centers can provide efficient global or local optimal allocation strategy, these centers can easily cause single point failure and the system performance bottleneck in large-scale dynamic systems, because the overhead of load information collection and load distribution strategy implementation is huge. In the decentralized load balancing schemes [-2, 7-], the load transfer nodes need to detect the load distribution of the system, then based on the detected load conditions, to achieving load balancing effect, they need to change their ID or transfer their some part load. In this model, decentralized load balancing scheme can avoid the single point failure, but its convergence rate is higher caused by the blindness of the load detection. The load effect is much closer to global or local optimal one, the load information collection overhead is much greater. In this paper, based on the analysis of advantages and disadvantages in centralized and decentralized load balancing algorithm, we propose an Ant-based load balancing in Structured P2P network, This work is supported by China-Finland Cooperation Project (o. 2DFA278), ational Key Program (o. 2ZX35-4-2) and Key Laboratory of Universal Wireless Communications, Ministry of Education. Journal of Convergence Information Technology(JCIT) Volume7, umber6, April 22 doi:.456/jcit.vol7.issue6.39 332

abbreviation. makes a compromise between the load balancing effect and balancing cost, and is greatly improved in scalability, adaptability and robustness. Such advantages of are as following: ) relies on all nodes in the system, so it can avoid a single point failure; ATalgorithm introduce the parameters of load balancing effect and balancing overhead into the load transfer nodes selection, it is a fundamental tradeoff between the load balancing effect and balancing overhead. So, this algorithm is more suitable for the actual system requirements. 2) Based on ant colony optimization, a load transfer nodes selection technique is proposed in ATalgorithm. This is a non-direct cooperation way, it can avoid the blind search and improve system convergence rate, which is greatly improved in extensibility. Compared to the proposed centralized and decentralized load balancing schemes, the load information collection overhead of is significantly reduced. 3) During the process of load transfer node selection, not only consider the load state of node, but also introduce a delay in the link between nodes which can avoid the load transfer on the large link delay between nodes, so can reduce the load transfer costs; 4) selects the available load of nodes as pheromone whose update is complete with the DHT routing table update. It does not need any additional structures to collect load information which enhance the system robustness, and it can solve the slow convergence problem which is caused by no significant differences of pheromone in the initial ant colony optimization. The rest of this paper is structured as follows. The related work is presented in Section 2. Section 3 describes the AT- algorithm. We evaluate performance of the proposed load balancing algorithm in Section 4. Finally, we conclude and present future work in Section 5. 2. Related work As previous description, the basic idea of DHT load balancing algorithm is design the load re-assign or the load transfer algorithm based on the real-time load state collection of the whole or partial system. According to the different entity which is responsible for collecting real-time load conditions and generating load re-assign or load transfer algorithm, the DHT load balancing can be divided into two categories: centralized and decentralized load balancing algorithm. The difference of these methods is the specific load information collection and load re-assign or load transfer program. In centralized dynamic load balancing model, reference [, 2] provides one or more directory nodes which are responsible for collecting the load information and generating the load transfer schemes. A directory node periodically calculates load information of random nodes in the system and achieves load balancing through transferring virtual servers. Reference [3] builds a structure on top of the P2P: k-ary tree, which is responsible for the collection and the release of node information, as well as the transfer strategy of virtual server. Reference [4] the system is divided into several rings, each ring represents a region. To begin with, it balances the load in each ring. If not fully balanced, it will balance the load among the rings. There is a super-node in each ring which is responsible for the load state collection and the load re-assign. In decentralized dynamic load balancing model, reference [, 7] clones a node into multiple virtual servers, the number of virtual servers is proportional to some performance parameters, such as computing power; each virtual server acts as a node in the logical space and responsible for the corresponding space; the capacity of node is stronger, the number of its virtual servers is more, the proportion of its shared resource space is greater, so it will gain more request load. Reference [9] provides two methods to adjust the node mapping space: first, a node has multiple virtual servers, but only one of them become active at any time, it can connect to other nodes randomly and split the load space of the heavy-loaded nodes to balancing the load; second, to achieve the load balancing, it change the ID of the light-load node which make it share the load of the heavy-load node. Reference [] also uses the similar method. In this paper, the proposed algorithm is based on Ant colony optimization (ACO). ACO is a novel evolutionary algorithm, which has the characteristics such as positive feedback, distributing computing and the use of a constructive heuristic etc. And these features quite match the demands of network optimization, some ant-based algorithms have been proposed for many application [2-3], such as mobile ad hoc networks routing, topology optimization, distributed QoS routing, resource allocation, etc. 333

3. Ant-based Load Balancing in DHT Systems This paper adopts ant colony optimization to design the. It includes the candidate nodes discovery mechanism and target nodes selection mechanism. Specially, two types of agents cooperate to realize these mechanisms: Search ant and Guide ant. Search ant simulates the foodseeking behavior of ants that searches for light-nodes. Guide ant is responsible for managing a candidate light-load node list, instructing source node to select target nodes. In this section, the related definition, data structure of search and guide ants, candidate light-load nodes discovery mechanism, target node selection mechanism and overall process of load balancing algorithm will be presented. 3.. Related Definitions Definition ode Capacity (C) The factors that affect the load capacity of the node, such as CPU speed, storage capacity, delay, bandwidth, and so on, are unified as one resource. C = P( CPU ) * w + P( Space) * w2 + P( Memory) * w3 + P( IO) * w4 + P( BandWidth) * w5 () Where wi represents the weight of each factor; P(x) represents the value of factor x s capacity. Different application systems have different weight of each factor. Definition 2 ode Load (L). In DHT network, the load of nodes is the demand for capacity. L = L( CPU ) * w + L( Space) * w2 + L( Memory) * w3 + L( IO) * w4 + L( BandWidth) * w5 (2) Where wi represents the weight of each factor; L(x) represents the value of factor x s load. Definition 3 Utilization Rate (μ/ m ) ode utilization rate μ refers to the ratio of load of node L to its largest carrying capacity C, that is, μ=l/c. ode utilization rate describes the situation of the current node load. System utilization rate m refers to the ratio of all the nodes load in the system to the largest carrying capacity, which represents the total load situation of network. It can be described as m= å L( i) / å C( i) (3) i= i= Definition 4 Load Deviation Rate (e) It can be described as the difference between utilization rate of node and system utilization rate. e = m - m (4) Definition 5 Physical Distance (D) In this paper, we adopt the delay(a,b) as the physical distance between node a and b. It can be achieved by network coordinates or actual network measure. In this paper, the physical distance of the simulation uses the real data of the delay measure: P2PSim data sets. 3.2. Data Structure of the Search and Guide ants Based on the different tasks performed by the ants, there are two types: Search ant and Guide ant. Search ant simulates the food-seeking behavior of an ant that searches for light-nodes. Guide ant is responsible for managing a candidate light-load node list, instructing source node to select target nodes. Search ant is generated by the source node whose e reach some setting threshold, that find candidate light-load nodes in the DHT. Each search ant has a tabu list to record the visited nodes which prevent repeat visit nodes and a maximal survival time TTL. Guide ant generated by candidate lightload node or node whose search ant s TTL is. Guide ant is responsible for managing a candidate light-load node list that consists of pheromone and physical distance (delay). According to the function of ants, we build data structures for them as shown in Table, 2. 334

Table. Structure table of search ant Ant Id Restricted condition Source ode Taboo list Delay list Pheromone list Survival time AntID res(m) s.t F tabu list Ds list Info list TTL Table 2. Structure table of guide ant Ant Id Source ode Candidate nodes list Delay list Pheromone list AntID F Candi list Ds list Info list 3.3. Candidate Light-load odes Discovery Mechanism In DHT systems, the specific circumstance of node s load is unknown, and nodes are heterogeneity. Adopting ant colony optimization algorithm, we can find candidate light-load nodes in the unknown load distribution system, and choose the suitable one or more nodes to data migration. In this section, we will elaborate the candidate node discovery mechanism from the following aspects: the generation and update of the pheromone, the routing mechanism and life-span control policy of Search ant. 3.3.. Generation and Updating of Pheromone Pheromone plays an important role in the candidate node discovery mechanism. It guides the route direction of search ant. Therefore, pheromone must fully reflect the node load. This paper defines the pheromone ph(i) as the available capacity of node i, that is, ph(i)=c(i)-l(i). How to generate and update pheromone is a key. For faster speed and lower cost, updating of pheromone is complete with DHT node routing table update process. The specific process is: () Source node S sends route table update message; (2) On receiving routing table update message, node i reply message with available capacity ph(i); (3) On receiving a reply message, node S records nodes information in the routing table entry. This generation and update mode does not need any additional structures to collect load information. It solves slow convergence problem caused by no significant differences of pheromone in the initial. 3.3.2. Routing Mechanism of Search Ant Pheromone is the core of the solution based ant colony optimization, which represents some prior knowledge, and its size represents the load balancing effect. Delay is also introduced in routing selection as a heuristic factor, which guaranteeing the load transfer overhead. Therefore, both pheromone and heuristic must be considered in DHT routing. Suppose that node i receives search ant k, node i will select the neighbor node j as next hop by forward probability p k (i, j). a b ì ph( j) *(/ delay( j)) a b, ph( j) & & j routingtable( i) tabu( k) > Î - ph( u) *(/ delay( j)) k, j ) = å í uîroutingtable ( i ) (5) p (i î, others Where, ph(j) represents the phenomenon value of node j; delay(j) represents the heuristic value of node j; α and β is the relative important factor of pheromone and heuristic; routingtable(i) identifies the routing table entry of node i; tabu(k )is the taboo list of search ant k. 3.3.3. Life-span Control Policy In this algorithm, the survival time of search ant is controlled by life-span control policy. To generate the Search ant, TTL will be set an initial value and minus one with every hop forward. When a node receives a search ant, the control of its life cycle as follows: () If the current node meets the restrictions, it should generate guide ant, update the TTL of search ant to by force, and end forward search ant. (2) If its all neighbor nodes are in tabu list of search ant, this node should generate guide ant, update the TTL of search ant to by force, and end forward search ant. 335

(3) If neither the current node meets the constraints, nor all neighbors are in tabu list of search ant, then it should update search ant TTL to TTL=TTL-; and according to the formula (5), this node chooses neighbor node whose forward probability is largest as the next hop, and forwards search ant. In the first case, search is ended as long as candidate node is found. In the other case, if TTL is, target transfer nodes should require two or more candidate light-nodes to share the overloaded load. 3.4. Target ode Selection Mechanism Source node takes the candidate light-load nodes from guide ant as a node list M, and chooses the optimal set of one or more light-load nodes from list M. The metrics should consider two factors: ) load balancing effect, here, may be expressed in the standard deviation of e. 2) load migration overhead, here, may be expressed in the bandwidth consumption for transfer load. The objective function (obf) for target node selected and the constraints (s.t.) as follows. M 2 2 A* e ( m) e ( s) M + å + m= obf= (6) M Load _ transfer ( m ) B* å * rep( m) Load _ transfer m= ì ü Load _ transfer ³ C( s) * ( e( s) -threshold( e)) s. t. í" m Î M L( m) + Load _ transfer( m) C( m) * m ý (7) M å Load _ transfer(m) ³ Load _ transfer î m= þ The selected target nodes (lists) are the candidate nodes who meet s.t. and their obf is largest. In formula (6,7), where A and B are the weight of load deviation rate and service reputation; Load_transfer is total load which source node transfers out; Load_transfer(m) is the load of node m receives. The constraints are as follows: lode deviation rate of overloaded node can drop below, lode deviation rate of light node can t exceed. 3.5. Overall Process of Load Balancing Algorithm Once source node F s load deviation rate e reach, the flow is as follows: () ode F generates kth (k is initialized to ) Search ant, and sets ant ID, constraints, node F s information (address, physical distance etc) and TTL; (2) According to the formula (5), node F chooses the neighbor whose forward probability is kth largest as the next hop, and forwards search ant k; (3) On receiving ant k, node D puts its ID, pheromones and delay into the data structure of ant k. According to s.t., node D judges whether it is a valid candidate node; a. If node D meets the s.t., it should generate guide ant which return to node F directly, update the TTL to by force, and end forward search ant k. b. If node D does not meet the s.t., and all neighbors are in tabu lists, it should generate guide ant which return to node F directly, update the TTL to by force, and end forward search ant k. c. If neither node D meets the s.t., nor all neighbors are in tabu list of search ant, it should update TTL to TTL=TTL-; If TTL=, node D generates guide ant which return to node F directly, otherwise, according to the formula (5), node D chooses the neighbor whose forward probability is largest as the next hop, and forwards search ant, then go to step (3). (4) On receiving guide ant, source node F should make all candidate node forms list, and selects target nodes who meet formula( 7) and the value of formula (6) is largest, then doing data migration. (5) Source node F calculates the new load deviation rate e, if e>, then k=k+, go to step (); Otherwise the algorithm is end. 4. Performance Evaluation In this section, we set some goal for load balancing effect using a list of. For each, we will compare load balancing effect and balancing overhead which includes load 336

information overhead and load transfer overhead among -algorithm, [2] (classic decentralized load balancing) and [2] (classic centralized load balancing) in Chord context, and evaluates how system and algorithm parameters affect the performance of ATalgorithm-algorithm. Parameters used in this section are listed in Table 3 and Table 4. Table 3. Basic experiment parameters Parameters Description Value node number [2,2 6 ] Load System load 6 C node s capacity [.5 C,2 C ] delay(a,b) the delay between node a and b m ode utilization rate [,] Table 4. Balancing algorithm parameters Parameters Description Value the threshold of e [,] TTL maximal survival time log 2 α the relative importance [,] factor of pheromone β the relative importance [,] factor of heuristic dum directory nodes number log 2 4.. Performance comparison and analysis with the classical algorithm In, once node s e>, it should execute to search for suitable light-load nodes to data migration; In, light-load nodes, whose μ< m, should random detect node periodically, if the node s e>, light-load node will do data migration from this node; In, the assignment of loads is typically performed by one or more directory nodes, a directory node periodically receives nodes load information and executes the load balancing algorithm. 4... Load Balancing Effect To evaluate load balancing effect, we compare the load distribution in, O2O, M2M algorithms and without load balancing. Load distribution is also compared among different (.,.2,.3,.4), =2, m =.4. And the simulation runs under different capacity settings. =. =.2.8.8 F(e).6.4.2 -.5 -.4 -.3 -.2 -...2.3.4.5 Load deviation rate e =.3 o load-balancing F(e).6.4 o load-balancing.2 -.5 -.4 -.3 -.2 -...2.3.4.5 Load deviation rate e =.4.8.8 F(e).6.4.2 o load-balancing -.5 -.4 -.3 -.2 -...2.3.4.5 F(e).6.4 o load-balancing.2 -.5 -.4 -.3 -.2 -...2.3.4.5 Load deviation rate e Load deviation rate e Figure. Cumulative Distribution Function of e. =2 Fig. shows the empirical CDF of e. As shown, maximum e is reduced a lot when using load balancing algorithm, and three algorithms can balance the nodes load below. However, minimum e of is slightly bigger than the other two algorithms; in, the proportion of nodes whose e> is obviously lower than the other two algorithms, and it is equal in the other two algorithms. It means that compared with classical decentralized load balancing algorithms, is more efficient; compared with classical centralized load balancing algorithms, ATalgorithm share many characters in common, and differ in relatively minor characters. 4..2. Load Balancing Overhead 337

For balancing system load, all solutions have to collect the load information and adjust the load through transferring data or virtual servers, which results in increasing load information collection overhead and load migration overhead a lot. Consequently, we will evaluate load collection overhead and load migration overhead. Here, load collection overhead is evaluated with the DHT routing hops for load collection per heavy-node which could also represent the convergence rate of the system; load migration overhead is evaluated with bandwidth consumption for transfer load, transfer load times delay. As shown in Fig. 2, experimental results as follows: DHT routing hops for load collection per heavy node 4 2 8 6 4 2 load collection overhead transfer load * delay 6 x 9 5 4 3 2 Load migration overhead..5.2.25.3.35.4..5.2.25.3.35.4 Figure 2. The load balancing Overhead () The mean of the DHT routing hops load collection per heavy nodes for, O2O and M2M algorithms are distributed in [5,2], [25,45], [,4] range respectively, indicating that can reduce the load collection overhead. Because it is a non-direct collaboration way, blind search services can be avoided. So, it can improve the convergence rate of the system. (2) The data migration overhead of our is lower than the other two algorithms obviously. It is because that candidate nodes discovery mechanism introduces physical distance, and target node selection mechanism makes a compromise between load balancing effect and overhead. 4.2. The Effect of System and algorithm parameters on algorithm performance This section analysis how system and algorithm parameters affect the performance of proposed algorithm in depth. It is evaluated with following considerations: () This simulation analysis how the system size and load balancing requirements affect the performance of. Here, load balancing requirement is expressed by, range [.,.4], the interval value is.5; system size is in the range of [2, 2 6 ]. (2) Mechanism based on Ant Colony contains some adjustable parameters. However, the values of these parameters are not strictly theoretical guidance. So, it needs to determine the optimal parameter through several experiments. In formula (5), [α, β] is important to, so, in simulation, the value of [α, β] is [.4,.6], [.5,.5], [.6,.4], [.8,.2] respectively. (3) Different algorithm parameters [α, β], with the change of or, this paper analysis the load balancing effect and overhead. Load deviation rate (standard deviation).25.2.5. Load balancing effect DHT routing hops for load collection per heavy node.5..5.2.25.3.35.4 9.6 9.4 9.2 9 Load collection overhead Transfer load * delay 8.8..5.2.25.3.35.4 5 x 4 3 2 Load migration overhead..5.2.25.3.35.4 Figure 3. Simulation results under different and algorithm parameter [α, β] 338

Load deviation rate (standard deviation).6.58.56.54.52.5 Load balancing effect DHT routing hops for load collection per heavy nodes.48 5 9 3 7 22 2 8 Load collection overhead Transfer load * delay 6 4 2 5 9 3 7 3.8 x Load migration overhead 3.6 3.4 3.2 3 2.8 2.6 5 9 3 7 Figure 4. Simulation results under different and algorithm parameter [α, β] As shown in Fig. 3 and Fig. 4, experimental results are as follows: () represents the load balancing requirements, so, becomes greater, the load balancing effect becomes worse and the load balancing overhead decreases. (2) With changing of the system size, load balancing effect and overhead appear to be leveling off. While the fluctuation of load collection overhead is relatively large, because as increases, the ant's TTL is is logarithmic growth. So has a good scalability. (3) In the candidate node discovery mechanism, α, β represent the relative importance of available capacity and delay respectively. With the increase of α (decrease of β), load balancing effect appears a good trend, the convergence rate becomes faster, and load balancing overhead is increased. 5. Conclusions This paper has presented an Ant-based load balancing in structured P2P network called ATalgorithm. Considering load balancing effect and overhead, it designs a candidate nodes discovery mechanism and a target nodes selection mechanism. Two types of agents are cooperating to realize these mechanisms: Search ant and Guide ant. Performance evaluation shows that, compared to the classical algorithms, the convergence rate and load balancing overhead are significantly raised. In the future, based on the collected load information, we still need to develop systemic analysis for the specific load adjustment schemes. 6. References [] Rao A, Lakshminarayanan K, Surana S, Karp R, and Stoica I, Load Balancing in Structured P2P Systems, Peer-to-Peer Systems II, Springer, vol.2735, pp.68-79, 23. [2] Godfrey B, Lakshminarayanan K, Surana S. and Karp R. and Stoica I, Load balancing in dynamic structured P2P systems, IFOCOM 24, IEEE, vol.4, pp.2253-2262, 24. [3] Zhu Y, Hu Y, Efficient, proximity-aware load balancing for DHT based P2P systems, IEEE Trans. on Parallel and Distributed Systems, vol. 6, no. 4, pp. 349-36, 25. [4] Bienkowski M, Korzeniowski M, der Heide F, Dynamic load balancing in distributed hash tables, Peer-to-Peer Systems IV, Springer, vol. 4, pp. 27-225, 25 [5] PEG Limin, XIAO Wenjun, A Binary-Tree based Hierarchical Load Balancing Algorithm in Structured Peer-to-Peer Systems, JCIT, Vol. 6, o. 4, pp. 42-49, 2. [6] Wei Mi, Chunhong Zhang, An Effective Load-Balancing Algorithm SDYA for Structured P2P Systems, Journal of Beijing University of Posts and Telecommunications, vol. 33, no. 5, pp. 6-2, 2. [7] Dabek F, Kaashoek M.F, Wide-Area Cooperative storage with CFS, ACM SIGOPS Operating Systems Review, vol. 35, no. 5, pp. 22-25, 2. [8] Godfrey P.B, Stoica I, Heterogeneity and load balance in distributed hash tables, IFOCOM 25, IEEE, vol., pp.596-66, 25. [9] Karger D, Ruhl M, Simple efficient load balancing algorithms for peer-to-peer systems, In LCS, 3-4, 25. [] Shen H, Xu C.Z, Locality-aware and churn-resilient load-balancing algorithms in structured peerto-peer networks, IEEE Transactions on Parallel and Distributed Systems, IEEE, vol. 8, no. 6, pp. 849-862, 27. 339

[] WAG Bin, SHE Qing-guo, An Effective Algorithm for Hierarchical P2P Load Balancing, JCIT, Vol. 6, o. 5, pp. 23-236, 2. [2] Ducatelle F, Di Caro G, Ant agents for hybrid multipath routing in mobile ad hoc networks, In 2th Annual Conference on Wireless On-demand etwork Systems and Services, pp. 44-53, 25. [3] Khan S.A, Engelbrecht A.P, A fuzzy ant colony optimization algorithm for topology design of distributed local area networks In Swarm Intelligence Symposium, pp. -7, 28. 34