Efficient Query Dissemination on a Wide Area Network using Peer To Peer network

Transcription

1 Efficient Query Dissemination on a Wide Area Network using Peer To Peer network Kyungyong Lee Electrical and Computer Engineering University of Florida klee@acis.ufl.edu 1. Problem Area Desktop grids using commodity computers connected over a local or wide-area network have been proven to be excellent platforms for high-throughput computing while providing sharing CPU power, storage capacity, and applications. These systems should be scalable, faulttolerant, and secure in the presence of abnormal or malicious behavior of a user or churn. These systems must be able to distribute jobs evenly amongst nodes to achieve maximum utilization and alleviate hot stops. To efficiently distribute loads, supporting efficient resource discovering method is vital component for the management system. To support resource discovery, centralized discovery system (Condor, Boinc, and Archer) and decentralized discovery systems (Sword, CAN-based query system) are proposed and widely used. 2. Problem Statement Middleware solutions for managing distributed resources, such as Condor and Boinc, often rely on a centralized master server that maintains global view of available resources for optimal scheduling decisions. Worker nodes, in turn, publish their resource information to the central master node, so that submitted jobs requirements can be matched against available resources to determine the most suitable nodes for executing the submitted jobs. The master server approach often imposes significant administrative, uptime, and scalability issues, and the failure of the master can render all its managed resources unusable. As a decentralized discovery system, SWORD supports DHT-based resource discovery method. SWORD requires nodes to periodically publish their resource information, which can impose significant network overheads even when no jobs are being submitted to the system. Furthermore, the retrieved resource information might be stale due to characteristics of DHTs entry update period. 3. Paper Approach Moara is a decentralized resource discovery system runs on top of a Pastry Peer To Peer (P2P) network. In Pastry, each node has own 128 bits P2P id, which is generated by a hash function. Each node keeps a routing table consisted of P2P for routing a message to the destination P2P id. For resource discovery query dissemination, it uses Scribe multicast channel. Scribe runs on top of Pastry while providing multicasting a message. I will discuss the Sword in detail in the

2 next section. Because Scribe supports propagating message on a Pastry, it is an appropriate selection for a query dissemination method. Moara builds a Scribe multicast tree for each resource attribute. At a given multicast tree, every child node sends no-prune, if the node satisfies a tree requirement. Otherwise, prune message is sent to its parent node. Parent nodes keep child nodes tree requirement satisfaction status locally. When a query is received at the node, it checks the local table to determine whether to disseminate the query to a given child node or not. Figure 1 shows an example of Moara query tree. The tree can be assumed as one attribute requirement.(e.g. nodes which has Matlab installed). In the figure, only node F is Matlab installed. Because node D and E do not satisfy the requirement, it sends prune message to node B. Node B also does not satisfy the requirement, so node B finally sends prune message to Node A. Node F satisfies the requirement, so it sends no-prune message to its parent node, C. Then, node C sends no-prune message to the parent node A, though node C does not satisfy the requirement. Using prune and no-prune message, each node can minimize the number of queries produced in the network. Other than basic query dissemination, Moara supports Dynamic Adaptation Policy (DAP) to minimize the network traffic. In the DAP, each node counts the cost for update and query. If the query cost is higher than the update cost, the Moara works in the basic prune and no-prune method by sending update messages. Otherwise, Moara simply sends queries to the entire tree without sending update to its parent node. To minimize the number of hops to deliver a message, Moara supports Separate Query Plane (SQP). In SQP, system wide threshold is defined. When the number of requirement satisfying nodes is below the threshold value and the parent node does not satisfy the requirement, the parent nodes is not involved in the query delivery. Otherwise, the parent node is still involved in the query dissemination. For efficient query dissemination, composite query is supported. In composite query multiple attributes are converged to minimal attributes according to their relationships. 4. Critique Figure 1 Moara Architecture supported by a node. Moara authors did various kinds of experiments to show the applicability of their solution. However, they do not mention about the cost for maintaining a tree for each attribute. Because each attribute has to create a tree for efficient query dissemination, it will incur non-negligible overhead to each participating nodes. Thus it may not be scalable for the number of resource attributes Next, Moara tree is created statically. It means that every node has a predefined resource requirement. It will be all right if the attribute is boolean type (e.g., Operating system == linux ).

3 When it comes to range search, Moara is not available any more. If a tree is defined as memory > 2GB, a user cannot find nodes whose memory size is between 1GB and 1.5GB. Due to static tree configuration, it has a critical drawback for scalable query support. As the next problem for the paper, it does not support handling lagging node in the tree. The problem of tree-based communication is that waiting for result from lagging node sometimes make the entire tree stuck. In this case, it will be a better choice pruning the lagging node from the tree. However, it is not an easy choice when to prune the lagging node. Pruning a lagging node implies that the query results from the lagging node s child nodes will be discarded. Under this situation, Moara does not support a method to handle lagging nodes, which results in a very poor performance metric for PlanetLab experiments. I will propose a handling lagging node method in a tree-based query dissemination system in section 6. The last critique for this paper is about the correctness of the experiment result. The authors compared Moara s PlanetLab experiment result with that of naïve entire network query method. From their performance metrics, it looks that querying entire network is useless when the number of node is over 100. They seem to propagate a query to the entire network using Scribe algorithm. However, the performance can be differentiated a lot by choosing other query dissemination method. To prove the incorrectness of their experiment result, I will propose another query dissemination method on a P2P network. 5. Evaluation/analysis of the approach The authors implement Moara on top of Scribe, which supports multicasting a message to a subset of nodes in a P2P network. In this section, I am going to describe how Scribe works. The Scribe explanation will help to understand the reason of poor performance of Moara due to the underlying multicast method. The Scribe will be comparable to Boundedboradcast, which will be discussed at the next section. Scribe is developed at Microsoft research center and Rice University in Scribe creates a static tree for each multicast group. The static tree is formed as follows: The root node is selected by hashing the multicast group description. After hashing the multicast group description, the nearest id node performs as a root node. From the root node, the tree is formed by differentiating only one digit at each tree level. For example, if the root node id is 0xFFFF, the child nodes address can be 0xFFF0, 0xFFF1, 0xFFF2,, 0xFFFD, 0xFFFE. The multicast group joining message is always routed to the root node. In the course of routing to the root node, if a node which is already a member of the multicast group receives the join message, the member node terminates the join message, and keeps the join message initiating node s address in its local table. This join message termination lessens the overhead of the root node. When a node wants to disseminate a message to the multicast group, the message initiating node has to send the message to the root node first, then the root node propagates the message to the multicasting group. Because the tree is created statically, it cannot react to the non-graceful node departure. Thus, every node has to send periodic live message to its parent node. Because of routing characteristics of Pastry, nodes which are not interested in the message group might be involved in the message dissemination as a forwarder, which imposes additional overhead for maintaining a group and message dissemination.

4 6. Possible Improvements Query Dissemination on the P2P network In order to show the practicality of query the entire network, I explain Boundedbroadcast method developed at the University of Flordia, ACIS lab. Boundedbroadcast spreads a message over a sub-region of a P2P network by leveraging a structured P2P system to self-organize a multicast tree. Boundedbroadcast is currently implemented on top of Brunet, which implements Symphony, a 1-d Kleinberg s small-world network. Each node maintains two types of connections: (1) a constant number of near connections to its nearest left and right neighbors on the P2P ring, and (2) approximately log(n), where N is the number of nodes, far connections to random nodes on the P2P ring; such that a routing cost is O(log(n)). During a broadcast a node is allocated a sub-region of the P2P ring over which to disseminate a message. The node then redistributes the message to its neighboring nodes inside its allocated sub-region while allocating new sub-regions to the neighboring nodes. This process continues until the message is disseminated over the entire sub-region. Figure 2 Bounded-broadcast architecture For example, to broadcast a message over the sub-region [A,L] in Figure 2, a message initiator issues a broadcast command to node A with sub-region information, [A,L]. Node A recognizes node B, P, E, K, L, and M as neighbors. The node A then broadcast messages to its neighbors, by specifying broadcast range as [B E), [E K), [K L), [L] to node B, E, K, and L, respectively. After receiving the message, node B, E, K, and L broadcast the message only to their neighboring nodes inside the specified sub-region recursively. After disseminating the broadcast message until the leaf node, a graph like Figure 1 right figure is formed. If a message initiating node does not lie within the boundedbroadcast domain (sub-region of the P2P ring), the

5 message is first routed to a center node inside the bounded-broadcast region by using greedy routing. Figure 3 (a)moara PlanetLab experiment (b)bounded-broadcast Planetlab experiment Different from Scribe, boundedbroadcast does not use a static tree for query dissemination. It uses current connection table for self-organizing a tree. It lessens additional overhead for heartbeat event between parent and child node. In bounded broadcast every message is a one hop away in a P2P address. In case of Scribe, one message can take multiple hops to get to the destination. These differences make performance difference on a PlanetLab experiment shown in Figure 3. Handling Lagging Node in a Tree-Based Communication Method As we have seen in the Figure 3 (a) and (b), tree-based query method sometimes shows bad performance due to extremely lagging nodes. Because Scribe and bounded-broadcast method wait until all child nodes return results. Thus a small amount of nodes late response can make the entire system have high response time. To prevent it, we can run a timeout handler at each parent nodes. If child nodes do not return response within the time, the parent node can immediately return currently aggregated result. However, defining an appropriate timeout value in the tree communication is not a trivial issue, because each node in a different layer in the tree cannot have same timeout value. Obviously nodes near the root node should have higher timeout value than nodes near leaf nodes. Based on the characteristics of tree-based communication method, I propose two timeout defining methods: 1) Gradually decreasing timeout value. When a parent node distributes a query to its child node, the parent node notifies its timeout value to its child nodes. Then, the child nodes decrease the timeout value by x%, and set the new timeout value as its timeout value for its child nodes. This step continues until the query reaches leaf nodes. Assuming the timeout value decreasing rate is a, where 0<a<1, and the root node s timeout value is b seconds, i th level nodes timeout value is b*a i 2) Complete score-based timeout value: Gradually decreasing timeout value has its drawback in the static timeout value in each level; It does not differentiate the timeout value according to the completeness of the query. For example, the timeout value when 10% of child nodes returns result should be longer than that of 90% child nodes returning result case. In addition, each

6 Latency (seconds) node has to maintain different timeout value according to its current network load. One of the metrics to measure the network load is the query latency for child nodes which already return results for the same query session. The user can also define a multiplier (Wait Factor) to determine the length of a timeout value. Assuming user-defined timeout multiplier is m, the query latency is a seconds, and query complete score is b, where 0<b<1, the timeout value of each node will be m* a seconds. Based on this timeout determination method, I did a query b latency measurement test on PlanetLab with 569 nodes. Resource discovery query is sent to the entire network at this experiment. The test result is shown in Figure 4. WF=1 WF=2 WF=4 WF=8 WF=0 (No Timeout) Latency Percentage Figure 4 (a) Query latency according to the Wait Factor (b) Number returned node As we can see from figure 4, query latency is not much different when the overall query latency is small (e.g., latency percentage from the shortest to 60%). When it comes to over 90%, there seems to be a large latency differences according to the wait factor. From Figure 4(a), we can see that By setting using the timeout value (i.e., Waif Factor is 1, 2, 4, and 8), the longest query latency is much less than no timeout case (i.e., Wait Factor is 0). I counted the number of returned result, and showed the result in Figure 4 (b). As the Wait Factor gets smaller, the number of returned nodes also gets smaller. Instead the query latency is short. According to a user s requirement for a query (quick response VS. more accurate query result), the Wait Factor can be set up. 7. Conclusion In this project, I analyzed Moara, a decentralized resource discovery system. Based on the critiques, I described bounded-broadcast, a multicast method on a P2P network which is developed at the University of Florda ACIS Lab, to compare the message dissemination performance. From the PlanetLab experiment, we can see that bounded-broadcast outperforms Moara s Scribe message dissemination. In addition, I propose two message handling methods in a tree based message communication.