Discovery and Routing in the HEN Heterogeneous Peer-to-Peer Network

Discovery and Routing in the HEN Heterogeneous Peer-to-Peer Network Tim Schattkowsky Paderborn University, C-LAB, D-33102 Paderborn, Germany tim@c-lab.de Abstract. Network infrastructures are nowadays getting more and more complex as security considerations and technical needs like network address translation are blocking traffic and protocols in IP-based networks. Applications in such networks should transparently overcome these limitations. Examples for such applications range from simple chat clients to collaborative work environments spanning different enterprises with different heterogeneous network infrastructure and different security policies, e.g., different firewalls with different configurations. Overlay networks are a convenient way to overcome this problem. In many cases, diverse barriers like multiple facing firewalls would require significant user knowledge to establish a connection. Self-organizing peer-to-peer networks appear to be a convenient solution, but contemporary systems still have limitations in overcoming connectivity problems in heterogeneous networks. Thus, we introduce a self-organizing peer-to-peer infrastructure that overcomes these issues by transparently interconnecting networks with different protocols and address schemes. 1 Introduction The Internet led the way to new distance-spanning applications enabling communication and collaboration between distant computers and individuals. However, when trying to interconnect applications and their users in heterogeneous networks, e.g., between protected Intranets, connectivity problems arise. These problems are often caused by different network technologies and security measures. The same problem also applies to many typical end user network applications. Many interesting new applications like IP telephony clients and instant messengers suffer from the same problems (e.g., when two DSL users with routers try to interact). Current overlay networks including peer-to-peer networks cannot resolve the problem in many of the outlined scenarios, especially not when routing over multiple intermediate nodes is required. In order to overcome these problems, we introduce HEN as a self-organizing peer-to-peer network infrastructure that transparently overcomes communication barriers when possible. It establishes a peer-to-peer network interconnecting nodes in different physical networks using different transport protocols. It includes support for features like routing, relay, polling, tunnelling to overcome connectivity problems between heterogeneous networks like those caused P. Lorenz and P. Dini (Eds.): ICN 2005, LNCS 3421, pp. 653 661, 2005. Springer-Verlag Berlin Heidelberg 2005

654 T. Schattkowsky by firewalls and network address translations. It is designed with the goal to allow the transmission of single data packets from one Node in the network to another while dealing transparently with the involved obstacles like routing and relay. The remainder of this paper is structured as follows. The next section discusses related works in the field of peer-to-peer based systems. Section 3 introduces the HEN network model. The EGG protocol used for network discovery and routing is described in section 4 before section 5 closes with conclusions and future work. 2 Related Work Many peer-to-peer systems like Napster and Gnutella seem to explicitly focus on sharing data. These networks usually are bound to an IP protocol and do not deal with heterogeneity in the underlying networks. Centralized networks like Napster overcome some of the connectivity problems by the use of centralized servers or relaying supernodes (e.g, FastTrack), but this does not solve the general problem. Other approaches like FreeNet [1] focus on anonymity at the cost of performance. Pastry [6] and Tapestry [10] are peer-to-peer architectures that both use dynamically assigned random unique identifiers to identify network nodes. Thus, named nodes are not supported at this level. Routing is performed by forwarding a message to a known node having a node identifier sharing a longer prefix with the target node identifier than the current hop. This mechanism is originally inspired by Paxton et al. (PRR) [4] [2] and is to some extent similar to RFC1518 [5]. The join protocol presented in [3] illustrates how several nodes can concurrently join such a network. As it seems that both Pastry and Tapestry target towards homogeneous IP networks without communication barriers, the routing actually introduces a significant overhead by most messages through O(log 2 #nodes) nodes where an initial discovery using the routing mechanism could also enable direct communication on the underlying network. This also generally avoids the effects of latency differences in an overlay network that is elaborated in [9]. Tapestry uses the same mechanism to locate named objects. The object identifiers in Tapestry are syntactically equal to node identifiers. For a particular object, the node with the most similar node identifier stores a reference to the node containing the object. Thus, the routing to the node can be used to discover the actual object. The root node is the root of a tree that is dynamically created by back-propagation of the object reference once messages for the object arrive at the root node. The reference will be propagated back the path to the original sender. Each node on that path will later redirect messages for the object to the actual node containing the object. However, the messages are still subject to the original routing mechanism. Other approaches like Chord [8] introduce centralized lookup services for object discovery that could be used to discover a named node. However, this does not resolve the routing problem and introduces a single point of failure. In previous work, we have introduced ANTS [7] as a peer-to-peer infrastructure for interconnecting Web Services across heterogeneous networks. However, ANTS had serious limitations in scalability. All communication was SOAP-based and thus required an application server. Furthermore, fixed relay nodes where needed. While this was sufficient for small-scale collaborative workspaces, it is not sufficient as a

Discovery and Routing in the HEN Heterogeneous Peer-to-Peer Network 655 general-purpose platform for network applications. Thus, we decided to completely separate the transport from the actual services and significantly extend its capabilities to a general purpose self-organzing peer-to-peer architecture for heterogeneous networks, which we called the HEterogenous Network (HEN). 3 The HEN Network Model The network model underlying HEN is shown in Figure 1. In our model, a computer in the network is a Node. Each Node is identified by a fixed location-independent NodeID. This NodeID is used by the Node when connecting to the HEN network. A real-world network enabling direct bi-directional communication between all the nodes in that network is a Network in our model. Thus, Nodes in the same Network must support exactly the same Protocol. Both are again identified by a GUID. For the Internet, these protocols are derived from the IP. In our notion, Nodes in the Internet supporting HEN Connections using the TCP Protocol form a Network among many others. It is important to notice that the same nodes may be part of other Networks as well, e.g., just by supporting an additional Protocol like HTTP. We aim at creating an overlay peer-to-peer network that contains Nodes from several different Networks. Nodes within the same Network as a Node and Nodes in Networks directly connectable by the Node are said to be local Nodes. cd Logical Model Node + NodeID: GUID + doesrelay: boolean +RelayNode * +PickupNode * * Connector + Address: String * 1 + NetworkID: GUID * 1 * directly connected * to Protocol + ProtocolID: GUID Fig. 1. Basic Network Model The ability to use certain protocols in certain real-world networks may differ from Node to Node. Thus, each Node is connected to each of these Networks using a Connector exposing its identity in that network. This Connector is a piece of software implementing the Network Protocol (e.g., TCP for the Internet). A Node residing in two networks is called a Gateway between these Networks. These Gateways are essential to routing in the HEN. Furthermore, a network maybe directly connected to another network. This implies that nodes of the originating network can act as nodes from the target network without being reachable in the target network. In practice, this means usually that these nodes are behind a router or firewall. However, this indicates also the need to establish a relay to enable direct communication between the networks. This situation is actually detected by nodes

656 T. Schattkowsky joining the source network and detecting a missing relay connection. These nodes act as a PickupNode by establishing a connection to a RelayNode node from the target network to enable direct communication between the networks. The node thus becomes a full member of the target Network. However, it is important to notice that it is assumed here that established connections in the underlying network are bidirectional, which is essential to both the relay mechanism and network discovery. Figure 2 shows an example model showing the communication path from A to C. This example corresponds to Figure 4, which is elaborated later in this paper. It describes how Node A can connect to Node C because it is in a protected sub-network of the Internet, use C and D as intermediate Node to have E forwarding its messages picked up from a direct connection to D to the final destination Node B. A :Node EnterpriseATCP : InternetTCP : B :Node C :Node TCP :Protocol HTTP :Protocol EnterpriseBTCP : Network EnterpriseBHTTP : InternetHTTP : E :Node +PickupNode +RelayNode D :Node Fig. 2. Communication Model Example - (Path from A to B highlighted) 4 The EGG Network Protocol The EGG protocol for network discovery and transport is responsible for all data exchange in the network. It is based on these messages: PACKET(NodeID,(byte)*) sends a packet to a local Node. The packet may be subject to routing there to reach its final destination Node. DROPPED(NodeID,Number) indicates packets dropped at a distant node.

Discovery and Routing in the HEN Heterogeneous Peer-to-Peer Network 657 DISCOVER(NodeID,[NetworkID,Address,timestamp](Network ID)*)to discover a certain Node. If NetworkID and Address of the Node are believed to be known, these are included. The message contains a history to avoid loops. ROUTE(NodeID,(NodeID,NetworkID,Address,timestamp)*) returns routing information to a discovering node. All Nodes in the message will share this information to enable packet transport through these Nodes. PING(id,timestamp)causes the destination node to respond with a PONG(id,timestamp)message containing the local time. This is also used to coordinate the time between nodes. DO_PING(NodeID)ask a remote Node to ping the given node results in a DO_PING_RESULT(responsetime)message containing the response time for the PING message. NEW_GATEWAY(NetworkID,NodeID,Stamp,Address)indicates the presence of a new local Gateway to the given Network. GET_GATEWAYS()causes the remote node to respond with a GATEWAYS((NetworkID,NodeID,Stamp)*)message containing a list of all known local Gateways. All messages additionally contain the NodeID of the originating Node. These messages are sent directly through the Connectors of a Node without being subject to routing. Messages may be either data packets contained in PACKET messages that are subject to routing or protocol messages. Messages are limited to a total size of 64k. During runtime, each HEN Node holds four tables that are essential for routing packets in the network. The Networks Table contains an entry for each Network ever discovered. Such an entry consists of three items, a Network and a Protocol identifier as well as a time stamp. Both identifiers are GUIDs identifying the network and the protocol to be used in the network respectively. While Protocol identifiers are most likely to be taken from the set of pre-defined identifiers including identifiers for TCP, HTTP, HTTPS and SMTP/POP3, the network identifiers are likely to be randomly created except for the Internet, which has a pre-defined identifier. The Nodes Table contains an entry for each Node in every Network ever discovered. Each entry consists of five items. The first two items are an UID for the Node and the Network. Thus, the table may contain different entries for the same Node in different Networks. The third item indicates whether the node is believed to be active or down. The remaining two items are a time stamp for the entry and a string containing the actual protocol-address of the Node in the Network. The Gateways Table stores time-stamped pairs of Network identifiers. Such a pair indicates that the first network contains a direct gateway to the second network enabling data transport from the first network to the second network. It is important to notice that the Gateways Table cannot be derived from the Nodes Table as certain Nodes may refuse to gate traffic for other Nodes although it is technically possible. The Forward Table contains pairs of NodeIDs indicating that packets send to the first Node have to be routed over the second Node.

658 T. Schattkowsky The time stamps in the tables are 64 bit numbers indicating the coordinated time at which the original information was created. Thus, time stamps from different nodes are comparable. By including a tolerance (i.e., a minute), time differences between the nodes can be largely ignored and items from different nodes become comparable. 4.1 Joining the Network When a Node starts up, it has to discover the existing network. Initially, it has to connect to the network at all. This is done by using a stored Nodes Table. This Nodes Tables is either initially provided somewhere (for a new node) or has been persistently stored from the when the Node was last connected. A Node has to determine its current Network if the physical address of a Node has changed since its last connection to the HEN network, the Node connects to the HEN network for the first time or it obviously has trouble to connect any Node in its current Network. The last situation indicates that the node has trouble using its underlying network connection, which may be caused by a failure or configuration change. However, to actually determine the current Network, a Node attempts to contact some of the most recently known Nodes of each Network from its Node Table that uses a protocol compatible the Node s capabilities. The order in which the Networks are tested is determined by a history of Networks the Node has been in. The Node will first attempt to locate itself in these networks, trying the most recent one first. However, later it will attempt to test all known Networks including the Internet. If the Node detects the ability to connect to a certain network, it is still necessary to test if Nodes from that Network can access the current node or are blocked (e.g., by NAT or a firewall). The Node will send a DO_PING message containing its NodeID, the currently used ProtocolID and the Node address. The remote address will attempt send a PING message to the node to test the connection and finally indicate a successful attempt where a PONG response has been received by responding with a DO_PING_RESULT message with a positive result. This indicates that the current Node is in the remote Node s Network. Otherwise, the current Node is not, but can access that network. Thus, the Node can establish a connection between these Networks. However, first the test for the current network will be completed. If the Node is not located in any of the known networks, it will use a random NetworkID. 4.2 Gateways When a Node identifies itself as a potential Gateway between Networks because it resides in both Networks, the node should publish itself as a Gateway by sending a NEW_GATEWAY message containing the NetworkID of the remote Network to some other Nodes in the same Network from its Node Table unless this is prohibited by its configuration. Upon reception of such a message, a Node will check its Gateway and Node Tables and update it with the new information. If the Gateway was already known, the message is discarded, otherwise it will be forwarded to other local Nodes. However, each Node must use the GET_GATEWAYS message first to ask a local Node if there are already enough such Gateways published. Currently, HEN Nodes only publish themselves as a Gateway when less then 16 other Gateways to the same Network are known. This avoids useless traffic in situations where actually are Nodes

Discovery and Routing in the HEN Heterogeneous Peer-to-Peer Network 659 in one Network could establish a Relay-Connection (e.g., this is the case for the firewall scenario). Anyway, this mechanism still currently holds back the approach as it has scalability issues especially for large networks connected to many very small networks (i.e., the internet connected to a lot of NAT users). For Sub-Networks, each Node is its own Gateway to the parent Network, but does not publish this information as this is a property of the sub-network. However, each of these nodes needs to open a persistent connection to the parent network to enable bidirectional communication. The relay node will start acting as a proxy for the original Node. Thus, no Gateway announcement is necessary as the Node is now effectively a member of the parent network. This actually avoids the broadcast problem for most real life configurations. 4.3 Discovery and Routing Routing in the HEN network is based on finding a route to the destination node based on the Nodes and Gateways Tables. These tables must contain a path to the destination Node where each hop is a Network. If no such path exists or the destination node is unknown, the HEN node will start a discovery to find such a path. Fig. 3. Discovery of a path (ACDEB) from 1234 to 8765 Discovery of a Node in a Network starts by attempting to locate that node using a variant of the PRR approach. Thus, the discovery is based on sending a DISCOVERY message that traverses the local network by moving from each Node to a Node sharing a longer prefix with the requested Node than the current Node does. If a Node has an entry for the destination Node in its Nodes Table and this entry is newer than the information contained in the DISCOVERY message (if any), it replaces this information. If the message addresses a Node in a Network containing the Node currently processing the message, it will attempt to forward the message to that Node. If this fails, all information in the DISCOVERY message is cleared. If no more Networks are reachable via Gateways, the message is discarded. Once the message arrives at the desired destination node, the node responds with a ROUTE message containing all networks passed as in the DISCOVERY message. This message travels back the path along the Gateways from the DISCOVERY messages and updates the forward table of each of the Gateways accordingly. At each Node, when a packet has to be sent its final destination address is examined. A route to this address is computed from the Nodes and Networks Tables. The next hop on this route is a Network because otherwise the message would have been delivered directly to a local Node. This Network ID is used to determine a Gateway to the Network which will be used as the next hop and the message will be transmitted. If this fails, the Gateway will be marked as down and another Gateway

660 T. Schattkowsky will be used. This next hop NodeID and NetworkID are cached to be directly available when the next packet for that node arrives. Figure 3 shows example with simplified NodeIDs where Node A (1234) attempts to discover Node B (8765). Node A is in an Enterprise Network with unlimited access to the Internet (i.e., NAT). All nodes within this network have established relay connection and proxies. These are not shown here, as the proxies are only involved in receiving a message, as shown for Node E which has Internet access using an HTTP proxy while Node B is in a protected enterprise network. This Node uses Node D as a relay and proxy for E, which is a Gateway like Node C. Discovery starts by moving to Node 8444 and 8721 sharing an increasing part of the destination NodeID. However, no Nodes with a longer prefix exist. Thus, the search moves through a Gateway (8721 itself) into the Internet and ends again at 8762. Here the search moves to Node D as a Getaway into EnterpriseNetB(HTTP) and end continues on E. As E is also in EnterpriseNetB(TCP), it initates the search there which finally leads to B. Node B responds by sending a ROUTE message the same path back resulting in all Nodes along the path to receive the necessary updates Fig. 4. Resulting Path from A to B for their tables to directly transport packets to B as needed. The resulting path consists of all Gateways taken and is shown in Figure 4. This situation is modelled in the communication model in Figure 2. 5 Conclusions and Future Work We have introduced HEN as a self-organizing de-centralized peer-to-peer network spanning heterogeneous networks with communication barriers. HEN demonstrates how the concept of self-organizing networks can be extended to support heterogeneous networks with multiple protocols, relay and routing. HEN has been implemented in JAVA and successfully applied to interconnect Web services in a collaborative engineering environment. Future work will include a layer 3 transport protocol implementation and detailed simulation work on the network behavior. References [1] Clarke, I., Sandberg, O., Wiley, B., Hong, T.W.: Freenet: A Distributed Anonymous Information Storage and Retrieval System. In Proc. of the ICSI Workshop on Design Issues in Anonymity and Unobservability, Berkeley, CA, 2000. [2] Hildrum, K., Kubiatowicz, J. D., Rao, S., Zhao, B. Y.: Distributed object location in a dynamic network. In Proc. SPAA, August 2002.

Discovery and Routing in the HEN Heterogeneous Peer-to-Peer Network 661 [3] Liu, H., Lam, S. S.: Consistency-preserving neighbor table optimization for p2p networks. TR-04-01, Dept. of CS, Univ. of Texas at Austin, January 2004. [4] Plaxton, C. G., Rajaraman, R., Richa, A. W. : Accessing nearby copies of replicated objects in a distributed environment. In Proc. of ACM SPAA. 1997. [5] Rekhter, Y.: An architecture for IP address allocation with CIDR. RFC 1518, 1993. [6] Rowstron, A. I. T., Druschel, P.: Pastry: Scalable, distributed object location and routing for large-scale peer-topeer systems. In Proc. Middleware 2001, 2001. [7] Schattkowsky, T., Loeser, C., Müller, W.: Peer-To-Peer Technology for Interconnecting Web Services in Heterogeneous Networks. In Proc. 18th International Conference on Advanced Information Networking and Applications (AINA 2004), April 2004. [8] Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for Internet applications. TR-819, MIT, March 2001. [9] Zhang, H.. Goel, A., Govindan, R.: Incrementally improving lookup latency in distributed hash table systems. In Proc. of SIGMETRICS, June 2003. [10] Zhao, B. Y., Huang, L., Stribling, J., Rhea, S. C., Joseph, A. D., Kubiatowicz, J. D.: Tapestry: A resilient global-scale overlay for service deployment. IEEE Journal on Selected Areas in Communications, Vol.22(No.1), January 2004.