PSON: A Scalable Peer-to-Peer File Sharing System Supporting Complex Queries

Transcription

1 PSON: A Scalable Peer-to-Peer File Sharing System Supporting Complex Queries Jyoti Ahuja, Jun-Hong Cui, Shigang Chen, Li Lao jyoti@engr.uconn.edu, jcui@cse.uconn.edu, sgchen@cise.ufl.edu, llao@cs.ucla.edu UCONN CSE Technical Report: UbiNet-TR05-01 Last Update: July 2005 Abstract In this work, we design a P2P file sharing system called PSON to address the important issues of scalability, routing efficiency and complex query support. We propose a semantic overlay network of logical nodes, in which queries are routed on the basis of semantics. A logical node is formed by a cluster of peers that are close to each other in the physical network. Each cluster selects a powerful peer as super peer to support routing in the overlay network. All the super peers are organized in the form of a balanced tree. By exploiting the concepts of hierarchy and semantics, PSON can support complex queries in a scalable and efficient way. We conduct extensive simulations and the results show the promising performance of PSON. I. INTRODUCTION In recent years, the peer-to-peer (P2P) computing model has gained tremendous popularity in the research community as well as the industry [7] [23] [24] [13] [11] [12] [21]. One of the key incentives is probably its high scalability. In the traditional client-server computing model, each node or process is either client or server, and clients mainly depend on servers for resources. In this model, servers are the system performance bottleneck, and thus it does not scale well to large numbers of clients. By contrast, in the P2P computing model, each node has equal responsibilities. Nodes (or end users) at the edge of the network (called as peers) can provide as well as use services, and there are no centralized servers. In this model, resources at peers are better utilized; thus, when there are large numbers of peers, the system performance will be improved significantly since all the peers resources will contribute. In other words, the P2P model is highly scalable. Among various applications of the P2P technology, P2P file sharing systems are probably the fastest growing and the most popular Internet application. It is recorded that P2P file-sharing accounts for more traffic than any other application include the Web on the Internet [15]. A successful P2P file sharing system should achieve the following design goals. First, the system should scale to large numbers of peers spreading throughout a wide network across different administrative domains. Second, the system needs to provide an efficient and effective file lookup (or search) scheme. It should return the location (e.g., the IP addresses of the peers who have the file) of a requested file with minimal communication and computation 1

2 overhead. Third, the system should support complex (or partial-match) queries. Shared files have attributes that describe their properties, e.g., singer, composer and title for a music file. It is desired that the search mechanism could support partial match queries that contain a subset of the attributes and may even contain typos. There are many P2P files sharing systems in the literature [34] [30] [31] [26] [22] [20] [29] [16] [2] [10] [1] [5] [25] [8] [19] [14] [4] [3]. And some of them have been implemented and even widely used. However, few systems could achieve the above design goals simultaneously. Among the initial designs of P2P file sharing systems, most are not scalable. For example, in Napster [34], shared files of all participating peers are indexed in a central directory server. An application instance running in a new peer informs this directory server of its IP address and names of the objects it is making available for sharing. To retrieve a file, a peer request searches at this central server using the desired file s attributes and obtains the IP addresses of the peers who are sharing this file. The requesting peer pings these peers and looks for the best data transfer rate. The file transfer is then performed directly between the peers. This centralized architecture makes it difficult to scale the central directory and is vulnerable because of its central point of failure. To avoid centralized servers, Gnutella [30] employs a decentralized method of locating files. The peers first self organize into an overlay network which has a flat and unstructured topology. The application instance of a peer serves to store an index of files the peer is sharing. It also routes query from and to its neighboring peers in the overlay network using query flooding. Flooding on every request is obviously not scalable as the query propagates to every other peer in the overlay network leading to a huge amount of traffic. And if the routing is curtailed, it may fail to locate a file that is actually shared by the system. In response to these scalability problems, several new structured P2P systems, such as CAN [20], Chord [26], Pastry [22], and Tapestry [29], to name a few, have been proposed that support distributed hash table (DHT) functionality. A DHT-based protocol implements one basic operation: lookup (key) which returns the identity (e.g., IP address) of the peer currently responsible for the given key. In general, these systems are much more scalable than the initial P2P file sharing designs: applications are able to locate objects in log(n) hops where n is the number of peers in the overlay network. However, there are some fundamental limitations of these systems. First, they only support exact match queries. Second, the search mechanism is blind i.e. independent of the query context. Moreover, their performance under dynamic conditions common for peer to peer systems is still unknown [20]. To address the issues of scalability, efficiency and complex query support in existing solutions, in this technical report, we present a system, called Peer-to-peer Semantic Overlay Network file sharing system (PSON). The basic design essentially explores the concepts of hierarchy and semantics. In PSON, peers are classified into normal peers (or just peers) and super peers. Powerful nodes can serve as super peers. Normal peers connect to a super peer and together form a cluster. If we treat each cluster as a logical node, then all logical nodes form an overlay network (referred as to semantic overlay) in which queries are routed on the basis of semantics. Each of the logical nodes (i.e., a cluster) has a directory of content to manage. Peers belonging to a cluster share the directory of content that is assigned to the cluster. In this way, when a peer wants to publish a file, it first extracts metadata and generates 2

3 a location-metadata pair (i.e., a directory item), then the directory item is distributed into the semantic overlay network, and finally it is stored in some peer. Similarly, when a peer wants to search some file, a search query is first issued, and then forwarded to a cluster that is responsible for storing the directory related to the requested content. By a local search in the cluster, the location for the requested item is obtained. PSON is designed as a scalable and efficient file sharing system supporting complex queries. Though the basic idea is not complex, to make the system work effectively and efficiently, there are many important issues to address, among which we identify four as follows. How to construct a semantic overlay? The semantic overlay should reflect the semantics of shared files, while facilitating query routing at the same time. How to handle the file dynamics and node dynamics? Peers join and leave, and files are inserted and deleted. These dynamics should not hurt the quality of the semantic overlay; in other words, the overlay should be stable in the presence of high dynamics. How to conserve the robustness of the P2P model? When some peer fails or even super peer fails, the overlay should still function without much performance degraded. How to well balance the load of the peers in the network? Load balancing is one of the key concerns in P2P system designs. Will the semantic overlay achieve the goal of load balancing when it introduces semantics? In this technical report, we address all these design issues of PSON. To build an effective and efficient semantic overlay, PSON employs a balanced binary search tree structure (red-black tree), for which an in-order tree walk could yield an encoded semantic tree 1, as helps to conserve the semantics of the shared content in a very elegant way. Since the searching time of red-black tree is bounded by O(log(n)), the routing (logical) hops of the semantic overlay is bounded. Then, centered on the basic semantic overlay structure, we propose solutions for the issues of (node and file) dynamics handling, fault tolerance, and load balancing. Through extensive simulations, we demonstrate that the proposed system, PSON, is capable of supporting complex queries very efficiently. The rest of the technical report is organized as follows. In Section II, we review some background and related work. Then, in Section III, we present the PSON architecture and discuss the basic design issues of PSON, including overlay construction, overlay routing, and system dynamics. In Section IV, we propose some techniques to enhance the performance of PSON in the aspects of load balancing and fault tolerance. Finally, we conduct comprehensive performance evaluation in Sections V, and conclude the technical report in Section VI. II. BACKGROUND AND RELATED WORK As mentioned earlier, PSON explores both the concepts of hierarchy and semantics. In this section, we first present some background on semantic hierarchy and hierarchical P2P systems, then we describe some related work. 1 We will present the concept of semantic tree in Section III-A 3

4 A. Background 1) Semantic Hierarchy: Content shared in a file system can usually be semantically classified into a hierarchy (referred to as semantic hierarchy or classification hierarchy in this technical report) based on genres. For example, music files can be first categorized into jazz, pop, rap, and rock, etc, and rock music files can be further subdivided into soft rock and hard rock. Each subcategory can further create a sub-hierarchy based on composer and year etc. This kind of content classification is proved very useful in content organization, especially in centralized databases. However, in P2P systems, semantic hierarchy is not widely explored yet. The paper by Crespo et al. [6] leads a pioneer work in this direction though. In this work, nodes with semantically similar content are clustered together and form an overlay network. The authors used a classification hierarchy based on genres obtained from All Music Guide [32] to form the semantic groups of the documents being shared by the participants. There is one overlay network for each semantic group in the classification hierarchy; therefore, a node joins all the overlay networks for which it has semantically related content. Flooding or a centralized directory is used to find the nodes that belong to those overlay networks. Search queries are classified using the same criterion as that used to form the semantic groups and are then routed to the overlay networks which have a high probability of containing matching items. The nodes in these overlays can again be found using flooding or a centralized directory. Our work bears some similarity with the work by Crespo et. al. in terms of the concept of classification hierarchy to classify documents. However, the mechanisms to distribute content and route queries are totally different. 2) KaZaa: A Hierarchical P2P System: PSON explores the heterogeneity of peers. It divides peers to normal peers and super peers. And normal peers connect to a super peer and together form a cluster. This approach is similar to KaZaa [31], an enormously successful P2P file sharing service. In KaZaa, the peers and their communication relationships form an abstract, logical network called an overlay network. Peers that have more connection bandwidth and better availability are designated as super peers. Each super peer keeps track of IP addresses of its descendants and the content they are sharing in an index. A list of potential super peers is included with the downloaded software. A new peer bins itself with the operational super peers, and establishes an overlay connection with the super peer that has the shortest RTT. A peer submits a query to its super peer which first checks index to see if another peer in its own territory can provide the object. Otherwise, it forwards the query to its neighboring super peers in the overlay network. KaZaa is more scalable compared with Napster [34] and Gnutella [30]. However, it cannot support complex queries as the queries are routed irrespective of their content. Moreover, the communication among super peers is not well organized and thus flooding or partial pooling is used, which is inefficient. B. Related Work Several strands of work in the related areas can be distinguished. 1) DHT Based Systems: DHT based P2P systems provide efficient retrieval of data items (at least for exact-match queries). The main idea behind these systems is to assign peers to hold particular content or pointers to it. When a 4

5 peer requests for some content, the query is routed to the peer that is supposed to have or know about it. A hash function is introduced to map the object being searched for to a unique identifier and the range of this function is distributed among all peers in the network. Different DHT based systems (CAN [20], Chord [26], Pastry [22], Tapestry [29]) differ only in the routing algorithm. The routing algorithms in these systems are blind, i.e., they do not take semantics of the query into account and because of that they are only able to provide exact-match query facility. 2) Hierarchical Systems: There is a large amount of work on document clustering using hierarchical systems [18]. However, most of these assume that the documents are part of a centralized database. Clustering algorithms for distributed architectures again depend on crawling the data into a centralized site and then using clustering methods to make the web results accurate and easier to understand. This type of systems have a central point of failure like Napster [34] and are usually not scalable to a large number of users. The concept of hierarchy was also employed by L. Erice et al [9]. They proposed to use two tier DHTs in such a way that peers organized in the same group are geographically close. Each group maintains its own overlay network and intra-group overlay service. A top-level overlay is defined among the groups. Within each group more powerful peers are designated as super peers which are used by the top-level overlay to route messages among groups. Chord is used for the top-level overlay. Thus, this system inherits all the regular disadvantages of DHT based systems, i.e., blind searches and no support for complex queries. 3) P2P Systems Supporting Semantics: Recently, there are several proposals, such as psearch [27], and SSW [17], which support semantics in P2P systems. We discuss each of them in the following. In psearch [27], semantics of a document are generated by applying Latent Semantic Indexing (LSI) on a term (or semantic) vector which is generated from the document by using Vector Space Model (VSM). CAN is then used to create an overlay by using the document semantics as the key to store document index. In principle, this work extends the classical IR (Information Retrieval) algorithms to the P2P environment so as to provide content based search. In other words, the semantics in psearch are basically abstracted from the document content. SSW [17] employs a similar approach to the one used in psearch to generate semantic vectors. In SSW, peers are clustered according to the semantics of local data and self-organized as a small world overlay network. Further, a dynamic dimension reduction method is used to decrease the dimensionality of this overlay network. One of the fundamental differences between these two systems and PSON is the concept of semantics. Authors in psearch and SSW have used the terms content and semantics interchangeably. Semantics in our system refer to the meaning of a document, or more precisely, its content classification. And the semantic entry of a document is corresponding to a subcategory, which the document belongs to, in the classification (or semantic) hierarchy. However, in psearch and SSW, the semantic vector of a document is generated from various terms (e.g., concept or key word for a text document) in the document, that is, a semantic vector is more a reflection of the content of a document rather than its semantics. The basic motivation behind using semantics instead of content in our system is to be able to answer partial match queries, and the ability of psearch and SSW to solve such queries is 5

6 not clear. III. PSON ARCHITECTURE In this section, we first give an overview of PSON and discuss the principles to construct semantic overlay. Then, we present in detail the basic design issues of PSON that includes algorithms for semantic overlay construction and query routing. We also discuss how system dynamics affect the overlay. A. Overview PSON is a self-organizing semantic overlay network, which is composed of a number of logical nodes. A logical node here stands for a cluster of peers that are geographically close to each other. A powerful peer (e.g., with more bandwidth and/or more availability) in the cluster is selected as a super peer. Different from KaZaa, the super peers are not necessary to maintain the local directories; instead, they are mainly responsible for routing in the network: communicating with other super peers and coordinating local normal peers. The overlay network among super peers is well organized based on semantics. Before demonstrating the semantic overlay network construction, the concept of semantic tree is elaborated as follows. 1) Semantic Tree: Files shared in PSON are classified into a semantic hierarchy. Fig. 1 gives a simple example. The illustrated hierarchy has a tree structure. 2 Each tree node represents a semantic entry. Each semantic entry can be denoted as a tuple with three attributes: semantic label, semantic level and ancestor list. For example, in Fig. 1, Root, music, movie, jazz, rock, etc are all semantic labels which characterize the class of files. Semantic level is the tree level of the semantic entry in the semantic tree. Ancestor list of a semantic entry is the list of all its ancestors semantic labels and levels. For instance, the three attributes of the semantic entry for pop would be: Semantic Label: Pop Semantic Level: 2 Ancestor List: Music 1, Root 0 2) Ordering Semantic Entries: With three attributes, semantic entries can be easily compared and ordered. For any two semantic entries, if they have the same parents, their order is decided lexicographically. If they have different parents, then find their lowest-level uncommon ancestors (according to their ancestor lists) and compare them lexicographically. For example, referring to Fig. 1, games is smaller than movies since they have the same parent root and g is smaller than m. Similarly, we can easily tell comedy is smaller than pop because their lowest-level uncommon ancestors are movies and music respectively, and movies is smaller than music. 2 Semantic tree is probably the most popular and useful classification hierarchy. Other classification hierarchies are possible (see [6]). In our work, we use the tree structure to demonstrate the PSON design concept. 6

7 Root games movies music comedyhorror jazz pop rap rock Fig. 1. An example of semantic hierarchy 3) Semantic Overlay Construction: The basic design philosophy of PSON is to construct an overlay based on semantics. In other words, PSON organizes various semantic entries into an overlay. Once we fix the overlay structure, the simplest way for overlay construction is to do a direct mapping. For example, we can easily construct an overlay tree which matches exactly the semantic tree: each super peer (or cluster) maintains the directory of files which belong to one class denoted by one semantic entry; super peers forms an overlay tree with the same shape as that of the semantic tree. However, search in such an overlay will not be efficient. Moreover, we have to keep in mind that P2P file sharing systems have high dynamics: files are inserted and deleted, and peers join and leave. To maintain the dynamic P2P system in an efficient and scalable way is very challenging. Correspondingly, choosing a good overlay structure, and mapping the semantic tree to the overlay are the key issues in the system design. Overlay Structure In our current design of PSON, we use balanced binary search tree (i.e., red-black tree) for overlay structure. It is possible to employ other overlay structures, such as meshes and rings. We choose red-black tree because of its many characteristic features: in-order walk yields an ordered list; search time is bounded by O(log(n)); and tree maintenance is simple. From Semantic Tree to Overlay Tree In PSON, the red-black overlay tree is built in an incremental fashion. In the overlay tree, each cluster (or logical node) corresponds to one semantic entry in the semantic tree (i.e., the peers in the cluster maintain the directory of the files belonging to the semantic class). However, the mapping clusters of semantic entries only appear in the overlay when necessary. For example, at the starting stage of the system, only few files are available for sharing, and there is no necessity to build a big overlay. In this case, maybe only one cluster is enough, and thus the overlay tree is actually a degraded tree with one node (which corresponds to the Root semantic entry). Later on, with more music files available, we fork a cluster which stores the music file directory. And the overlay now grows to a two-node tree. But how to connect the new logical node to the existing overlay? Remember we use binary search tree as overlay structure. Thus, the new node is inserted to the existing tree based on its semantic ordering. For instance, music is smaller than Root, then the node storing music files will be on the left side of the node storing Root files (or unclassified files). In such a manner, we can construct a semantic overlay with the form of red-black tree. The details of the overlay construction algorithm 7

8 Fig. 2. The physical view of a PSON system Fig. 3. The logical view of a PSON system will be presented in Section III-B. To further illuminate the concept of semantic overlay tree, consider Fig. 2 and Fig. 3. Fig. 2 gives a physical view of a PSON system, while Fig. 3 illustrates the logical structure of the same PSON system. The corresponding overlay tree is shown in Fig. 4. Note that in the overlay tree, the root node is the cluster representing rap, while the logical node for the semantic entry Root becomes a leaf node. In the following, we discuss the two basic PSON design issues: overlay construction and overlay routing. We rap movie music comedy pop rock Root games horror jazz Fig. 4. The structure of the overlay tree 8

9 Root music music rap music movie music Root rap Root rap Root rap music movie movie music Rotation rap Root pop pop Root movie pop Fig. 5. An example of overlay node insertion also analyze the complexity of the algorithms, and discuss the issue of peer and file dynamics. B. Building a Semantic Overlay In this subsection, we describe how to organize the super peers (i.e., clusters, or logical nodes) into a semantic overlay network (in the form of red-black tree). As to the scenarios in which new clusters are formed, we will discuss in later sections. As we mentioned earlier, the overlay network is built in an incremental fashion. Clusters (or logical nodes) are inserted in a way that they form a balanced tree structure. The insertion procedure has two parts. First, a logical node is inserted according to its semantics, i.e., according to its semantic entry; and second, an insert fixup procedure is invoked so as to ensure the red-black tree properties. The insert fixup method may include some rotations of the tree to make it balanced. Every node in the overlay tree keeps pointers to its left and right children, parent, and its smallest and largest descendant in its (right and left) subtrees respectively. Smallest and largest peers pointers are kept for query routing purposes. Algorithm 1 shows the logical node insertion procedure. In this algorithm, the pointers of a super peer node are denoted by node.left, node.right, node.parent, node.smallest, and node.largest. root is the root of the overlay tree, while Root is the root semantic entry in the semantic hierarchy. Here we explain the overlay construction algorithm with the help of an example (depicted in Fig. 5). If we read through Algorithm 1, we can conclude that in PSON overlay, semantic entry Root is always the first node inserted with the level 0, and a new node can only be created by its semantic parent. This design is motivated by our intuition: if there are few files in one subcategory, then it is not necessary to construct one overlay node to store the corresponding directory information. Instead, the parent category node should be sufficient. Now, referring to Fig. 5, initially, Root is the only node in the system and handles all kinds of files (See lines 5-9 in Algorithm 1). Then suppose that the number of music files increases beyond a certain number. Root initiates a new node insertion in the overlay that has semantic label music and level 1. The directory of music files is given to this 9

10 Algorithm 1 Overlay-Insertion(newnode) 1: //, and denotes semantic comparison 2: Contact bootstrap node, B 3: Add newnode s address to the super peer list in B 4: B checks if the new node is the first node (excluding root) in its list of super peers 5: if true then 6: root=newnode 7: newnode.right = Root 8: newnode.largest = Root 9: Root.parent = newnode 10: else 11: randomly selects an existing super peer, and return address to newnode 12: newnode contacts this super peer(referred to as oldnode ) 13: semantic comparison(newnode, oldnode) 14: if newnode oldnode then 15: A: 16: if (oldnode.smallest = NULL) or (newnode oldnode.smallest) then 17: if oldnode.parent = NULL then 18: make newnode as oldnode s smallest descendant 19: else 20: oldnode = oldnode.parent 21: semantic comparison(newnode,oldnode) 22: if newnode oldnode then 23: make newnode as the smallest child of oldnode.right 24: else 25: go to A 26: end if 27: end if 28: else 29: insert lexicographically to oldnode.left 30: end if 31: else 32: same as if clause with right and left exchanged and smallest and largest exchanged 33: end if 34: end if 10

11 new node which can further create its own semantic children. Rap is the next node to be inserted and its insertion is initiated by its semantic parent music. Music first checks to see whether rap lies in its own left subtree by comparing rap semantically to its smallest descendant (Line 16). Since music does not have a left descendant in the current overlay, it checks to see if rap belongs to its parent s left tree or not (Line 17). music does not have a parent either because it itself is the root of tree; thus, it makes rap its own left child (Line 18). movie and pop are then inserted by their respective semantic parents, Root and music according to Lines Clearly, the overlay tree is not balanced after the insertions and we now need to make it balanced so as to be able to support efficient routing. A right rotation is performed by rap to restore the balance of the overlay tree. The subtree at rap gets one level closer to root and the subtree at music gets one level further from the root of the overlay tree. As shown in Fig. 5, the final overlay tree is balanced. In red-black tree, both right and left rotations are local operations, and they run in O(1) time. Rotations are called when nodes are inserted or deleted, and the tree becomes unbalanced. Since each node in a red-black tree maintains a color bit, no global information (for example, the hight of the (sub)tree) is needed to decide if a rotation should be performed or not. Thus, rotations in PSON overlay trees are distributed. Maintaining the Semantics In Fig. 5, the overlay nodes are inserted in the following order : Root, music, rap, movie, pop. Fig. 4 is resulted when all nodes of Fig. 1 are inserted. Its in-order walk is: games, comedy, horror, movie, jazz, pop, rap, rock, music, root. Though the overlay tree depends on the order in which individual nodes are inserted, it is made sure that for the same nodes, in-order traversal of tree yields the same semantic ordering of the semantic entries irrespective of the insertion order, thus keeping the semantics. Overlay Node Deletion The overhead of overlay node deletion is comparatively less than that of insertion. The reason is that for insertion, the new node s position in the overlay is searched first and then the fixup is performed. While in the deletion case, the node leaves the overlay directly (at the maximum, it tells its semantic parent that it is leaving the overlay). Then, a deletion fixup mechanism is invoked to restore the red-black tree properties of the overlay which may again involve some rotations. C. Query Routing In this subsection, we discuss the mechanism of locating a file in the overlay tree. The peers in the system, depending upon their roles, are of two types: super peers and normal peers. A super peer is a special peer that participates in locating files. By selectively using only the super peers for directory lookup, we narrow down the search scope, as search requests are exchanged between the super peers only. Every super peer maintains two directories: a local directory and a look up directory. While the contents of local directory indicate whether or not the requested file is available within its own territory, the look up directory is used to forward queries to other super peers. A normal peer on the other hand just stores a part of the directory belonging to its cluster. It is a 3-step method to find the IP addresses of the peers that have the requested file. Here, we are assuming 11

12 that the super peer of the cluster maintains a local directory of the files being shared by the member peers of its territory. The requesting peer submits a query to its cluster s super peer. This super peer first checks its local directory to quickly determine if the requested file is shared by someone from its own cluster. In case of a hit, the IP address of the peer sharing this file is directly returned to the requesting peer. For a miss case,the super peer determines whether the directory of content related to the requested file is managed by its own cluster. If it does then the query is broadcasted to all members of the cluster. Peers that have the desired information responds with the results. Finally, the super peer invokes an overlay routing procedure and forwards the query to another super peer, which may be its child super peer, parent super peer or the smallest or largest super peers in its subtrees. That super peer now checks for the information in its own cluster and so on. Eventually the query reaches a super peer of a cluster that has the desired information. After local searching, this node then returns the addresses of the peers that actually have the required file to the node which initiated the request for the same file. The requesting node can now initiate a direct file transfer request with the nearest live peer that possesses this file. Overlay query routing is shown in Algorithm 2. Algorithm 2 Overlay Query(oldnode,query node) 1: //, denotes semantic comparison 2: if oldnode query node then 3: if query node oldnode.smallest then 4: send to oldnode.parent 5: else 6: go to left 7: end if 8: else 9: if query node oldnode.largest then 10: send to oldnode.parent 11: else 12: go to right 13: end if 14: end if To illustrate, consider a query for rock songs from a cluster that is responsible for the directory of comedy movies. comedy is compared with rock (Line 2) and since rock is greater than comedy, it is now compared with the largest of sub tree at comedy (Line 9) i.e. horror. rock is even greater than horror, therefore, it is further forwarded to comedy parent, movie (Line 10). movie performs the same comparison again (Line 2) and routes the query accordingly. The complete query path from the contacted node comedy to query node rock is shown in Fig

13 rap movie music Query for rock comedy pop rock Root games horror jazz Fig. 6. An example of query routing. Dashed arrows denote the query path. D. Complexity Analysis From the above presentation, we can easily know that the complexity of both the overlay construction and routing algorithms is O(log(n)) due to the fact that red-black tree is utilized as the overlay structure (where n is the number of overlay nodes). If we assume there are totally N peers, and the average cluster size is M, then the average search path length is O(log(N/M)). If we compare with the average search path length of SSW 3, which is O( log2 (2N/M) l ) (where l is a constant, and is set as 4 in the evaluation of SSW [17]), we can conclude that query search in PSON is much faster than SSW in average. E. How System Dynamics Affect the Overlay? One of the key characteristics of a P2P system is high dynamics: peers join and leave, and files are inserted and deleted. How will these system dynamics affect the overlay? In this following, we present how to handle peer and node dynamics in PSON. 1) Peer Dynamics: Peer Join Addition of a peer to the system may lead to a new cluster formation. A new cluster is created in the following events: 1) when there is no candidate cluster for the newly joined peer to join; and 2) when the number of member peers in the desired cluster exceeds its maximum limit. We propose to keep a maximum size limit for a cluster so as to achieve better search performance and to minimize cluster s maintenance overhead. A cluster is not assigned a directory to manage until the number of peers in that cluster exceeds a minimum threshold. This is necessary because there would be a lot of overhead in storing a directory due to excessive dynamics if the number of peers in a cluster is too small. The system ensures that the existing cluster handles all requests while the new cluster is being created. This is significant as it can provide continuous service during configuration changes. Peer Leaving Successive peer leave events from the same cluster can lead to the cluster decay. If the number of peers in a cluster decrease beyond a certain threshold then the directory assigned to that cluster is revoked, and may be moved to another cluster, because it will be a lot of overhead for a small number of peers to handle 3 We have identified the fundamental difference between SSW and PSON in Section II. Here, we simply compare the average search path length of the two systems. 13

14 a considerably sized directory. Further, if there are two such clusters physically close by in the network, we ll combine the two and a new directory is assigned to the merged cluster. On the other hand, if there is no cluster available for merging, the first one remains without a directory to manage and waits for new member peers. 2) File Dynamics: File Insertion Addition of files to a directory may lead to its split. We propose to keep the directory size within a maximum limit so that if the current size increases beyond that limit then the directory is split into two and the other half is assigned to some other cluster. This is performed so as to improve the search performance and keep the overhead of directory management low. File Deletion Successive file deletion from a single cluster may lead to significant decrease in directory size. In this scenario, we plan to merge two such directories so that member peers of that cluster have enough content to manage or we can merge the shorter directory with its semantic parent. It also helps to achieve file load balancing amongst clusters. IV. PSON ENHANCEMENTS In this section, we present some techniques to enhance the performance of PSON in the aspects of load balancing and fault tolerance. A. Load Balancing In PSON, we can classify the load balancing problems into query traffic load balancing and file load balancing. 1) Query Traffic Load Balancing: Since PSON s overlay is structured as a balanced binary search tree (i.e. red-black tree), it is natural to think that approximately half of the queries will traverse from one half of the tree to other half. This phenomenon may lead to a lot of query traffic at root and at high level nodes of the overlay tree. We propose to add cross links to solve the problem. Where to add those links? We ask the queries themselves to tell us. First, if the queries are biased (not uniformly distributed, which is likely the case), then we are able to add cross links geared towards majority queries. Second, if there is locality in queries (similar queries are made by a user), then the cross link established by the first query will immediately help the following queries. There are four strategies for adding cross links. A query has a source and a destination. Source is the (logical) node that makes a query. Destination is the (logical) node that has the answer. Strategy I: Add a cross link from the source to the destination after the query. Strategy II: Add a cross link from the source to an arbitrary node sampled from the query path. Strategy III: Add a cross link from an arbitrary node sampled from the query path to the destination. Strategy IV: Add a cross link between two arbitrary nodes sampled from the query path. We make a uniformly random sample. We first sample the source and then replace the sample with certain probability as we travel on the query path. The probability is 1/k, where k is the path length so far. This gives every node on the path an equal chance to be sampled. 14

15 Among the above four strategies, if the source and the destination are leaves, the cross link between them is less likely to be used by queries from other sources, which makes Strategy I less effective. For the other strategies, Strategy IV is actually a superset of Strategy II and III. Consider many queries made to a popular destination cluster. The paths of these queries are from everywhere and converge to the destination, forming a tree rooted at the destination cluster. The resulting cross links (using Strategy IV) are also from everywhere and converging to the destination, cutting short the tree. Thus, we adopt Strategy IV in our design. Discussions After adding cross links, for overlay routing, we need to check if there exists a cross link before going up to the parent, and each (logical) node may keep a list of cross links. In addition, not every query has to generate a new cross link. Instead, each query has a certain predefined probability to create a cross link. If a new cross link is from x to y, then it will replace the oldest cross link stored at x if x s cross links have already reached the pre-defined number. 2) File Load Balancing: Since we have discussed the file load balancing among clusters in Section III-E.2, here we mainly focus on the file load balancing in a cluster. Each cluster in PSON has a directory of location-metadata pairs. Peers belonging to a cluster share this directory distributively. Since the behavior of peers in a P2P system is dynamic, we need to make sure that the load at each peer is balanced. Peers in a system generally have heterogenous capacities to handle load. For instance, a peer having a DSL Internet connection might be ready to handle large load compared to a peer having a dial-up connection. The underlying assumption in our scheme is that a peer notifies the super peer of its current load target. A load check algorithm is invoked by a super peer that verifies if the load on each of the peers is proportionate to their specified target values or not. If not, the algorithm allows dynamic re-mapping of load in the system. This check is performed periodically in each cluster. Also, if the peer itself realizes that the recent directory item additions have increased its load, it can inform its super peer which then invokes the load check forcefully. Alternatively, we can also explore a Chord [26] like setting in each of the clusters for load balancing purpose. It is important to make sure in the overlay that we are not punishing one cluster (or logical node) by giving it a big directory and rewarding another one with a smaller directory to manage (both the directories are still under the maximum and minimum threshold values). An approach similar to above described one can be employed to achieve load balancing amongst different clusters. A crawler like program checks directory load at each of the clusters and if needed, performs dynamic re-balancing of load at clusters. The re-balancing may involve splitting a directory and merging two directories. B. Fault Tolerance P2P systems are characterized by their highly dynamic nature. So, whenever a peer fails or even super peer fails, the overlay should still function without much performance penalty. In the following, we examine both the scenarios: normal peer failure and super peer failure, and we also propose an approach to enhance the robustness of the overlay tree structure. 15

16 Normal Peer Failure Our basic idea is to have a Chord [26] like arrangement of peers within a cluster. Since Chord is proved to be robust to membership changes and node failure, we expect a similar kind of robustness to node failure in a PSON cluster. Super Peer Failure We propose to have backup shadow nodes for a super peer. When a super peer is dead, a shadow node (which has the identical overlay connectivity information) is elected as the super peer. If the number of shadow nodes is below some threshold, some normal peers will be elected into the shadow node pool. Note that, at normal time, shadow nodes can also share the load of the system. Making Overlay Tree Robust It is commonly accepted that a tree structure is not robust against partition. Though we believe that the probability for this type of system failure in PSON is small (given the strong super peer backup mechanism), we propose to add grandpa and grandchildren links in the overlay tree to enhance its robustness. V. SIMULATION In this section, we evaluate the performance of PSON by extensive simulations. We compare PSON overlay with flooding overlay, the state-of-art unstructured P2P overlay. We first describe flooding overlay and discuss the reasons for experimentally comparing it with PSON. Then we report our simulation setup, different parameters and performance metrics. Finally, we present results from a range of simulations. A. Flooding Overlay As we mentioned in Section II, the two recently proposed semantic-based P2P search mechanisms psearch [27] and SSW [17] are fundamentally different from PSON, because they build the systems based on content instead of semantics (which is used by PSON). To the best of our knowledge, there is no other system (except the work reported in [6], which uses flooding overlay) that builds a semantic based overlay and then uses that to support complex queries. Thus, we chose to experimentally compare our system with flooding overlay which is explained below. In a random unstructured overlay, nodes are randomly connected to each other irrespective of the content they share and their physical locality. Every node has a maximum limit on the number of neighbors in order to build a distributed system. A joining node contacts a random node in the overlay and becomes its neighbor if the number of neighbors of the contacted node is less than the maximum threshold. A query in a random overlay is forwarded to a node s all known neighbors. The neighbors check to see whether they can reply to the query. If they find a match, they reply; otherwise they forward query to their neighbors and so on. In this way, for each query the whole overlay network is searched, which is clearly not scalable. One way to curtail the search process is to use time-to-live (TTL) controlled flooding mechanisms. Whenever a node forwards a query, it increments the hop count of the query. If the hop count exceeds the TTL field of the message, the node stops forwarding the query. This TTL value controls the query propagation in the system, but it may fail to find a query even if it exists in the system. Hence, the TTL field decreases the successful percentage of queries. 16

17 It should be noted that the node we talk about in this simulation section is overlay node or logical node since we mainly evaluate the PSON overlay performance, which is the key to the whole system design. B. Simulation Setup We implement PSON in NS [33], and conduct the simulation experiments on Redhat Linux platform. The physical network topologies are generated using the transit-stub approach in the GT-ITM models [28]. The simulation is initialized by having one node pre-exist in the network acting as a root of the insertion tree and then injecting node join operations into the network till the network reaches a certain size. Afterwards, nodes leave and search events are injected into the network. At this point only, we collect the metrics described below. Now we describe some important issues related to our simulation experiments: the semantic tree generation scheme, the query generation mechanism, and the performance metrics. 1) Semantic Tree Generation: A semantic tree hierarchy is an input to the PSON system because it is used to classify various files and queries. In the simulations, we generate random words of 5 letters each. A random word stands for the class of files (such as Music, Comedy ) a logical node would be responsible for, and it characterize the label of a semantic tree node. For the first random word, a random number of random words are generated as the semantic children of the first node. This procedure is repeated until the desired number of semantic tree nodes are generated. We also control the height of the semantic tree. In our simulations, we set the number of semantic tree nodes as 1000, and the maximum height of the tree as 20. In addition, the number of children for a tree node can also be controlled. We set this maximum number as 5 unless clarified otherwise. 2) Query Generation: To exhibit the real world s query behavior, overlay queries are generated in the following way. Every node in the system maintains a file containing all of the semantic entries with one semantic entry on each line. Whenever a node is prompted to issue a query, it generates a random number and reads the corresponding semantic entry from the file. The node then releases this semantic entry as the query into the overlay. 3) Performance Metrics: Although the main strength of PSON is to support complex queries while not compromising the search efficiency and scalability, we also try to explore some of the parameters that can affect the performance of PSON. We use the following metrics for our evaluation. Search Delay is the delay in searching a requested (class of) file(s). We measure this delay by the average number of logical hops traversed by a query from the source to the destination. Search Success Ratio is the percentage of successful searches that are able to locate existing files in the overlay. Insertion Delay is the delay in inserting a new node at its proper position in the overlay. Insertion delay is measured as the average number of logical hops traversed by an insertion message in the overlay. This does not include the extra overhead caused by rotations. Stabilization Overhead is the time it takes for the overlay network to stabilize itself after an insertion or deletion. After an insertion, it needs to do a fixup that may involve some rotations to maintain the balance 17

18 Fig. 7. Effect of overlay size on insertion performance of the overlay tree. Deletion itself incurs no delay but the overlay tree may still need to perform a fixup and restore its balance. This metric is measured by the total number of logical hops traversed by an insertion or deletion message and it includes the hops used in fixups and rotations. C. Results and Analysis 1) Insertion/Deletion Cost: The delay caused by node dynamics is measured, and the cost PSON paid to stabilize the overlay network after such events is also recorded. For simplicity, the node join and leave events are not interleaved. We will see the individual results for node insertion and deletion in the following. Node Insertion By this experiment, we measured how much time the overlay needs to add a new node at its proper position. Fig. 7 shows the number of logical hops traversed per insertion. The size of the tested physical network (i.e., the number of physical nodes in the network) is kept constant at Overlay nodes are inserted one by one into the system, and thus the overlay size (i.e., the number of overlay nodes in the network) is increasing. Note that this measure does not include the logical hops traversed in rotations that are carried out to balance the overlay after an insertion is finished. The logarithmic nature of the figure clearly shows the overlay construction is scalable to large overlay size. Insertion Overhead An insertion in the PSON overlay tree is followed by an insertion fixup mechanism that checks if any of the essential properties of a red-black tree has been violated. If so, the fixup mechanism restores that property. This fixup procedure may involve some rotations and hence cause some overhead in terms of the number of logical hops traversed per insertion. Fig. 8 shows the total number of logical hops traversed by an insertion message. The figure shows that as we increase the size of the overlay, the stabilization overhead also increases for some of the insertions. This is because an insertion fixup operation can cause more fixups if the tree is large. An insertion fixup actually works on a part of the tree, and when carried out on that part it may un-balance other parts and trigger another fixup involving some more rotations. This process is repeated until the whole overlay tree is 18

19 Fig. 8. Number of logical hops traversed per insertion including fixup Fig. 9. Average number of logical hops traversed per delete fixup balanced. However, this does not occur for every insertion because not every insertion requires rotations, and not every fixup triggers a chain of further fixups. Node Deletion Deletion itself does not cause any delay as we do not need to search the node to be deleted, and it can just leave the system on its own. However, every deletion operation is followed by a fixup mechanism that restores the red-black tree properties if violated. A deletion fixup may also perform some rotations to stabilize the overlay. We measure the number of logical hops traversed by a fixup after a deletion is performed. Fig. 9 depicts the result. Clearly, the curve in the deletion case is not logarithmic as the insertion case. For an insertion, a node s position in the overlay is searched first and then a fixup is performed. That s why the number of logical hops traversed by an insertion message is logarithmic. On the other hand, a node is deleted right away and the deletion fixup method is invoked. It is evident from the figure that the number of hops traversed by a deletion message decreases with the increasing number of deletions because of the decreasing overlay size. 2) Query Performance: In this section, we will evaluate the performance of PSON for both exact match queries and complex queries. 19

20 Average logical hops Number of overlay nodes Fig. 10. Effect of number of overlay nodes on average number of logical hops for 1000 exact queries Exact Query Performance First we present the performance of exact match queries in PSON. It is worth mentioning here that the search success ratio is 1 for every overlay query in PSON. Since overlay queries correspond to the nodes of the semantic hierarchy, an exact query is termed as successful if the corresponding node is found and unsuccessful if it is not found in the overlay (as opposed to complex query case explained below). If the logical node being searched is present then the PSON s search mechanism guarantees that it will be found in O(logn) hops. Fig. 10 shows the average number of logical hops traversed for different overlay network sizes when the number of queries and the number of physical nodes is kept constant at Fig. 10 shows that the query performance is nicely bound and hence scalable, which is consistent with our theoretical analysis that the overlay tree is always balanced. Complex Query Performance Now we show the performance of complex queries in PSON. A complex query in PSON can be a leaf as well as non-leaf node of the semantic tree. A query that corresponds to a leaf node is characterized as a leaf query and the one corresponding to a non-leaf node, as a non-leaf query. When a non-leaf query is released into the system and is found, the idea is to return the requested node as well as all the semantic descendants of the the node in the semantic hierarchy. For instance, if music and two of its semantic children, jazz and classical are present in the overlay then a query for music will return music, jazz and classical. This is because a query for music is a kind of complex query (because the query initiator didn t give all of the attributes like which kind of music file it is requesting), and since jazz and classical are also classified as music, they should be returned too. On the other hand, if a leaf query is found, just that node is returned. However, if the query node itself is not found, the system then looks for the first semantic parent that is present in the system, because intuitively that parent should have the requested file. For instance, if the file being requested for is of type jazz and it is not present in the overlay, the system should look for music, the semantic parent of jazz. The reason is, if the file exists in the system, it would be with music only, as there is no separate cluster for jazz music files in the overlay. 20