Databases with Autonomous Nodes Where P2P systems meet database techniques

Transcription

1 Databases with Autonomous Nodes Where P2P systems meet database techniques Austin Parker University of Maryland Databases with Autonomous Nodes p.1/26

2 What is P2P? Many (relatively) equal peers, all participating in the network of their own volition. They have the ability to leave at any time. Databases with Autonomous Nodes p.2/26

3 What is P2P? Many (relatively) equal peers, all participating in the network of their own volition. They have the ability to leave at any time. Direct connectivity (no central server). Databases with Autonomous Nodes p.2/26

4 What is P2P? Many (relatively) equal peers, all participating in the network of their own volition. They have the ability to leave at any time. Direct connectivity (no central server). Existing P2P programs: Kazaa, BitTorrent, FreeNet, GNUnet, NICE, P5, etc... Databases with Autonomous Nodes p.2/26

5 P2P Systems and Databases Problem: Want large P2P systems for data storage and quick lookup / search of same. Databases with Autonomous Nodes p.3/26

6 P2P Systems and Databases Problem: Want large P2P systems for data storage and quick lookup / search of same. Solution: Existing research in databases allows for ease of storage and quick lookup / search. Databases with Autonomous Nodes p.3/26

7 P2P Systems and Databases Problem: Want large P2P systems for data storage and quick lookup / search of same. Solution: Existing research in databases allows for ease of storage and quick lookup / search. Conflict: Many standard database assumptions are no longer practical (node perpetuity, constant time pointer access, and the importance of load balancing). Databases with Autonomous Nodes p.3/26

8 Research Use existing database solutions under the significantly different P2P philosophy. Has both practical and theoretical interest. New Issues: non-uniformity of data and query frequency, efficient use of network, node fallibility. I.E. CHORD, CAN, SCRAP, P2P R-tree, SWAM-V, and others. Databases with Autonomous Nodes p.4/26

9 Load Balancing Peers join the system voluntarily. Peers will leave if they are treated unfairly: they cannot be asked to do too much more than their share. Thus: any good P2P system needs some sort of load balancing: cannot have overloaded nodes. There are two kinds: storage load and query load. Databases with Autonomous Nodes p.5/26

10 Efficient Use of Network In standard database techniques, links between nodes are pointers to memory (disk or RAM). In P2P systems, links between peers travel over the internet and suffer from various amounts of latency. Thus: in P2P systems, a means less. path length Databases with Autonomous Nodes p.6/26

11 Node Perpetuity Peers are allowed to simply leave the network, without telling anyone (for instance, if the machine crashes). P2P systems should not be breakable by such falures. This is generally solved by adding redundancy to the system (each peer keeps the state of several of its neighbors). Databases with Autonomous Nodes p.7/26

12 CHORD (Stoica, Morris, Darger, Kaashoek, Balakrshnan) A distributed hash using a skiplist. Example. Each peer gets an ID (generally a hash of the node s IP address). Each data item gets an ID (generally a hash of the data s contents). Nodes form a skiplist sorted on their IDs. Data is stored in the peer with the closest ID. Routing / Search algorithm is the same as skiplist traversal. Databases with Autonomous Nodes p.8/26

13 SCRAP (Ganesan, Yang, Garcia-Molina) Modifies CHORD to allow range queries. Instead of each data tuple getting a random ID, it is assigned an ID using a space-filling curve (i.e. z-ordering). To do a range query, one checks each node in the ring from the lowest ID in the range to the highest ID in the range. Databases with Autonomous Nodes p.9/26

14 SCRAP Example Here we have a 4 4 space ordered by row ordering. Red numbers represent IDs of peers in the system. Data items in [2,2] have ID 6 and are stored at the nearest peer with ID less than 6 (in this case, node with ID 5). Databases with Autonomous Nodes p.10/26

15 SCRAP Example To search in range [1,1] to [1,4], we must query all peers with IDs between 1 and 13 (13 queries to cover a 4 square region). Inefficient! (for any spacefilling curve, such situations arise). Databases with Autonomous Nodes p.10/26

16 Theoretics I CHORD and SCRAP (with Lookup time: peers). state kept by each peer. times the latency. SCRAP allows for nearest neighbor searches, CHORD doesn t but gives load balancing qualities. No latency optimizations. Databases with Autonomous Nodes p.11/26

17 CAN (Ratnasamy, Francis, Handley, Karp, Shenker) Generally compared to a K-D tree (though much different). Each peer gets a d-dimensional ID. Neighboring peers know of one another (in dimensions, a neighbor is one who shares an edge of dimension considered to be a torus). also, space is Each peer is responsible for data near their ID. Databases with Autonomous Nodes p.12/26

18 CAN Example Index: Graph: Databases with Autonomous Nodes p.13/26

19 CAN Example Pink Join: New Graph: Databases with Autonomous Nodes p.13/26

20 Relation to K-D tree Similarities: Partitions space similarly. Has similar search and insert algorithms. Differences: Nodes associated with regions, not points. All nodes can contain data no tree structure on top of index. Databases with Autonomous Nodes p.14/26

21 CAN CAN Optimizations: Caching: Nodes remember recently answered queries for a period of time. Replication: Overloaded nodes replicate data on their neighbors, who answer the queries in proxy some percentage of the time. Zone Overloading (binning): Allow up to MAXPEERS nodes in each region before dividing. Databases with Autonomous Nodes p.15/26

22 Theoretics II CAN (with dimensional hash ftn and peers). state kept by each peer. Lookup time: times the latency. Doesn t allow for nearest neighbor searches. Note: CAN s optimizations can significantly reduce latency and improve average peer query load. Databases with Autonomous Nodes p.16/26

23 MURK (Ganesan, Yang, Garcia-Molina) MURK is a variant of CAN. Differences: Data is no longer hashed: Grid s dimension depends on data s dimension. Add a constant number of skip pointers at each peer. Handles range queries, doesn t handle nonuniform data. Databases with Autonomous Nodes p.17/26

24 Theoretics III MURK (with -dimensional data, random links, and peers) Lookup time: state kept by each peer. times the latency. Allows for nearest neighbor searches. Databases with Autonomous Nodes p.18/26

25 P2PR-tree (Mondal, Lifu, Kitsuregawa) This is an attempt to harness the pruning power of an rtree in P2P. Divide the space statically into blocks, divide each block statically into groups. Inside each group, divide into subgroups using MBRs. Subgroups can be further divided into groups a la RTree techniques. Databases with Autonomous Nodes p.19/26

26 P2PR-tree CON T It seems that each peer has pointers to At least one peer from each block. One peer for groups in the same block. One peer for subgroups in the same group. All peers in the same subgroup. Each query is given counter, whose value determines the level to which the query should be forwarded (if 0, it goes to a block, if 1, a group, etc). Databases with Autonomous Nodes p.20/26

27 Theoretics IV P2PR-tree (with peers) state per peer (though it is unclear from the paper exactly what this would be). Lookup time: times the latency (though average time lookup will be significantly smaller). Allows for nearest neighbor searches. Databases with Autonomous Nodes p.21/26

28 SWAM (Banaei-Kashani, Shahabi) Small World Access Methods A family of P2P data storage methods satisfying: Monotonic approach towards key (always possible to forward query to a closer peer). Localized index topology (similar data objects are likely to be connected). Logarithmic forwarding-path length. Databases with Autonomous Nodes p.22/26

29 SWAM CON T Each data item has its own node (one p2p peer can function as multiple nodes in the mesh). SWAMs combine a grid and a random graph. Taken from CKIM, nov 2004, p308. Databases with Autonomous Nodes p.23/26

30 SWAM-V A proof-of-concept for SWAM. Structure based off of Voronoi diagrams. Each peer links to adjoining neighbors and one random node whose distance from the peer is large (Delaunay graph plus a 1-random graph). Databases with Autonomous Nodes p.24/26

31 Theoretics IV SWAM-V (with data items). State per peer: 6 links to other peers per stored data item (on average). Lookup time: average ( times the latency on in the worst case). Allows nearest neighbor searches. Databases with Autonomous Nodes p.25/26

32 Conclusion P2P data storage techniques harness database techniques. P2P data storage algorithms require different sets of assumptions. The solutions are interesting and usable. This a wonderful oppurtunity to Database people to learn some networking and for some networking people to learn databases. Databases with Autonomous Nodes p.26/26