P2P networks, free riding, and counter-mechanisms

Transcription

1 P2P networks,, and counter-mechanisms İbrahim Körpeoğlu Department Computer Engineering Bilkent University Seminar Place: Department Electrical and Electronics Engineering Bilkent University, Ankara Date: October 19, Outline the talk Content sharing and distribution in Internet P2P networks and architectures Gnutella architecture and protocol Free and Gnutella Preventing in Gnutella İbrahim Körpeoğlu, Bilkent University 2

2 Outline Content sharing and distribution in Internet P2P networks and architectures Gnutella architecture and protocol Free and Gnutella Preventing in Gnutella İbrahim Körpeoğlu, Bilkent University 3 Internet and accessing content Internet is full interesting content (objects). HTML pages document files (.pdf,.doc,.ps, ) music files (mp3,.). video files (.mpg, ). images (.gif,.jpg). programs There are content provides and content consumers. There are various ways for content consumers to access and use the content. İbrahim Körpeoğlu, Bilkent University 4

3 Content providers and users Who are the content provides Companies Institutions Individuals Governments Who are the content consumers Individual users Companies Institutions. Groups.. content (object) content providers Internet content consumers/users İbrahim Körpeoğlu, Bilkent University 5 Ways to download content: pure client-server architecture Use client server architecture and associated protocols. Put content in servers. Clients can download the content via an Internet protocol FTP, or HTTP Servers can become bottleneck points. Reliability and performance problems. clients Internet FTP Get FTP Data server content HTTP request (GET) HTTP response İbrahim Körpeoğlu, Bilkent University 6

4 Ways to download content: web caching Install web caches (web proxies) in between clients and servers. Requests are first directed to the proxy. If a requested object is in the cache, it is satisfied from there, otherwise from the origin server. Better performance. client HTTP request HTTP reply HTTP request HTTP request Better reliability. client HTTP reply cache HTTP reply server İbrahim Körpeoğlu, Bilkent University 7 Web caching: where can we install the proxies Proxies can be installed on network boundaries A campus edge A department edge A country edge. We may have a hierarchy caches. Update caches is required when content changes. cache campus A backbone cache campus B origin server with the use caches, we can have better response time, better scalability, and better reliability. İbrahim Körpeoğlu, Bilkent University 8

5 Outline Content sharing and distribution in Internet P2P networks and architectures Gnutella architecture and protocol Free and Gnutella Preventing in Gnutella İbrahim Körpeoğlu, Bilkent University 9 Ways to download content: P2P file sharing Content is stored directly on edge devices: PCs. They are called peers. A peer can be both a client and server. Call it a SERVENT An important problem is how to locate the content? content peer P2P Network peer content peer content İbrahim Körpeoğlu, Bilkent University 10

6 P2P network features Decentralized architecture An application level network self-organizing network No central control Consists autonomous and anonymous peers Router Peer P2P Network Internet Backbone Logical links Physical links A peer may be connected to one or more other peers İbrahim Körpeoğlu, Bilkent University 11 Example use P2P services Peer X wants a file F. X invokes a search operation in the P2P network for file F. P2P network returns the location file F, i.e. the address a peer Y holding the file. Peer X downloads the file directly from peer Y. search F X Y = location F. download F P2P Network Internet Y file F İbrahim Körpeoğlu, Bilkent University 12

7 Closer look to downloads Peer X P2P Network Peer Y Object F P2P App HTTP HTTP request HTTP reply with file P2P App HTTP Object F TCP IP LL TCP connection Internet TCP IP LL transient web server İbrahim Körpeoğlu, Bilkent University 13 P2P network architecture and protocols How is a P2P network designed to perform two functions: 1) locate a desired file X. 2) download the desired file X. There are several P2P architectures that can realize these services. The scheme to locate the content usually directly affects the properties the architecture. P2P Network Architectures Centralized P2P networks Super-Peer based P2P networks Unstructured P2P networks DHT based P2P networks We will not cover in this talk! İbrahim Körpeoğlu, Bilkent University 14

8 Centralized architecture There is a central directory server which stores location information (index) for files shared. A peer that wants to download a file consults to this server for the location the file. Disadvantages: Single point failure Performance bottleneck Legal issues. peer index F F is in Y X X Where is F? central directory HTTP request File F Y HTTP response (the file X) İbrahim Körpeoğlu, Bilkent University 15 Centralized architecture: connection/disconnection to the network Disconnections When a peer disconnects from the network, this fact has to be detected by the central server. Periodic probing The server then deletes all location information for files stored in this peer. When a new peer connects to the network, it has to upload information about its shared files to the central server. central directory probe upload file info Peer X İbrahim Körpeoğlu, Bilkent University 16

9 Hierarchical overlay architecture (super-peer architecture) Location directory (index) is distributed. Peers are organized into groups Two types peers: Normal peers. Group leaders (super peers) A group leader keeps the location information for files in its peer group. A peer has connection to only to its group leader. peers group leader (super peer) peers İbrahim Körpeoğlu, Bilkent University 17 Outline the talk Content sharing and distribution in Internet P2P networks and architectures Gnutella architecture and protocol Free and Gnutella Preventing in Gnutella İbrahim Körpeoğlu, Bilkent University 18

10 Unstructured P2P architecture and Gnutella Totally decentralized architecture Flat topology Each peer has equal responsibility. A peer is both a server and client: servlet No directory maintained. Location is found by flooding a location query through the P2P network. servent servent servent servent Gnutella is a P2P network with this architecture. İbrahim Körpeoğlu, Bilkent University 19 Gnutella connection (i.e. Gnutella link) A peer is connected to the P2P network through one or more peers (neighbors). A connection to another peer is done using TCP sockets. Reliable delivery Gnutella protocol messages between peers. Gnutella connections go over TCP connections. peer P2P Network peer peer peer peer TCP connection TCP sockets İbrahim Körpeoğlu, Bilkent University 20

11 Gnutella stware and TCP connections A Gnutella servlet running on a machine is just an application level stware. Uses TCP to connect to other peers Implements HTTP protocol and uses that during downloads. Gnutella P2P network peer TCP IP LL TCP connection peer TCP connection Internet Infrastructure peer TCP IP LL İbrahim Körpeoğlu, Bilkent University 21 How does a peer initially know which peer(s) to connect to? Use host caches A peer consults to a host cache machine. A host cache is a special server that knows the IP addresses some active peers. A list IP addresses active peers can be downloaded from the host cache. TCP connection peer select and try an IP address Host cache list IP addresses TCP connection P2P Network İbrahim Körpeoğlu, Bilkent University 22

12 How does a peer learn more about other peers? Ping/Pong messages A peer can use its neighbors to learn about other peers. It sends Ping messages to its neighbors. Pong messages will be returned as replies A Pong message includes one IP address ( the peer or a peer). Peer X Peer Y knows the IP addresses its neighbors TCP connection TCP connections Peer A Peer B Peer Y TCP connection Peer C Ping Pong (A) Pong (B) Pong (C) A: B: C: IP addresses İbrahim Körpeoğlu, Bilkent University 23 How does a peer learn more about other peers? Ping messages Ping messages can be relayed through more than one hop. A peer receiving a Ping message may forward it to all its neighbors. If time-to-live (TTL) value Ping has reached to zero, no forwarding is done. Peer X Ping Ping Peer Y Ping Ping Peer A Peer B Peer C İbrahim Körpeoğlu, Bilkent University 24

13 How does a peer learn more about other peers? Pong messages Pong messages will follow the reverse path Ping messages. Pong message includes also a port number besides the IP address. The (port number, IP address) pair can be used to establish a TCP connection to the machine holding that IP address. Peer X Pong(A) Pong(B) Pong(C) Peer Y Pong(A) Pong(B) Pong(C) Peer A Peer B Peer C İbrahim Körpeoğlu, Bilkent University 25 Message Format Gnutellla 16 bytes 1 bytes 1 bytes 1 bytes 4 bytes Variable size Message ID Type TTL Hops Message Header Payload Length Each Gnutella message starts with a header. Payload is different for each message type: Ping message Pong message message Hit message Push message Payload İbrahim Körpeoğlu, Bilkent University 26

14 ing Message ID Type Length 0x80 TTL 0 X Min Speed (Search) Keywords A peer prepares a message and sends it to each its neighbors. Each neighbor forwards the to their neighbors. TTL limits the flooding. Peer Peer Peer Peer Peer Peer Peer İbrahim Körpeoğlu, Bilkent University 27 ing: vertical view Peer Peer Peer P2P network TCP IP LL TCP IP LL Internet TCP IP LL one or more physical hops Internet İbrahim Körpeoğlu, Bilkent University 28

15 message generation at a peer Generate ( Q) Create an empty message Q; Q.QID = select a unique 16 byte number ; Q.TTL = a predefied value, say 7; Q.Hops = 0; Q.Type = 0x80; Originator Q O Q X Q.Minspeed = minimum required speed an answering peer. Q.Keywords = search keywords received from the user. Create an entry for query Q in the query table. Send the query to all the neighbors. İbrahim Körpeoğlu, Bilkent University 29 message processing at a peer Process ( Q received from a peer P) If (Q.QID has been seen earlier) Drop the Q. Else Create an entry E in table queries E.QID = Q. QID; E.PreviousPeer = P; Local search for match. If (match found) Send back Hit Decrement Q.TTL increment Q.Hops If (Q.TTL == 0) Drop the Q. Else Send the Q to all neighbors excluding the query sender. peer X O Q X table ID Previous Peer. İbrahim Körpeoğlu, Bilkent University 30

16 Hit message processing at a peer match Process (Hit QH received from a peer P) If (the corresponding is generated at this node) display the results in the hit to user; Else Search for the table entry E matching the QH.QID Pv = E.PreviousPeer; E.NextPeer = P. Send the Hit to previous peer Pv. O Hit QH Hit QH X table Y ID Previous Peer NextPeer İbrahim Körpeoğlu, Bilkent University 31 Gnutella search in action B A D B G D J G I and J have the file. A Does search A C E B H D K H L K F C I E QUERY QUERY HIT A will get two replies. İbrahim Körpeoğlu, Bilkent University 32

17 Outline the talk Content sharing and distribution in Internet P2P networks and architectures Gnutella architecture and protocol Free and Gnutella Preventing in Gnutella İbrahim Körpeoğlu, Bilkent University 33 What is? Free means exploiting P2P network resources, but not contributing to the P2P network at desirable levels. A rider may do search and download operations, but: May not share any content May not share any interesting content. May not forward protocol messages, etc. peer rider consumes a lot P2P Network contributes little or none İbrahim Körpeoğlu, Bilkent University 34

18 Why peers ride? The peer may not want to share and contribute because: the peer has security concerns. the peers doesn t want to consume its network connection s bandwidth. the peer doesn t want to consume its local storage and processing resources. İbrahim Körpeoğlu, Bilkent University 35 Why is not desired? Good peers will be overloaded. Available content will decrease in time if only few people share. Fault tolerance will degrade If good peers get down, content will not be reachable. Search horizon will effectively shrink if too many riders exist in the network. query TTL=4 good peers P2P Network riders İbrahim Körpeoğlu, Bilkent University 36

19 Free in Gnutella* all gnutella peers content requested 30 % shares some files 37 % 63 % content never-requested riders sharing uninteresting content 70 % shares nothing riders sharing nothing (*) Source: Eytan Adar and Bernardo Huberman, Free Riding in Gnutella. FirstMonday, 5(10), October About only 10 % peers provide content that is searched for and downloaded. İbrahim Körpeoğlu, Bilkent University 37 Outline the talk Content sharing and distribution in Internet P2P networks and architectures Gnutella architecture and protocol Free and Gnutella Preventing in Gnutella A scheme to detect and discourage riders. İbrahim Körpeoğlu, Bilkent University 38

20 A distributed scheme against Designed for Gnutella Works in a distributed manner. No central control and management Based on monitoring protocol messages and updating counters. A proposal for: Detecting riders. Applying discouraging counter-actions against riders. İbrahim Körpeoğlu, Bilkent University 39 The approach B D G J Peer I will control its neighboring peers E, F, H, and K. A E H L C K F I The approach is based on having each peer control its direct neighbors. İbrahim Körpeoğlu, Bilkent University 40

21 The approach D G A B J Is I Free Rider? E H L E will control I and I will control E. C K F I Is E Free Rider? Neighboring peers will control each other. İbrahim Körpeoğlu, Bilkent University 41 Free Riding Types We classify behavior into three groups. A rider peer can behave as a: 1) Non-Contributor 2) Consumer 3) Dropper A rider may exhibit one or more these behaviors. İbrahim Körpeoğlu, Bilkent University 42

22 What is a non-contributor? Peer X P2P Network Hit Hit P2P Network It may forward others and Hits x No Hit Hit It never answers queries. It may generate Queries and can get Hits İbrahim Körpeoğlu, Bilkent University 43 What is a consumer Peer X P2P Network P2P Network Hit Hit It may forward others Queries and Hits Hit Peer consumes more that it contributes. Hit It answers Queries. It generates Queries and gets Hits İbrahim Körpeoğlu, Bilkent University 44

23 What is a dropper Peer X P2P Network P2P Network Hit x x Hit It does not forward other peers and/or Hits Hit It may answer Queries. It does not act as an intermediate P2P network node. Hit It may generate Queries and get Hits İbrahim Körpeoğlu, Bilkent University 45 How do we decide about and its type in the context Gnutella? Count the protocol messages received from and sent to neighboring peers. Update counters in tables maintained for each neighbor. N1 tables for neighbors controlling peer N2 N3 İbrahim Körpeoğlu, Bilkent University 46

24 Counters Monitoring peer Controlled peer Q RQ TQ Hit QH Hit Hit RQH Hit Hit Hit SQH Hit İbrahim Körpeoğlu, Bilkent University 47 Detecting a non-contributor Monitoring Peer X Controlled Peer Y Queries transmitted towards the neighbor Hit TQ QH Hit hits originated from the neighbor If (QH / TQ) < Threshold NONCONTRIBUTOR then Peer Y is Non-Contributor İbrahim Körpeoğlu, Bilkent University 48

25 Detecting a consumer Monitoring Peer X Controlled Peer Y Hit Hit QH hits generated by the neighbor SQH Hit Hit hits destined to the neighbor If (QH / SQH) < Threshold CONSUMER then Peer Y is Consumer İbrahim Körpeoğlu, Bilkent University 49 Detecting a dropper Monitoring Peer X Controlled Peer Y Queries received through the neighbor RQ TQ Queries sent towards the neighbor hits received through the neighbor Hit Hit RQH Hit If ( [RQ + RQH] / TQ) < Threshold DROPPER then Peer Y is Dropper İbrahim Körpeoğlu, Bilkent University 50

26 Actions We define three levels discouraging counteractions that can be applied against neighboring riders. Level 1: Decrement the TTL value in Queries Decrease the search horizon the rider Level 2: Ignore Queries partially Free rider will not be able to search through this neighbor Level 3 actions: Disconnect the peer. Free rider has to try to connect through another peer if it still wants to use the P2P network services İbrahim Körpeoğlu, Bilkent University 51 Level 1 action: decrement TTL rider Y (TTL = N) Action decrease TTL by M (TTL = N-M) M can be 1 or more. This will cause the horizon for the neighbor to shrink İbrahim Körpeoğlu, Bilkent University 52

27 Level 2 action: ignore requests rider Y Action: drop, or reduce probability forwarding İbrahim Körpeoğlu, Bilkent University 53 Level 3 action: disconnect peer x rider Y Action: disconnect İbrahim Körpeoğlu, Bilkent University 54

28 Exactly Exactly one one type type is is Normal No No types types is is Exactly Exactly two two types types are are Decr. TTL TTL Exactly Exactly two two types types are are Ignore Reqs. All All three three types types are are Exactly Exactly one one type type is is All All three three types types are are Disconnect All All three three types types are are İbrahim Körpeoğlu, Bilkent University 55 Summary and Future Work Great amount is in Gnutella network. Free has adverse affects on the proper operation P2P networks. We described a distributed and measurement based scheme to: Detect riders in a Gnutella network Take discouraging actions against riders in a Gnutella network. We are currently working on a simulation tool that is simulating on a Gnutella network model. The simulator will be used to evaluate the described scheme. İbrahim Körpeoğlu, Bilkent University 56