Peer- to- Peer File Sharing

Transcription

1 Server DistribuPng a Large File F bits - to- File Sharing d 4 Jennifer Rexford OS 461: omputer Networks Lectures: MW 10-10:50am in rchitecture N101 upload rate u s d 1 Internet d 2 d 3 hip:// download rates d i 2 Server DistribuPng a Large File Sending an F- bit file to N receivers TransmiSng NF bits at rate u s takes at least NF/u s Pme Receiving the data at the slowest receiver Slowest receiver has download rate d min = min i {d i } takes at least F/d min Pme Download Pme: max{nf/u s, F/d min } Speeding Up the File DistribuPon Increase the server upload rate Higher link bandwidth at the server MulPple servers, each with their own link lternapve: have the receivers help Receivers get a copy of the data and redistribute to other receivers To reduce the burden on the server 3 4 s Help DistribuPng a Large File F bits upload rate u s Internet d 1 d 3 u 1 u 2 u 3 d 2 d 4 u 4 s Help DistribuPng a Large File omponents of distribupon latency Server must send each bit: min Pme F/u s Slowest peer must receive each bit: min Pme F/d min Upload Pme using all upload resources Total number of bits: NF Total upload bandwidth u s + sum i (u i ) Total: max{f/u s, F/d min, NF/(u s +sum i (u i ))} upload rates u i 6 download 5 rates d i 1

2 - to- is Self- Scaling Download Pme grows slowly with N lient- server: max{nf/u s, F/d min } - to- peer: max{f/u s, F/d min, NF/(u s +sum i (u i ))} ut s may come and go s need to find each other s need to be willing to help each other LocaPng the Relevant s Three main approaches entral directory (Napster) Query flooding (Gnutella) Hierarchical overlay (Kazaa, modern Gnutella) Design goals Scalability Simplicity Robustness Plausible deniability to- Networks: Napster Napster history: the rise 1/99: Napster version 1.0 5/99: company founded 12/99: first lawsuits 2000: 80 million users Shawn Fanning, Northeastern freshman Napster history: the fall Mid 2001: out of business due to lawsuits Mid 2001: dozens of decentralized P2P alternapves 2003: growth of pay services like itunes Napster Directory Service lient contacts Napster (via TP) Provides a list of music files it will share and Napster s central server updates the directory lient searches on a Ptle or performer Napster idenpfies online clients with the file and provides their IP addresses lient requests the file from the chosen supplier Supplier transmits the file to the client oth client and supplier report status to Napster 9 10 Napster ProperPes Server s directory conpnually updated lways know what music is currently available Point of vulnerability for legal acpon - to- peer file transfer No load on the server Plausible deniability for legal acpon (but not enough) andwidth Suppliers ranked by apparent bandwidth and response Pme Napster: LimitaPons of Directory Single point of failure Performance boileneck opyright infringement File transfer is decentralized, but locapng content is highly centralized So, later P2P systems were more distributed Gnutella went to the other extreme

3 - to- Networks: Gnutella Gnutella: Search by Flooding 14 Gnutella history 2000: J. Frankel & T. Pepper released Gnutella Soon aier: many other clients (e.g., Morpheus, Limewire, earshare) 2001: protocol enhancements, e.g., ultrapeers Query flooding Join: contact a few nodes to become neighbors Publish: no need! Search: ask neighbors, who ask their neighbors Fetch: get file directly from another node xyz.mp3 search xyz.mp3? Flooding 13 Gnutella: Search by Flooding 15 Gnutella: Search by Flooding 16 xyz.mp3 xyz.mp3? transfer search Flooding Gnutella: Pros and ons dvantages Fully decentralized Search cost distributed Processing per node permits powerful search semanpcs Disadvantages Search scope may be quite large Search Pme may be quite long High overhead, and nodes come and go oien - to- Networks: Kaz KaZa history 2001: created by Dutch company (Kazaa V) Single network called FastTrack used by other clients as well Eventually protocol changed so others could no longer use it Super- node hierarchy Join: on start, the client contacts a super- node Publish: client sends list of files to its super- node Search: queries flooded among super- nodes Fetch: get file directly from one or more peers

4 KaZa: MoPvaPon for Super- Nodes Query consolidapon Many connected nodes may have only a few files PropagaPng query to a sub- node may take more Pme than for the super- node to answer itself Stability Super- node selecpon favors nodes with high up- Pme How long you ve been on is a good predictor of how long you ll be around in the future - to- Networks: ittorrent ittorrent history 2002:. ohen debuted ittorrent Emphasis on efficient fetching, not searching Distribute same file to many peers Single publisher, many downloaders PrevenPng free- loading IncenPves for peers to contribute ittorrent: Simultaneous Downloads Divide file into many chunks (e.g., 256 K) Replicate different chunks on different peers s can trade chunks with other peers can (hopefully) assemble the enpre file llows simultaneous downloading Retrieving different chunks from different peers nd uploading chunks to peers Important for very large files ittorrent: Infrastructure node Keeps track of peers parpcipapng in the torrent s registers with the tracker when it arrives selects peers for downloading Returns a random set of peer IP addresses So the new peer knows who to contact for data an have trackerless system Using distributed hash tables (DHTs) ittorrent: Overall rchitecture ittorrent: Overall rchitecture.torrent Get-announce

5 ittorrent: Overall rchitecture ittorrent: Overall rchitecture Response-peer list Shake-hand Shake-hand ittorrent: Overall rchitecture ittorrent: Overall rchitecture ittorrent: Overall rchitecture 29 Get-announce Response-peer list 30 ittorrent: hunk Request Order Which chunks to request? ould download in order Like an HTTP client does Problem: many peers have the early chunks s have liile to share with each other LimiPng the scalability of the system Problem: eventually nobody has rare chunks E.g., the chunks need the end of the file LimiPng the ability to complete a download SoluPons: random selecpon and rarest first 5

6 ittorrent: Rarest hunk First Which chunks to request first? The chunk with the fewest available copies I.e., the rarest chunk first enefits to the peer void starvapon when some peers depart enefits to the system void starvapon across all peers wanpng a file alance load by equalizing # of copies of chunks Free- Riding in P2P Networks Vast majority of users are free- riders Most share no files and answer no queries Others limit # of connecpons or upload speed few peers essenpally act as servers few individuals contribupng to the public good Making them hubs that basically act as a server ittorrent prevent free riding llow the fastest peers to download from you Occasionally let some free loaders download it- Torrent: PrevenPng Free- Riding has limited upload bandwidth nd must share it among mulpple peers Tit- for- tat: favor neighbors uploading at highest rate Rewarding the top four neighbors Measure download bit rates from each neighbor Reciprocate by sending to the top four peers OpPmisPc unchoking Randomly try a new neighbor every 30 seconds So new neighbor has a chance to be a beier partner 34 ittyrant: Gaming ittorrent ittorrent can be gamed, too uploads to top N peers at rate 1/N E.g., if N=4 and peers upload at 15, 12, 10, 9, 8, 3 peer uploading at rate 9 gets treated quite well est to be the N th peer in the list, rather than 1 st Offer just a bit more bandwidth than low- rate peers nd you ll spll be treated well by others ittyrant soiware Uploads at higher rates to higher- bandwidth peers hip://biiyrant.cs.washington.edu/ onclusions Finding the appropriate peers entralized directory (Napster) Query flooding (Gnutella) Super- nodes (KaZa) ittorrent Distributed download of large files np- free- riding techniques Great example of how change can happen so quickly in applicapon- level protocols 35 6