Pollution in P2P File Sharing Systems

Transcription

1 Pollution in P2P File Sharing Systems Keith W. Ross Leonard J. Shustek Professor of Computer Science Polytechnic University Brooklyn, N.Y. Joint work with: Jian Liang, Rakesh Kumar, Yongjian Xi

2 File sharing is big KaZaA: 3 million peers connected at same time Sharing 5,000 terabytes of content Impact on music industry: Music industry lost > $700 million in 2003 Each week > 1 billion songs downloaded Over 60 million Americans have downloaded music

3 Attempts to thwart file sharing Legal attacks Attack company: Napster Attack users: law suits Sabotage file sharing systems Pollution

4 Pollution Process original content polluted content pollution company

5 Pollution Process pollution server pollution company pollution server file sharing network pollution server pollution server

6 Unsuspecting users spread pollution! Pollution Process

7 Pollution Process Unsuspecting users spread pollution! Yuck

8 Classification of Pollution Content pollution: replacing parts with white noise cutting duration shuffling blocks of bytes inserting warnings and advertising Meta-data pollution: Taking old song and replacing metadata with that of currently popular song

9 Unintentional Pollution Users often accidentally create damaged files and insert in FS systems Inadvertently truncate a song while ripping Record song from radio and accidentally pick up disk jockey s voice Radio stations sometimes slightly tamper songs to reduce air time

10 Popular songs investigated

11 KaZaA Crawling Platform Farm of multi-threaded crawling nodes Crawls virtually all of the 20,000+ supernodes minutes Not easy: KaZaA is 100 times larger than Gnutella KaZaA is a proprietary protocol Messages are encrypted

12 KaZaA Overlay ON = ordinary node SN = super node SN ON ON ON Each SN maintains a local index

13 KaZaA Operation ON = ordinary node SN = super node SN ON ON ON Each SN maintains a local index

14 Overview of KaZaA: SN Refresh Lists Each SN maintains list of other known SNs Neighboring SNs share lists Each ON maintains cache of 200 SNs When ON connects, gets list of 200 SNs Some entries in cache replaced

15 KaZaA Crawling Platform

16 KaZaA Crawling Platform (2) Each node: 4 processes, each with 40 threads 1,600 simultaneous threads Each thread partially emulates the KaZaA connect and query protocol Crawling takes place in rounds of 30 seconds In stdy state, Process Manager dispatches 1,600 IPs to threads in each round thread operation 1. Thread given IP address and query string 2. Thread attempts to communicate with IP adr 3. Thread receives SN list, passed to process mgr 4. Sends each query string to SN 5. Rcves matching query results query result 6. Result frwded to measurement dbase

17 30-second round unvisited IPs visited IPs Process Manager SN lists 1,600 unvisited IPs + query string Measurement Database query results Threads Confirmed SNs

18 Each hour Cycles continue for one hour Each hour, start over Initial unvisited SN list = confirmed SN list of previous round Aggregate results in each hour 24 hours

19 Crawling Convergence (1) # of SNs = 20,000 30,000 morning afternoon

20 Crawling Convergence (2) Number of peers Number of SNs

21 Songs, versions, and copies If two files have same ContentHash, they belong to the same version. A given song can have thousands of versions For each version, there are many copies in the KaZaA network

22 Results for 7 test songs

23 # copies per version: CDF top 100 versions top 500 versions

24 # copies per version: PMF top 100 versions top 500 versions Zipf : f ( n) = K α n

25 Duration of versions Hey Ya Naughty Girl

26 Pollution detection Say a version is polluted if either: File is not fully decodable Length is outside -10%, +10% of official CD length No false positives: never declares version to be polluted when it isn t But possibility of false negatives: can declare version to be clean when it isn t

27 False Negatives 324 of 500 versions of Hey Ya declared clean Manually listened to all 324 versions 5 versions content polluted 18 meta-data polluted Remaining clean False negative: 7% Naughty Girl: 8% false negative Conclusion: pollution statistics are representative

28 Pollution Results top 100 versions top 500 versions For each of 7 songs, of top 500 versions are polluted!

29 Older songs

30 KaZaA Rating Excellent, average, poor, delete KaZaA rating ineffective in identifying polluted files Pollution companies likely giving polluted files high ratings

31 Anti-Pollution Mechanisms Detection without downloading: From query results, without downloaing, attempt to determine the files in results that are polluted Example, KaZaA ratings Detection with downloading Download all or portions of file to automatically determine if file is polluted Example: simple method used in this paper

32 Detection with File Downloading Matching: trusted database with fingerprints of authentic content Hash, frequency representation of song, time-domain summary of song Client matches what it has downloaded with fingerprint in database Incentives to encourage users to manually check songs before putting them in shared folder: sieving

33 Detection without Downloading Rigid trust: group of friends agree to only insert clean versions and download only from each other Web of trust: download from friends of friends Reputation systems: Lets peers rate each other a lot of research going on here

34 Conclusion Popular contemporary songs can have over 50,000 versions and 1 million copies More than 50% of the copies of some popular songs are polluted older songs are much less polluted KaZaA s rating system is largely ineffective Reviewed several antipollution mechanisms KaZaA crawling platform reverse engineered KaZaA distributed deployment unnecessary crawls all SNs in less than 1 hour Also useful for other studies evolution of volumes of content & pollution metadata indexes