For information on all Morgan Kaufmann publications, visit our Web site at or

Size: px
Start display at page:

Download "For information on all Morgan Kaufmann publications, visit our Web site at or"



2 Morga Kaufma Publishers is a imprit of Elsevier. 30 Corporate Drive, Suite 400, Burligto, MA 01803, USA This book is prited o acid-free paper. # 2009 by Elsevier Ic. All rights reserved. Desigatios used by compaies to distiguish their products are ofte claimed as trademarks or registered trademarks. I all istaces i which Morga Kaufma Publishers is aware of a claim, the product ames appear i iitial capital or all capital letters. All trademarks that appear or are otherwise referred to i this work belog to their respective owers. Neither Morga Kaufma Publishers or the authors ad other cotributors of this work have ay relatioship or affiliatio with such trademark owers or do such trademark owers cofirm, edorse or approve the cotets of this work. Readers, however, should cotact the appropriate compaies for more iformatio regardig trademarks ad ay related registratios. No part of this publicatio may be reproduced, stored i a retrieval system, or trasmitted i ay form or by ay meas electroic, mechaical, photocopyig, scaig, or otherwise without prior writte permissio of the publisher. Permissios may be sought directly from Elsevier s Sciece & Techology Rights Departmet i Oxford, UK: phoe: (þ44) , fax: (þ44) , You may also complete your request olie via the Elsevier homepage (, by selectig Support & Cotact the Copyright ad Permissio ad the Obtaiig Permissios. Library of Cogress Catalogig-i-Publicatio Data Applicatio Submitted ISBN: For iformatio o all Morga Kaufma publicatios, visit our Web site at or Prited i the Uited States of America

3 To my wife Gia ad our daughter Jacquelie JFB To my lovely daughters Agie ad Kiki HY To our adorable so Athoy EKL

4 Preface Ratioale Peer-to-peer etworkig has emerged as a viable busiess model ad systems architecture for Iteret-scale applicatios. Although its techological roots trace back through several decades of desigig distributed iformatio systems, cotemporary applicatios demostrate that it is a effective way to build applicatios that coect millios of users across the globe without reliace o specially deployed servers. Istead, by combiig the resources of each user s computer, these systems automatically self-orgaize ad adapt to chagig peer populatios while providig services for cotet sharig ad persoal commuicatios. Public attetio to peer-to-peer applicatios came first from highly popular file-sharig systems, i which decetralizatio was used to support a busiess model that eeded to legitimize licesed cotet sharig. The subsequet success of the Skype Iteret telephoy applicatio showed the geerality of the peer-to-peer approach ad its feasibility to provide acceptable service quality to millios of users. Subsequetly there has bee growig iterest i improvig o these systems as well as cosiderig ew desigs to attai better performace, security, ad flexibility. Today it is aticipated that peer-to-peer techologies will become geeral-purpose, widely used vehicles for buildig a broad rage of applicatios for social etworkig, iformatio delivery, ad persoal commuicatios applicatios i the future. There are may importat questios about the evolutio of peer-to-peer techologies. What ew applicatios will drive this evolutio? Will P2P be used as a geeral-purpose techique for buildig ay distributed applicatio? How do treds i wireless etworkig, cosumer electroics, home etworkig, high-defiitio cotet, digital rights maagemet, ad so forth itersect with peer-to-peer? Is P2P a paacea for desigig large-scale applicatios, or if ot, what are the characteristics of applicatios for which it is well suited? How should other architectures coexist with ad adapt to peer-to-peer desig? Will the P2P ladscape be Balkaized by may icompatible peer-to-peer protocols ad systems? The topics covered i this book provide a comprehesive survey of both the practice of P2P ad mai research directios ad are iteded to frame the aswers for these questios. Orgaizatio ad Approach The first two chapters itroduce the mai cocepts of peer-to-peer systems. We examie the operatio of a basic P2P system, icludig behavior for self-orgaizig, routig, ad searchig. We also describe a umber of represetative commercial applicatios. The ext four chapters describe the fudametal peer-to-peer overlay architectures, icludig both ustructured ad structured overlays. xvii

5 xviii Preface The last chapter i this group covers importat implemetatio issues such as protocol desig, NAT traversal, ad peer capability assessmet. Detailed discussio of P2P mechaisms to support key applicatios follow, icludig chapters o search, cotet delivery, peercastig ad overlay multicastig, ad overlay-based Iteret telephoy. Importat uses of peer-to-peer overlays that we describe here iclude differet techiques for cotet search, delivery of real-time streamig cotet, ad sessio iitiatio usig the overlay. We the discuss i separate chapters how requiremets for overlay performace, peer mobility, security, ad maagemet itersect with the P2P overlay desig. Throughout the book, to motivate ad illustrate the material, we iclude examples of systems i use ad describe importat research prototypes. We also refer to ope-source implemetatios for readers who seek a hads-o illustratio of the ideas. I particular we use examples from OverlayWeaver, a ope-source toolkit developed by Kazuyuki Shudo that supports a umber of importat peerto-peer algorithms. Access to the ope-source tools ad updates to the book ca be obtaied via the compaio Website at Audiece This book is iteded for professioals, researchers, ad computer sciece ad egieerig studets at the advaced udergraduate level ad higher who are familiar with etworkig ad etwork protocol cocepts ad basic ideas about algorithms. For the more advaced parts of the book, the reader should have geeral familiarity with Iteret protocols such as TCP ad IP routig but should ot eed to kow the details of etwork routig protocols such as BGP or OSPF. For some sectios of the book such as discussios of mobility or multicastig, familiarity with mobility i IP ad IP multicastig will be helpful but ot required. The reader will also fid it helpful to be familiar with otatio for comparig algorithm performace, such as O() or O(log ). For istructors who wat to use the book as a textbook i a class o peer-topeer etworkig, a set of exercises for each chapter, with a aswer key for selected exercises are available by registerig at Peer-to-peer etworkig is geerally see as a ew techology with a disruptive busiess model ad may possibilities for further iovatio. These treds make the subject matter i this book highly relevat to the techology commuity. We hope the book is a valuable startig poit for readers who are ew to the subject ad a importat referece to those who are active i the field. Throughout the book we coclude each chapter with suggestios for further readig for readers who would like to dig deeper ito specific topics. Ackowledgemets Durig the preparatio of this book, may people provided help i reviewig portios of the text ad the origial book proposal. We greatly appreciate their suggestios ad efforts i improvig the quality of the book.

6 Preface xix First, we would like to thak those idividuals who reviewed the origial proposal ad made importat commets about structure, topics, ad emphasis: Germao Caroi, Google; Christos Gkatsidis, Microsoft Research; Wolfgag Kellerer, DOCOMO Euro-Labs; Xuemi (Sherma) She, Uiversity of Waterloo; ad Xiaotao Wu, Avaya Labs Research. I additio, Wolfgag Kellerer, DOCOMO Euro-Labs, reviewed a substatial portio of the book ad provided may useful suggestios. We are also grateful to those who reviewed ad commeted o portios of the book: Yi Cui, Vaderbilt Uiversity; Awitama Datta, Nayag Techological Uiversity; Aaro Harwood, Uiversity of Melboure; Vaa Kalogeraki, Uiversity of Califoria Riverside; Mario Kolberg, Uiversity of Stirlig; Be Leog, Natioal Uiversity of Sigapore; Li Li, Commuicatios Research Cetre Caada; Ludy Lewis, Souther New Hampshire Uiversity; Muthucumaru Maheswara, McGill Uiversity; Kurt Tutschku, Uiversity of Viea; Megku Yag, Easter Ketucky Uiversity; Weju (Kevi) Zeg, Uiversity of Missouri-Columbia. The efforts ad costructive commets of all the reviewers are greatly appreciated. Ay mistakes that remai are the resposibility of the authors. Thaks are due to the staff at Morga Kaufma--Rick Adams, Seior Acquisitios Editor; assistat editors Gregory Chalso, Maria Aloso, ad Lidsey Gedall; ad our project maager, Melida Ritchie. Fially we thak our families for their support ad uderstadig while we worked o this book. Joh F. Buford, Priceto, NJ Heather Yu, Priceto, NJ Eg Keog Lua, Japa ad USA

7 About the Authors Joh F. Buford is Research Scietist, Avaya Labs Research, Baskig Ridge, NJ. Previously he was Lead Scietist at Paasoic Techologies, VP of Software Developmet at Kada Systems, Director of Iteret Techologies at Verizo, ad Assoc. Prof. of Computer Sciece at Uiversity of Massachusetts Lowell. PhD i Computer Sciece, Graz Uiversity of Techology. Heather Yu is Seior Maager of Media Techologies at Huawei Techologies USA, Bridgewater, NJ where she leads the research o multimedia cotet etworkig ad digital media techologies. PhD i Electrical Egieerig, Priceto Uiversity. Eg Keog Lua is Faculty Member of College of Egieerig, Iformatio Networkig Istitute at Caregie Mello Uiversity, Pittsburgh, USA, ad Systems Scietist, Caregie Mello CyLab USA ad Japa. Previously he held research fellowship ad idustry cosultig positios at the NTT Laboratories, Itel Research, Microsoft Research, ad Hewlett-Packard R&D/Cosultig. His research areas iclude Peer-to-Peer etworks, Iteret-scale P2P overlay multimedia commuicatios, etwork security ad future Iteret. PhD i Computer Sciece, Uiversity of Cambridge, UK. xxi

8 CHAPTER Itroductio 1 Our discussio of peer-to-peer (P2P) cocepts starts with a overview of the key applicatios ad their emergece as maistream services for millios of users. The chapter the examies the relatioship of P2P with the Iteret ad its distictive features compared to other service architectures. A review of P2P ecoomics, busiess models, social impact, ad related techology treds cocludes the chapter. P2P EMERGES AS A MAINSTREAM APPLICATION The Rise of P2P File-Sharig Applicatios Nearly 10 years after the World Wide Web became available for use o the Iteret, decetralized peer-to-peer file-sharig applicatios supplated the serverbased Napster applicatio, which had popularized the cocept of file sharig. Napster s cetralized directories were its Achilles heel because, as it was argued i court, Napster had the meas, through its servers, to detect ad prevet registratio of copyrighted cotet i its service, but it failed to do so. Napster was subsequetly foud liable for copyright ifrigemet, dealig a lethal blow to its busiess model. As Napster was cosumed i legal challeges, secod-geeratio protocols such as Gutella, FastTrack, ad BitTorret adopted a peer-to-peer architecture i which there is o cetral directory ad all file searches ad trasfers are distributed amog the correspodig peers. Other systems such as FreeNet also icorporated mechaisms for cliet aoymity, icludig routig requests idirectly through other cliets ad ecryptig messages betwee peers. Meawhile, the top labels i the music idustry, which have had arguably the most serious reveue loss due to the emergece of file sharig, have cotiued to pursue legal challeges to these systems ad their users. Regardless of the outcome of these court cases, the social perceptio of the acceptability ad beefits of cotet distributio through P2P applicatios has 1

9 2 CHAPTER 1 Itroductio bee irrevocably altered. I the music idustry prior to P2P file sharig, audio CDs were the domiat distributio mechaism. Web portals for olie music were limited i terms of the size of their catalogs, ad dowloads were expesive. Although P2P file sharig became widely equated with cotet piracy, it also showed that cosumers were ready to replace the CD distributio model with a olie experiece if it could provide a large portfolio of titles ad artists ad if it icluded features such as a search, previews, trasfer to CD ad persoal music players, ad idividual track purchase. As portals such as itues emerged with these properties, a tremedous growth i the olie music busiess resulted. I a typical P2P file-sharig applicatio, a user has digital media files he or she wats to share with others. These files are registered by the user usig the local applicatio accordig to properties such as title, artist, date, ad format. Later, other users aywhere o the Iteret ca search for these media files by providig a query i terms of some combiatio of the same attributes. As we discuss i detail i later chapters, the query is set to other olie peers i the etwork. A peer that has local media files matchig the query will retur iformatio o how to retrieve the files. It may also forward the query to other peers. Users may receive multiple successful resposes to their query ad ca the select the files they wat to retrieve. The files are the dowloaded from the remote peer to the local machie. Examples of file-sharig cliet user iterfaces are show i Figures 1.1 ad 1.2. FIGURE 1.1 LimeWire cliet.

10 P2P Emerges as a Maistream Applicatio 3 FIGURE 1.2 emule cliet search iterface. Despite their popularity, P2P file-sharig systems have bee plagued by several problems for users. First, some of the providers of leadig P2P applicatios ear reveue from third parties by embeddig spyware ad malware ito the applicatios. Users the fid their computers ifected with such software immediately after istallig the P2P applicatio. Secod, a large amout of polluted or corrupted cotet has bee published i file-sharig systems, ad it is difficult for a user to distiguish such cotet from the origial digital cotet they seek. It is geerally felt that pollutio attacks o file-sharig systems are iteded to discourage the distributio of copyrighted material. A user dowloadig a polluted music file might fid, for example, oise, gaps, ad abbreviated cotet. A third type of problem affectig the usability of P2P file-sharig applicatios is the free-rider problem. A free rider is a peer that uses the file-sharig applicatio to access cotet from others but does ot cotribute cotet to the same degree to the commuity of peers. Various techiques for addressig the freerider problem by offerig icetives or moitorig use are discussed later i the book. A related issue is that of peer chur. A peer s cotet ca oly be accessed by other peers if that peer is olie. Whe a peer goes offlie, it takes time for other peers to be alerted to the chage i status. Meawhile, cotet queries may go uaswered ad time out.

11 4 CHAPTER 1 Itroductio The leadig P2P file-sharig systems have ot adopted mechaisms to protect licesed cotet or collect paymet for trasfers o behalf of copyright owers. Several vetures seek to legitimize P2P file sharig for licesed cotet by icorporatig techiques for digital rights maagemet (DRM) ad superdistributio ito P2P distributio architectures. I such systems, cotet is ecrypted, ad though it ca be freely distributed, a user must separately purchase a ecrypted licese file to reder the media. Through the use of digital sigatures, such licese files are ot easily trasferred to other users. See this book s Website for liks to curret P2P file-sharig proposals for DRM-based approaches. Other vetures such as QTrax, SpiralFrog, ad TurItUp are proposig a adbased model for free music distributio. The user ca freely dowload the music file, which i some models is protected with DRM, but must liste to or watch a ad durig dowload or playback. I these schemes, the advertiser istead of the user is payig the cotet licesig costs. Questios remai about this model, such as whether it will udercut existig music dowload busiess models ad whether the advertisig reveue is sufficiet to match the licesig reveue from existig music dowload sites. Voice over P2P (VoP2P) Desktop VoIP (voice over IP) cliets bega to appear i the mid-1990s ad offered free desktop-to-desktop voice ad video calls. These applicatios, though ecoomically attractive ad techically iovative, did t attract a large followig due to factors such as lack of voice quality ad limited availability of broadbad access i the cosumer market. I additio, the iitially small size of the etwork commuity limited the potetial of such applicatios to supplat covetioal telephoy. This cotiues to be a practical issue facig ew types of P2P applicatios how to create a commuity of users that ca reach the critical mass eeded to provide the value propositio that comes with scale. Startig i 1996 with the lauch of ICQ, a umber of istat-message ad presece (IMP) applicatios became widely popular. The leadig IMP systems, such as AIM, Microsoft Messeger, Yahoo! Messeger, ad Jabber, all use cliet/server architectures. 6 Although several of these systems have subsequetly icluded telephoy capabilities, their telephoy features have ot draw a large user commuity. Skype is a VoP2P cliet lauched i 2003 that has reached more tha 10 millio cocurret users. The VoP2P techology of Skype is discussed i Chapter 11. Compared to earlier VoIP cliets, Skype offers both free desktop-to-desktop calls ad low-cost desktop-to-public switched telephoe etwork (PSTN) calls, icludig iteratioal calls. The call quality is high, geerally attributed to the audio codec Skype uses ad today s wide use of broadbad access etworks to reach the Iteret. I additio, Skype icludes features from IMP applicatios, icludig buddy lists, istat messagig, ad presece. Ulike the file-sharig systems, Skype promises a o spyware policy.

12 P2P Emerges as a Maistream Applicatio 5 FIGURE 1.3 Skype cliet. The Skype user iterface is show i Figure 1.3. It icludes a buddy list that shows other buddies ad their olie status. The user ca select buddies to iitiate free chat, voice, ad group coferece sessios. The user ca also eter PSTN umbers to call, ad these calls are charged. P2PTV The success of P2P file sharig ad VoP2P motivated use of P2P for streamig video applicatios. P2PTV delivery ofte follows a chael orgaizatio i which cotet is orgaized ad accessed accordig to a directory of programs ad movies. Ulike file-sharig systems i which a media file is first dowloaded to the user s computer ad the played locally, video-streamig applicatios must provide a real-time stream trasfer rate to each peer that equals the video playback rate. Thus if a media stream is ecoded at 1.5 Mbps ad there is a siglepeeractigasthesourceforthestream,thepathfromthesourcepeerto the playback peer must provide a data trasfer rate of 1.5 Mbps o average. Somevariatioitheplaybackratealog the path ca be accommodated by prebufferig a sufficiet umber of video frames. The if the trasfer rate temporarily drops, the extra cotet i the buffer is used to prevet dropouts at the rederig side.

13 6 CHAPTER 1 Itroductio A attractive feature of peer-to-peer architectures for delivery of video streams is their self-scalig property. Each additioal peer added to the P2P system adds additioal capacity to the overall resources. Eve powerful server farms are limited to the maximum umber of simultaeous video streams that they ca deliver. I a P2P etwork, ay peer receivig a video stream ca also forward it to a few other peers. If D > 1 peers are directly coected to the source peer ad each peer ca i tur support D peers, the up to D 2 þ D peers ca receive a video stream withi two hops from the source. Likewise, if each of the secod-tier D peers ca i tur support D peers, up to D 3 þ D 2 þ D peers ca receive the video stream i three hops. Note that each hop adds a small forwardig delay, which is usually ot a problem i oe-way video-streamig applicatios. This simple model is good if all peers watchig the specific video stream are viewig the same positio i the stream close to simultaeously, equivalet to a broadcast televisio chael. However, i video-o-demad type applicatios, peers start viewig a stream at arbitrary times, ad those peers that start viewig a stream cocurretly may soo diverge i stream positio due to user actios such as pause or rewid. To avoid trasferrig complete copies of video files to peers at playback, a method is eeded for a peer to locate the ext chuk of video i its playback schedule from some other peers i the P2P system. Oe techique used i BitTorret 243 is for the source of the cotet to seed other peers with chuks of the cotet. These peers the access the torret created by the source to idetify each other ad retrieve the cotet chuks directly from other peers. Such a group of peers exchagig chuks is called a swarm. As is the case with other P2P applicatios, the volatility of peers could cause gaps i stream playback if the peer that is the source of the ext segmet of video suddely leaves the P2P system. Sice peers are user-cotrolled ed systems, upredictable ad uaouced departures are a assumed hazard. Most P2P etworks have specific protocols to recogize such departures ad to cotiually locate ew peers that are joiig the P2P system. For video-streamig applicatios, mitigatig the departure of a source peer ca ivolve periodically searchig for redudat sources (which are also volatile) ad providig a sufficiet buffer to reduce the impact o the user s viewig experiece whe a source peer departure does occur. Outside of P2P video applicatios, a great deal of research has dealt with the issues of reliable etwork delivery of real-time video. Due to etwork cogestio ad etwork failures, packets may sometimes get dropped. P2P etworks deped o the uderlyig physical etwork. Cosequetly, techiques already developed for reliable delivery of streams i packet etworks are applicable i P2P overlay etworks. Such techiques iclude adaptive video delivery, multiresolutio video, ad scalable video ad are discussed i Chapter 8. Several P2PTV applicatios are available. Examples iclude Babelgum, Joost, PPLive, PPStream, SopCast, TVats, TVUPlayer, Veoh TV, ad Zattoo (see Figure 1.4).

14 P2P Emerges as a Maistream Applicatio 7 FIGURE 1.4 P2PTV applicatios, (A) Bablegum # 2008 Bablegum. Reprited by permissio, (B) Zattoo # 2007 Zattoo. Reprited by permissio, (C) TVU Networks # 2008 TVU Networks. Reprited by permissio.

15 8 CHAPTER 1 Itroductio P2P NETWORKING AND THE INTERNET P2P Overlays ad Network Services Peers i P2P applicatios commuicate with other peers usig messages trasmitted over the Iteret or other types of etworks. The protocol for a P2P applicatio is the set of differet message types ad their sematics, which are uderstood by all peers. The protocols of various P2P applicatios have some commo features. First, these protocols are costructed at the applicatio layer of the etwork protocol stack. Secod, i most desigs peers have a uique idetifier, which is the peer ID or peer address. Third, may of the message types defied i various P2P protocols are similar. Fially, the protocol supports some type of message-routig capability. That is, a message iteded for oe peer ca be trasmitted via itermediate peers to reach the destiatio peer. To distiguish the operatio of the P2P protocol at the applicatio layer from the behavior of the uderlyig physical etwork, the collectio of peer coectios i a P2P etwork is called a P2P overlay. Figure 1.5 shows the correspodece betwee peers coectig i a overlay etwork with the correspodig hosts, devices, ad routers i the uderlyig physical etwork. Later i this book we discuss importat properties ad details of P2P overlays. For cosistecy, whe we wat to talk about a system of peers usig a commo P2P applicatio layer protocol, we will refer to it as a P2P overlay or simply a overlay. It might be coveiet to thik of a P2P system or P2P etwork as syoyms for P2P overlay. The practice of overlay etworks predates the P2P applicatio era. For example, protocols used i Iteret ews servers ad Iteret mail servers are early examples of widely used overlays that implemet importat etwork services. These specialized overlay etworks were developed for various reasos, such as eablig ed-to-ed etwork commuicatio regardless of etwork boudaries caused by etwork address traslatio (NAT). Aother importat reaso for the use of overlays is to provide a etwork service that is ot yet available withi the etwork. For example, multicast routig is a etwork service that to date has bee oly partially adopted o the Iteret. Multicast routig eables a message set to a sigle multicast address to be routed to all receivers that are members of the multicast group. This is importat for reducig etwork traffic for oe-to-may applicatios such as video broadcastig or videocoferecig. Sice multicast routig is ot uiversally supported i Iteret routers, researchers developed a applicatio layer capability for multicast routig called applicatio layer multicast (ALM) or overlay multicast (OM). These techiques, discussed i Chapter 9, use a type of overlay etwork to provide the multicast service for applicatios. Aother aspect of Iteret routig is that some messages are ot routed via the shortest path. This is due to the ecoomics of the etwork providers that collectively provide the backboe of the Iteret. These etwork providers establish etwork regios that coect at peerig poits to other etwork providers. The

16 P2P Networkig ad the Iteret 9 Peer Super Peer FIGURE 1.5 Peers form a overlay etwork (top) that i tur uses etwork coectios i the ative etwork (bottom). The overlay orgaizatio is a logical view that might ot directly mirror the physical etwork topology. traffic load at a peerig poit may be asymmetric. To maximize the value of the etwork to its customers, a etwork provider may route traffic comig ito its peerig poit differetly depedig o the source of the packet. Cosequetly, differet hosts sedig messages to the same destiatio could see sigificatly differet delays. Resiliet overlay etworks (RONs) are a type of overlay etwork that seeks to provide the shortest path i the physical etwork for a message. Such overlays are discussed i Chapter 11. Fially, other examples of etwork services that ca be supported usig a overlay iclude secure delivery of packets, trust establishmet betwee arbitrary edpoits, aoymous message delivery, ad cesorship-resistat commuicatios. Such services are icompletely provided i today s Iteret ad ca be

17 CHAPTER 1 Itroductio more rapidly delivered usig a overlay etwork because applicatio layer features do ot require etwork hardware upgrades. Impact of P2P Traffic o the Iteret The growig popularity of P2P applicatios has created additioal cotroversy due to its impact o etwork performace. Although traffic measuremets of the global Iteret are difficult to collect ad evaluate, data such as that show i Figure 1.6 idicate that a sigificat ad growig proportio of etwork traffic is due to the popularity of P2P applicatios. More recet measuremets 10 i large U.S. Iteret service providers (ISPs) show that P2P traffic cotiues to be aroud 50% of Iteret traffic i access etworks. This situatio is expected to cotiue as P2P applicatios grow i popularity ad are used to deliver more ad more video files to ed systems. From the perspective of the ISP, a relatively small proportio of etwork users ca overwhelm the capacity of the etwork. Sice ed users i may ISPs are charged either a flat rate or for coect time ad ot bit usage rate, the cost of such usage is bore by all the ISP s customers. A secod issue is that the broadbad access etworks were ot desiged for P2P traffic. P2P traffic is iheretly symmetric because each peer acts as both a cliet ad a server i the P2P overlay. But the widely available broadbad access etworks such as Digital Subscriber Loop (DSL) ad cable modems are asymmetric, with dowstream badwidth capacity beig at least five times that of Percetage of Iteret Traffic P2P FTP Web FIGURE 1.6 Relative percetage of Iteret traffic by applicatio category through (# 2006 Velocix, Reprited by permissio).

18 Motivatio for P2P Applicatios 11 upstream capacity. Thus the etworks of broadbad ISPs are beig overloaded by large volumes of upstream traffic produced by P2P applicatios. The outcome of these coflicts betwee P2P applicatios ad etwork providers depeds i part o the cotiued popularity of P2P applicatios, particularly for media delivery versus the emergece of other distributio models that provide the same cost/beefit. A discussio of curret approaches to ISP maagemet of overlay traffic is foud i Chapter 15. MOTIVATION FOR P2P APPLICATIONS P2P from the Ed User s Perspective Though P2P applicatios have trasformed the typical user s experiece of gettig cotet ad commuicatio services from the Iteret, at the same time other popular Iteret applicatios have ot bee built with P2P techology. Examples iclude social etworkig sites such as MySpace ad cotet-sharig sites such as YouTube. Both P2P applicatios ad these Web-based applicatios provide free services to large umbers of ed users. But the owers of the Web-based applicatios geerate reveue usig ilie advertisemets. These Websites ca measure ad viewership ad click-throughs. Depedig o the applicatio, the Websites ca also relate these statistics to user iformatio that the Website gathers. This reveue model has ot bee successfully itegrated ito popular P2P applicatios. Usage statistics gatherig, which drives ad-based reveue, is more difficult i the P2P architecture because it is highly distributed. Additioally, as discussed i Chapter 15, the distributed architecture ad depedece o user-cotrolled ed system resources mea that it is more difficult to provide expected levels of service quality. Oe might the argue that P2P is primarily a low barrier of etry, eablig techology for ew applicatios. Oce prove, such applicatios ca be replaced with easier-to-maage ad more reliable cliet/server techology. But P2P offers a uiquely self-scalig architecture, i which icreased participatio icreases the capacity of the system. This plus the cost differetial eabled by usig ed-system resources suggest that P2P should always be able to provide certai types of services at a cost level ot achievable by cliet/server architectures. Further, popular applicatios that have reached a critical mass have bee historically difficult to retire or replace, udermiig the practicality of replacig P2P applicatios with correspodig cliet/server oes. Whe a user iteracts with a applicatio, what features tell the user that it is implemeted usig a P2P overlay? There is o sigle fuctio that ca t be implemeted i both architectures. But as the usage grows to a global commuity with sigificat iformatio sharig, the differece i scalig properties meas that, ideally, P2P should be able to support a much larger degree of iteractio i terms of umber of cocurret users ad amout of cotet that ca be mutually

19 12 CHAPTER 1 Itroductio shared. For example, it is widely kow that Web search egies idex oly a portio of the Web. Could a P2P architecture eable Web search to cover more cotet ad provide more powerful sematic search capability? The aswers to such questios will impact the future of P2P architectures. Is P2P ¼ Piracy? P2P is ot the first techological iovatio to have its iitial success due to somewhat less tha ideal use. If P2P file sharig had from the start icluded ways for cotet owers to obtai licesig reveue, the role of P2P as a trasformatioal techology would ot have bee obscured by the piracy associatio. Certaily, methods exist for protectig licesed cotet that ca be applied i P2P file-sharig systems. Cosequetly, we believe P2P is a disruptive techology that has importat legitimate uses. I the case of music file sharig, early P2P systems demostrated a large market for a ew distributio model ad ew busiess model. This ew distributio model uses global search, exchage of cotet directly betwee ed users, highquality audio, the capability to select idividual tracks, ad the ability to use the cotet o a variety of persoal devices. Further, this distributio model is ot restricted to cotet provided by the major labels. It is a low-barrier-of-etry meas for idepedet artists ad others to publish cotet for cosideratio by a wide audiece, without the requiremet to go through a music publisher. As for the busiess model, paymet for idefiite persoal use of track playback is widely accepted. Subscriptio models have show viability. Others such as ad-drive models are i trial phases. P2P Stregths ad Beefits Much of this book discusses the details of desigig P2P applicatios ad overlays. As a prelude to that, we should cosider the beefits as well as the limitatios of buildig ad deployig a applicatio usig the P2P approach versus covetioal cliet/server or Web-based approaches. Naturally, P2P might ot be the best choice i may cases. A P2P overlay is a collectio of distributed etworked hosts whose resources are available for use by the P2P applicatios associated with the overlay. These resources iclude computatio, etwork capacity, ad file storage. While their host is coected to the overlay, each ed user shares i the cost of operatig the overlay. This cost sharig by the participats lowers the barrier of etry to overlay providers. The low barrier of etry meas that little hardware or etwork ivestmet is eeded to lauch a P2P applicatio. As discussed earlier, the P2P architecture is iheretly self-scalable, sice each ew peer adds additioal capacity to the system. However, the developers of early P2P applicatios soo discovered that ot all peers have equal capacity to cotribute. For example, the host might be relatively limited i terms of CPU speed ad memory capacity. Or the host might be behid a firewall, makig it

20 Motivatio for P2P Applicatios 13 difficult for that peer to participate i the routig algorithm of the overlay. Or the host might be used for other applicatios that cosume much of the available capacity. Cosequetly, some desigs have orgaized peers ito differet categories depedig o their capacity ad reliability. The more capable or super peers might perform all the overlay operatios, whereas the less capable peers play a more limited role. The self-scalig property by itself does t ecessarily traslate ito good performace uder heavy loads sice the load might ot be uiformly distributed across the overlay. To illustrate, cosider the well-kow pheomeo of flash crowds that occurs whe a very popular item is first available at a Website. As word spreads about the availability of this ew item, large umbers of users simultaeously try to retrieve it usig their Web browsers. This creates a sudde ad excessive load o the Web servers that provide the object. Examples of objects that cause flash crowds iclude major ews stories or ew music or video releases by popular artists. Flash crowds ad less dramatic ueve loadig ca also occur i P2P overlays. O the Web, oe techique to redistribute the load is to use Web caches that are distributed aroud the Iteret. Caches are placed alog the request path for a Web page ad cotai copies of objects that have bee recetly retrieved. Whe a browser requests a object, the request is first routed to earby caches. If the object is located there, it will be retured to the browser without the request ever reachig the Web server. Some P2P overlay desigs use a similar approach by keepig copies of objects that have bee retrieved at itermediate peers alog the search path. Whe aother request for the object is routed alog the same path, the itermediate peer will retur the object before the request reaches the origial peer servig the object. As discussed later i Chapter 11, such techiques do t work i all overlays. I additio, as objects lose popularity, a method is eeded to replace old ifrequetly refereced objects with ewer more popular oes. The typical peer lifetime i a P2P overlay is short ad of upredictable duratio. The lifetime of each peer is subject to local decisios of the user ad is outside the cotrol of the overlay desiger. To successfully operate, a overlay must compesate for the variability i peer membership ad dyamic behavior of participatig peers. A key parameter affectig the desig is the average rate at which peers joi ad leave the overlay, or the chur rate. A importat feature of P2P overlays is the ability to self-orgaize i the face of this dyamic behavior. We ll discuss self-orgaizatio further i Chapter 2. Volatility i peer membership effects peer reliability, aother importat property of P2P overlays. I geeral, reliability improves with icreased redudacy. For example, if copies of a object are placed at multiple ureliable peers, it is more likely for some searchig peer to locate the object tha if a sigle ureliable peer stores the object. Likewise, peers eed to idetify a set of eighbors for routig messages to the rest of the overlay. A larger eighbor set ca be useful for icreasig the likelihood that at least oe peer is olie whe a message is to

Outsourcing Toolkit. a special advertising supplement. published by asae & the center for association leadership

Outsourcing Toolkit. a special advertising supplement. published by asae & the center for association leadership Outsourcig Toolkit a special advertisig supplemet to the SEPTEMBER 2009 issue of Associatios ow published by asae & the ceter for associatio leadership 2009 Outsourcig Toolkit What s the best way to maage

More information

Crowds: Anonymity for Web Transactions

Crowds: Anonymity for Web Transactions Crowds: Aoymity for Web Trasactios Michael K. Reiter ad Aviel D. Rubi AT&T Labs Research I this paper we itroduce a system called Crowds for protectig users aoymity o the worldwide-web. Crowds, amed for

More information

Spin-out Companies. A Researcher s Guide

Spin-out Companies. A Researcher s Guide Spi-out Compaies A Researcher s Guide Cotets Itroductio 2 Sectio 1 Why create a spi-out compay? 4 Sectio 2 Itellectual Property 10 Sectio 3 Compay Structure 15 Sectio 4 Shareholders ad Directors 19 Sectio

More information

Cognizant s Core Values and Standards of Business Conduct

Cognizant s Core Values and Standards of Business Conduct Cogizat s Core Values ad Stadards of Busiess Coduct Table of Cotets Itroductio 2 Our Reputatio is i Your Hads 3 The Right Way at Cogizat 4 Our Core Values ad Stadards i Actio 4 To Whom these Stadards Apply

More information

By Deloitte & Touche LLP Dr. Patchin Curtis Mark Carey

By Deloitte & Touche LLP Dr. Patchin Curtis Mark Carey C o m m i t t e e o f S p o s o r i g O r g a i z a t i o s o f t h e T r e a d w a y C o m m i s s i o T h o u g h t L e a d e r s h i p i E R M R I S K A S S E S S M E N T I N P R A C T I C E By Deloitte

More information

Managing deliverability. Technical Documentation Adobe Campaign v6.1

Managing deliverability. Technical Documentation Adobe Campaign v6.1 Maagig deliverability Techical Documetatio Adobe Campaig v6.1 2014, Adobe All rights reserved. Published by Adobe Systems Ic. Terms of use Privacy Ceter A trademark symbol (,, etc.) deotes a Adobe trademark.

More information

SIP TRUNKING 101: A Primer

SIP TRUNKING 101: A Primer SIP TRUNKING 101: A Primer AUGUST 2012 KEY BENEFITS Cost reductio i the order of 30-50% Icreased flexibility, as umbers are o loger tied to locatios. Cost effective disaster recovery capability Support

More information

Big Data. Better prognoses and quicker decisions.

Big Data. Better prognoses and quicker decisions. Big Data. Better progoses ad quicker decisios. With Big Data, you make decisios o the basis of mass data. Everyoe is curretly talkig about the aalysis of large amouts of data, the Big Data. It gives compaies

More information

Work Placement in Third-Level Programmes. Edited by Irene Sheridan and Dr Margaret Linehan

Work Placement in Third-Level Programmes. Edited by Irene Sheridan and Dr Margaret Linehan Work Placemet i Third-Level Programmes Edited by Iree Sherida ad Dr Margaret Lieha Work Placemet i Third-Level Programmes Edited by Iree Sherida ad Dr Margaret Lieha The REAP Project is a Strategic Iovatio

More information

Optus Partners for Mobility Solutions.

Optus Partners for Mobility Solutions. PARTNER SOLUTIONS GUIDE Optus Parters for Mobility Solutios. Your essetial guide to improvig productivity ad growig your busiess with cost-effective, comprehesive mobility solutios o the Optus Mobile etwork.

More information

Leadership Can Be Learned, But How Is It Measured?

Leadership Can Be Learned, But How Is It Measured? Maagemet Scieces for Health NO. 8 (2008) O C C A S I O N A L PA P E R S Leadership Ca Be Leared, But How Is It Measured? How does leadership developmet cotribute to measurable chages i orgaizatioal performace,

More information

Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks

Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks Dryad: Distributed Data-Parallel Programs from Sequetial uildig locks Michael Isard Microsoft esearch, Silico Valley drew irrell Microsoft esearch, Silico Valley Mihai udiu Microsoft esearch, Silico Valley

More information

Giving Domestic Customers a Choice of Electricity Supplier

Giving Domestic Customers a Choice of Electricity Supplier Office of Gas ad Electricity Markets Givig Domestic Customers a Choice of Electricity Supplier REPORT BY THE COMPTROLLER AND AUDITOR GENERAL HC 85 Sessio 2000-2001: 5 Jauary 2001 Office of Gas ad Electricty

More information

Securing your business

Securing your business Iteratioal Chamber of Commerce The world busiess orgaizatio Securig your busiess A compaio for small or etrepreeurial compaies to the 2002 OECD Guidelies for the security of etworks ad iformatio systems:

More information

Smarter MRO 5 strategies for increasing speed, improving reliability, and reducing costs all at the same time

Smarter MRO 5 strategies for increasing speed, improving reliability, and reducing costs all at the same time Smarter MRO 5 strategies for icreasig speed, improvig reliability, ad reducig costs all at the same time Cotets Itroductio 3 The ed state: A itegrated approach to MRO 4 Achieve short ad cosistet turaroud

More information

Arecent solicitation from the National Science Foundation

Arecent solicitation from the National Science Foundation Cliet-Server Computig WWW Traffic Reductio ad Load Balacig through Server-Based Cachig Azer Bestavros Bosto Uiversity This cachig protocol exploits the geographic ad temporal locality of referece exhibited

More information

Handling clinical negligence claims in England. REPORT BY THE COMPTROLLER AND AUDITOR GENERAL HC 403 Session 2000-2001: 3 May 2001

Handling clinical negligence claims in England. REPORT BY THE COMPTROLLER AND AUDITOR GENERAL HC 403 Session 2000-2001: 3 May 2001 Hadlig cliical egligece claims i Eglad REPORT BY THE COMPTROLLER AND AUDITOR GENERAL HC 403 Sessio 2000-2001: 3 May 2001 Hadlig cliical egligece claims i Eglad REPORT BY THE COMPTROLLER AND AUDITOR GENERAL

More information



More information

Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising

Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising Joural of Machie Learig Research 14 (2013) 3207-3260 Submitted 9/12; Revised 3/13; Published 11/13 Couterfactual Reasoig ad Learig Systems: The Example of Computatioal Advertisig Léo Bottou Microsoft 1

More information

Student Handbook. Regulations, Policies, and Procedures

Student Handbook. Regulations, Policies, and Procedures Studet Hadbook 2014 15 Regulatios, Policies, ad Procedures This hadbook complemets the uiversity s Studet Maual of Uiversity Policies ad Regulatios ad provides a statemet of policies ad academic issues

More information

EMCDDA PAPERS Residential treatment for drug use in Europe

EMCDDA PAPERS Residential treatment for drug use in Europe ISSN 2315-1463 EMCDDA PAPERS Residetial treatmet for drug use i Europe Cotets: Summary (p. 2) I Itroductio (p. 2) I Historical perspective of residetial treatmet for drug users (p. 4) I Extet ad ature

More information

World Bank Group Strategy. October 2013

World Bank Group Strategy. October 2013 World Bak Group Strategy October 2013 Cotets Executive Summary 1 World Bak Group Strategy 5 1. Itroductio 5 A. The Developmet Ageda for Reducig Poverty ad Sharig Prosperity 6 B. Focusig the WBG o the

More information

Child: Family: Place: Radical efficiency to improve outcomes for young children. Croydon

Child: Family: Place: Radical efficiency to improve outcomes for young children. Croydon Child: Family: Place: Radical efficiecy to improve outcomes for youg childre Croydo 2 cotets Foreword...5 Executive summary...7 1 Cotext...13 The public sector eeds ew models to improve public services...14

More information


ASSET MANAGEMENT SoLUTioNS ASSET Maagemet Solutios A Trusted Service Provider ABS Nautical Systems, the software developmet divisio of ABS, desigs applicatios to meet the eeds of the maritime commuity. ABS Nautical

More information

Putting Cloud security in perspective

Putting Cloud security in perspective Capgemii Immediate Istat. Flexible. Revolutioary. the way we see it Puttig Cloud security i perspective The move towards Cloud Computig will be see as oe of the defiig treds of 2010. There is growig acceptace

More information

Things Your Next Firewall Must Do

Things Your Next Firewall Must Do 10 Thigs Your Next Firewall Must Do Itroductio: 10 Thigs Your Next Firewall Must Do Much has bee made about brigig applicatio visibility ad cotrol ito etwork security. The reaso is obvious: applicatios

More information

Big Data in Payments Unparalleled Opportunity for Strategic Excellence

Big Data in Payments Unparalleled Opportunity for Strategic Excellence A Poit of View Big Data i Paymets Uparalleled Opportuity for Strategic Excellece Small data is goe. Data is just goig to get bigger ad bigger ad bigger, ad people just have to thik differetly about how

More information

HOMEBUYING STEP BY STEP. A Consumer Guide and Workbook

HOMEBUYING STEP BY STEP. A Consumer Guide and Workbook HOMEBUYING STEP BY STEP A Cosumer Guide ad Workbook CMHC HOME TO CANADIANS Caada Mortgage ad Housig Corporatio (CMHC) has bee Caada s atioal housig agecy for more tha 65 years. Together with other housig

More information Guiding the Way to Higher Education: Step-by-Step To College Workshops for Students Late High School Curriculum Grades 11 and 12 Guiding the Way to Higher Education: Step-by-Step To College Workshops for Students Late High School Curriculum Grades 11 and 12 Guidig the Way to Higher Educatio: Step-by-Step To College Workshops for Studets Late High School Curriculum Grades 11 ad 12 Copyright 2008 These materials are copyrighted

More information

Guide for Canadian Small Businesses

Guide for Canadian Small Businesses Guide for Caadia Small Busiesses 5&(5HY T he success of small busiess is a essetial part of Caada s ecoomic growth. At the Caada Customs ad Reveue Agecy (CCRA), our goal is to provide all the support

More information