Overview : Computer Networking. Loss Recovery. Multicast Issues. Implosion. Retransmission. Multicast Challenges. Content Distribution Networks

Transcription

1 Overview Multicast Challenges 5-44: Computer Networking Lecture 25: Multicast Challenges, CDN and P2P systems Content Distribution Networks Peer-to-Peer Networks 2/3/0 2 Multicast Issues eliable transfer ACK/NACK Implosion Exposure eliable Multicast Protocols calable eliable Multicast eliable Multicast Transport Protocol Pragmatic General Multicast Lightweight Multicast ervice Congestion control Loss ecovery ender-reliable Wait for ACKs from all receivers. e-send on timeout or selective ACK Per receiver state in sender not scalable ACK implosion eceiver-reliable eceiver NACKs (resend request) lost packet Does not provide 00% reliability NACK implosion 2/3/0 3 2/3/0 4 Implosion etransmission Packet is lost 2 All 4 receivers request a resend e-transmitter Options: sender, other receivers How to retransmit Unicast, multicast, scoped multicast, retransmission group, Problem: Exposure 2/3/0 5 2/3/0 6

2 Exposure Ideal ecovery Model Packet does not reach ; eceiver requests a resend Packet resent to all 4 receivers Packet reaches but is lost before reaching other eceivers Only one receiver sends NACK to the nearest or with packet 2 esent packet 2 esent packet epair sent only to those that need packet 2/3/0 7 2/3/0 8 Aside: Using the outers Multicast Issues outer TX NACK outers do transport level processing: Buffer packets Combine ACKs end retransmissions Model solves implosion and exposure, but not scalable Violates end-to-end argument eliable transfer ACK/NACK Implosion Exposure eliable Multicast Protocols calable eliable Multicast eliable Multicast Transport Protocol Pragmatic General Multicast Lightweight Multicast ervice Congestion control 2/3/0 9 2/3/0 0 calable eliable Multicast (M) M equest uppression Originally designed for wb eceiver-reliable NACK-based Every member may multicast NACK or retransmission 2/3/0 Packet is lost; requests resend to ource and eceivers X 2 Delay varies by distance Packet is resent; and no longer have to request a resend esent packet X 2/3/0 2 X 2

3 equest Damping Deterministic uppression eceivers start timers with delay = C x d s,r tochastic uppression tart timers with delay = U[0,D2] x d s,r M tar Topology Packet is lost; All eceivers request resends X 2 Packet is resent to all eceivers esent packet Delay is same length 2/3/0 3 2/3/0 4 M (ummary) What s Missing? NACK/etransmission suppression Delay before sending Delay based on TT estimation Deterministic + tochastic components Periodic session messages Full reliability Estimation of distance matrix among members Losses at link (A,C) causes retransmission to the whole group Only retransmit to those members who lost the packet [Only request from the nearest responder] 0.99 C 0 0 A B D E F ender eceiver 2/3/0 5 2/3/0 6 Local ecovery Application-level hierarchy Fixed v.s. dynamic TTL scoped multicast outer supported eliable Multicast Transport Protocol (MTP) eliable Multicast Transport Protocol by Purdue and AT&T esearch Labs Designed for file dissemination (singlesender) Deployed in AT&T s billing network 2/3/0 7 2/3/0 8 3

4 MTP: Fixed Hierarchy MTP: Comments cvr unicasts periodic ACK to its Designated eceiver (D) D unicasts its own ACK to its parent cvr chooses closest statically configured (D) Mcast or unicast retransmission Based on percentage of requests coped mcast for local recovery D D 5 D D eceiver * outer D +: Heterogeneity Lossy link or slow receiver will only affect a local region : Position of D critical tatic hierarchy cannot adapt local recovery zone to loss points 2/3/0 9 2/3/0 20 Pragmatic General Multicast Pragmatic General Multicast Cisco s reliable multicast protocol NACK-based, with suppression epair only forwarded to the NACKers Packet reaches only ;,, request resends outers remember resend requests X 2 Packet resent to,, ; Not resent to esent packet 2/3/0 2 2/3/0 22 Light-weight Multicast ervice (LM) LM: Definitions Enhance multicast routing with selective forwarding LM extends router forwarding - what routers are meant to do in the first place No packet storing or processing at routers trictly IP: no peeking into higher layers eplier eceiver volunteered to answer requests Turning point Where requests start to move downstream Directed mcast Mcast to a subtree eplier link eplier X Turning point 5 6 2/3/0 23 2/3/0 24 4

5 LM with eplier Links LM with eplier Links Packet reaches only ; requests resend s from each receiver follow replier links equest from replier links go up towards the ource Packet is resent to all eceivers eplier link 2 eplier link eplier link esent packet eplier link Turning point X Turning point /3/0 25 2/3/0 26 Multicast Issues Multicast Congestion Control eliable transfer ACK/NACK Implosion Exposure eliable Multicast Protocols calable eliable Multicast eliable Multicast Transport Protocol Pragmatic General Multicast Lightweight Multicast ervice Congestion control What if receivers have very different bandwidths? end at max? end at min? end at avg????mb/s 00Mb/s Mb/s 00Mb/s Mb/s 56Kb/s 2/3/0 27 2/3/0 28 Video Adaptation: LM Layered Media treams eceiver-driven Layered Multicast Layered video encoding Each layer uses its own mcast group On spare capacity, receivers add a layer On congestion, receivers drop a layer Join experiments used for shared learning 0Mbps 0Mbps 52Kbps 0Mbps 28Kbps joins layer, joins layer 2 joins layer 3 join layer, join layer 2 fails at layer 3 joins layer, fails at layer 2 2/3/0 29 2/3/0 30 5

6 Drop Policies for Layered Multicast Priority Packets for low bandwidth layers are kept, drop queued packets for higher layers equires router support Uniform (e.g., drop tail, ED) Packets arriving at congested router are dropped regardless of their layer Which is better? Intuition vs. reality! LM Intuition Uniform Better incentives to well-behaved users If oversend, performance rapidly degrades Clearer congestion signal Allows shared learning Priority Can waste upstream resources Hard to deploy LM approaches optimal operating point Uniform is already deployed 2/3/0 3 2/3/0 32 LM Intuition eceiver-driven Layered Multicast Performance Uniform vs. Priority Dropping Uniform Priority Each layer a separate group eceiver subscribes to max group that will get through with minimal drops Dynamically adapt to available capacity Use packet losses as congestion signal Assume no special router support Packets dropped independently of layer 0 Offered load 2/3/0 33 2/3/0 34 LM Join Experiment Join Experiments eceivers periodically try subscribing to higher layer If enough capacity, no congestion, no drops Keep layer (& try next layer) If not enough capacity, congestion, drops Drop layer (& increase time to next retry) What about impact on other receivers? Layer Time 2/3/0 35 2/3/0 36 6

7 Overview Multicast Challenges Content Distribution Networks Peer-to-Peer Networks Motivation Problem of traditional client-server model ingle point of failure (Do Attack) Not calable olution eplication (CDN) Hosts connect to peers directly (P2P) 2/3/0 37 2/3/0 38 Content Distribution Networks eplicate content on many servers Challenges How to replicate content Where to replicate content How to find replicated content How to choose among know replicas How to direct clients towards replica Discussed in DN/server selection lecture DN, HTTP 304 response, anycast, etc. Akamai How Akamai Works How is content replicated? Akamai only replicates static content Modified name contains original file Akamai server is asked for content First checks local cache If not in cache, requests file from primary server and caches file 2/3/0 39 2/3/0 40 How Akamai Works Clients fetch html document from primary server E.g. fetch index.html from cnn.com ULs for replicated content are replaced in html E.g. <img src= > replaced with <img src= > Client is forced to resolve axyz.g.akamaitech.net hostname 2/3/0 4 How Akamai Works oot server gives N record for akamai.net Akamai.net name server returns N record for g.akamaitech.net Name server chosen to be in region of client s name server TTL is large G.akamaitech.net nameserver choses server in region hould try to chose server that has file in cache - How to choose? Uses axyz name and consistent hash TTL is small 2/3/0 42 7

8 How Akamai Works Akamai ubsequent equests cnn.com (content provider) DN root server Akamai server cnn.com (content provider) DN root server Akamai server Get index. html 2 3 End-user Get foo.jpg Get /cnn.com/foo.jpg Akamai high-level DN server Akamai low-level DN server Closest Akamai server Get index. html 2 Akamai high-level DN server End-user Get 0 /cnn.com/foo.jpg Akamai low-level DN server Closest Akamai server 2/3/0 43 2/3/0 44 Consistent Hash view = subset of all hash buckets that are visible Desired features moothness little impact on hash bucket contents when buckets are added/removed pread small set of hash buckets that may hold an object regardless of views Load across all views # of objects assigned to hash bucket is small 2/3/0 45 Consistent Hash Example Construction 0 Assign each of C hash buckets to 4 random points on mod 2 n circle, where, hash key size = n. Bucket 2 Map object to random position on unit interval Hash of object = closest bucket 8 Monotone addition of bucket does not cause movement between existing buckets pread & Load small set of buckets that lie near object Balance no bucket is responsible for large number of objects 2/3/ Overview Multicast Challenges Content Distribution Networks Peer-to-Peer Networks Peer-to-peer networks Typically each member stores content that it desires Basically a replication system for files Always the tradeoff between possible location of files and searching difficulties Peer-to-peer allows files to be anywhere searching is the challenge Other challenges: Dynamic member list cale 2/3/0 47 2/3/0 48 8

9 Example: Napster Example: Gnutella Centralized Indexing On startup, client contacts central server and reports list of files To download a file Client first contact centralized server to find the location of the file Transfer is done peer-to-peer Hybrid scheme Advantage? Disadvantage? 2/3/0 49 Distribute file location Idea: multicast the request Hot to find a file: end request to all neighbors Neighbors recursively multicast the request Eventually a machine that has the file receives the request, and it sends back the answer Advantages: Totally decentralized, highly robust Disadvantages: Not scalable; the entire network can be swamped with request (to alleviate this problem, each request has a 2/3/0 TTL) 50 Example: Freenet Addition goals to file location: Provide publisher anonymity, security esistant to attacks a third party shouldn t be able to deny the access to a particular file (data item, object), even if it compromises a large fraction of machines Architecture: Each file is identified by a unique identifier Each machine stores a set of files, and maintains a routing table to route the individual requests Freenet Query User requests key XYZ not in local cache Looks up nearest key in routing table and forwards to corresponding node If request reaches node with data, it forwards data back to upstream requestor equestor adds file to cache, adds entry in routing table Any node forwarding reply may change the source of the reply helps anonymity If data not found, failure is reported back 2/3/0 5 2/3/0 52 Freenet Features Nodes tend to specialize in searching for similar keys over time LU cache: Files are not guaranteed to live forever Files can be encrypted Messages have random 64 bit ID for loop detection andom initial TTL for strong anonymity Freenet ummary Advantages Provides publisher anonymity Totally decentralize architecture robust and scalable esistant against malicious file deletion Disadvantages Does not always guarantee that a file is found, even if the file is in the network 2/3/0 53 2/3/0 54 9

10 Conclusions The key challenge of building wide area P2P systems is a scalable and robust location service olutions covered in this lecture Naptser: centralized location service Gnutella: broadcast-based decentralized location service Freenet: intelligent-routing decentralized solution (but correctness not guaranteed; queries for existing items may fail) Other solutions: Chord, CAN 2/3/0 55 0