Chapter 7: Multimedia Networking Jim Kurose, Keith Ross: Computer Networking: A Top-Down Approach rd edition: Addison-Wesley, July 004 4 th edition: Addison-Wesley, July 007 Chapter 7: Multimedia Networking Jim Kurose, Keith Ross: Computer Networking: A Top-Down Approach Slides modified (edited, reordered, reduced, extended) by Lidia Yamamoto, Univ. Basel, June, 007 (most material comes from the rd edition, some from other sources, plus some gif figures from the 4 th edition) Slides modified (edited, reordered, reduced, extended) by Lidia Yamamoto, Univ. Basel, May 6, 008 7: Multimedia Networking 7- A note on the use of these ppt slides: We re making these slides freely available to all (faculty, students, readers). They re in PowerPoint form so you can add, modify, and delete slides (including this one) and slide content to suit your needs. They obviously represent a lot of work on our part. In return for use, we only ask the following: If you use these slides (e.g., in a class) in substantially unaltered form, that you mention their source (after all, we d like people to use our book!) If you post any slides in substantially unaltered form on a www site, that you note that they are adapted from (or perhaps identical to) our slides, and note our copyright of this material. Thanks and enjoy! JFK / KWR All material copyright 996-008 J.F Kurose and K.W. Ross, All Rights Reserved 7: Multimedia Networking 7- Contents: Multimedia, QoS, CDN, PP Multimedia, SDP, H. Peer-to-Peer (PP) Media: stimulate one of our 5 senses Sight: Text, Graphics, Images, Video Hearing: Audio Smell? Taste? Touch? Multimedia strict definition: more than one medium combined and synchronized together Web browsing or e-mail with text + images Audio-visual communication, e.g. videoconferencing Multimedia documents, Internet context: continuous audio and/or video (also when single medium) : Stored or live audio/video streaming Real-time and/or interactive audio/video 7: Multimedia Networking 7-7: Multimedia Networking 7-4 Multimedia Networking Map Multimedia Networking Map Streaming multimedia: Media main direction of information flow Media player Streaming control by 7: Multimedia Networking 7-5 Interactive multimedia: Application location interactive session invite s call setup statistics on media clock information information for accounting, billing agree on encoding schemes gateway fixed or mobile phone networks 7: Multimedia Networking 7-6
Multimedia Networking Requirements How to transmit audio or video over the Internet? Network performance requirements Parameters: delay, packet loss, bandwidth, jitter (delay variation) File transfer, web browsing, e-mail are elastic applications: flexible in terms of delay/bandwidth, but no loss tolerated Multimedia applications are typically delay-sensitive and require some minimum bandwidth but are loss tolerant: infrequent losses cause minor glitches Technical challenges Currently all packets receive equal, best effort service Except for TCP reliability (loss=0) guarantee, no other performance guarantees are available Scalability: one to many, many to many s 7: Multimedia Networking 7-7 Multimedia Networking Requirements File transfer elastic applications E-mail No loss (max loss = 0) Available transport protocols Web browsing Max loss TCP delay-sensitive, loss tolerant Streaming multimedia Min bandwidth Real-time multimedia Max delay, jitter performance requirements?? UDP? IP Network Layer: Best Effort Delivery Service 7: Multimedia Networking 7-8 Multimedia Networking: Solutions Contents: Multimedia, QoS, CDN, PP Add performance guarantees to the Internet stack Quality of Service (QoS), e.g. Intserv, Diffserv IP Multicast Several attempts, no real success so far Adaptive applications: make the best of best effort and (semi-)real-time support over UDP and even TCP Application-layer solutions: CDNs, PP, Application-layer multicast, Increasingly successful, SDP, H. Peer-to-Peer (PP) 7: Multimedia Networking 7-9 7: Multimedia Networking 7-0 Streaming Stored Multimedia file Media media data playout begins as soon as possible (i.e. before whole file is transmitted) control:vcr or DVD-like: pause, move forwards, backwards, time constraint: should arrive in time for playout control Streaming Live Multimedia examples Internet broadcast station media data control Internet radio talk show Live sports event playout buffer instead of file store a few tens of seconds before playback: cope with jitter no fast-forwarding possible! time constraint same as stored streaming 7: Multimedia Networking 7-7: Multimedia Networking 7-
Cumulative data Streaming Multimedia: Client Buffering Streaming Multimedia: Client Buffering constant bit rate video variable network delay video reception buffered video constant bit rate video playout at variable fill rate, r(t) constant drain rate, s playout delay: d playout delay Client-side buffering with playout delay to compensate for network-added delay and jitter time buffered video Buffer size: b > s*d If large buffer (~file size) possible then avg(r(t)) > s allowed Small buffer requires avg(r(t)) = s If avg(r(t)) < s: starvation 7: Multimedia Networking 7-7: Multimedia Networking 7-4 Streaming Multimedia: UDP or TCP? UDP sends at rate appropriate for (oblivious to network congestion!) often send rate = encoding rate = constant rate then, fill rate = constant rate - packet loss short playout delay (a few seconds) to compensate for jitter error recovery if time permits TCP send at maximum possible rate under TCP buffer fill rate fluctuates due to TCP congestion control larger playout delay in order to smooth TCP delivery rate popular: HTTP/TCP passes more easily through firewalls 7: Multimedia Networking 7-5 Streaming Multimedia: rate(s).5 Mbps encoding 8.8 Kbps encoding Q: how to handle different receive rate capabilities? 8.8 Kbps dialup 00 Mbps Ethernet A: stores, transmits multiple copies of video, encoded at different rates 7: Multimedia Networking 7-6 Multimedia Networking: Partial Map Real Time Streaming Protocol (RTSP) Streaming multimedia: Media player IETF RFC 6 www.rtsp.org User control of streaming media: VCR-like Media RTSP Streaming control by rewind, fast forward, pause, resume, repositioning, etc Client- application-layer protocol request-response: requests from player to ; each request confirmed with a response message from Syntax and operation intentionally similar to HTTP main direction of information flow 7: Multimedia Networking 7-7 7: Multimedia Networking 7-8
RTSP Operation Web browser media player HTTP GET desc. SETUP PLAY media stream PAUSE TEARDOWN Web media Client (browser) requests page containing multimedia Client obtains XML metafile: a of the multimedia, which can consist of several media streams. Link to each media stream: rtsp:// The browser invokes corresponding media player (helper application) based on the content type of the. Each player sets up an RTSP control connection, and a data connection to the streaming RTSP: (out-of band) control protocol request-response RTSP SETUP, PLAY, PAUSE, TEARDOWN 7: Multimedia Networking 7-9 XML Metafile Example <title>twister</title> <session> <group language=en lipsync> <switch> <track type=audio e="pcmu/8000/" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="dvi4/6000/" pt="90 DVI4/8000/" src="rtsp://audio.example.com/twister/audio.en/hifi"> </switch> <track type="video/jpeg" src="rtsp://video.example.com/twister/video"> </group> </session> 7: Multimedia Networking 7-0 RTSP Exchange Example C: SETUP rtsp://audio.example.com/twister/audio.en/lofi RTSP/.0 Transport: rtp/udp; compression; port=056; mode=play S: RTSP/.0 00 OK Session 4 C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/.0 Session: 4 Range: npt=0- C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/.0 Session: 4 Range: npt=7 C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/.0 Session: 4 S: 00 OK 7: Multimedia Networking 7- Multimedia Networking: Protocol Map Streaming multimedia: Media main direction of information flow RTP/UDP, TCP XML metafile RTSP Media player Streaming control by 7: Multimedia Networking 7- Contents: Multimedia, QoS, CDN, PP Interactive, Real-Time Multimedia, SDP, H. Peer-to-Peer (PP) applications: VoIP, IP telephony, video conference, distributed interactive worlds end-end delay requirements: audio: < 50 msec good, < 400 msec OK loss tolerance: % to 0% depending on encoding and application session initialization locate participants, agree on encoding formats, etc. 7: Multimedia Networking 7-7: Multimedia Networking 7-4 4
Multimedia Networking: Partial Map Interactive multimedia: data and control Application statistics on media clock information RTP/UDP RTCP RTP and RTCP RTP (Real-Time Transport Protocol): RFC 550 Transport-level packet format to carry monomedia data (video, audio, animations, ). Payload contains: Payload type, sequence number, timestamp, source identifier(s), media samples Multimedia via multiple synchronized RTP sessions Support for multipoint, bidirectional flows, translators and mixers Generally runs over UDP: unicast or multicast Companion Control Protocol: RTCP: Clock timestamp for media Periodic sender/receiver statistics 7: Multimedia Networking 7-5 7: Multimedia Networking 7-6 RTP Header Multimedia Networking: Partial Map Payload Type (7 bits): type of encoding, e.g.: 0 = PCM mu-law, 64 kbps = GSM, kbps = MPEG video Sequence Number (6 bits) of RTP packet Timestamp ( bits): sample number incremented by one for each sample (not real-time clock) even if source inactive SSRC ( bits): source identifier CSRC ( bits): contributing source id (e.g. mixing) Interactive multimedia: Application location interactive session invite s call setup SDP agree on encoding schemes cod. Seq. no. Timestamp SSRC type CSRC CSRC Samples 7: Multimedia Networking 7-7 7: Multimedia Networking 7-8 Session Initiation Protocol () Call Set Up: Known IP Address IETF RFC 6 Creation and management of multimedia sessions Establish, modify, terminate sessions Invite new participants to existing sessions Add or remove media User location for personal mobility (name mapping and redirection) identity URI = Universal Resource Identifier: locationindependent, e.g. sip:bob@biloxi.com, sip:alice@atlanta.com Address, e.g. sip:bob@9.64.0.89 Multimedia session s in (mainly): SDP = Session Description Protocol (RFC 7) SDP.. Alice s invite message contains SDP session C= protocol (IPv4) & IP address. M= media (audio), port number (8060), preferred encoding ( RTP/AVP 0 = PCM ulaw) to receive audio Bob s 00 OK message contains his own SDP descr.: IP address, port & preferred encoding to recv. ( RTP/AVP = GSM) messages can be sent over TCP or UDP; here sent over RTP/UDP. Default port number is 5060. 7: Multimedia Networking 7-9 7: Multimedia Networking 7-0 5
Call Set Up: Location- independent URI Call Set Up: Location- independent URI sip:alice@atlanta.com Proxy atlanta.com Proxy biloxi.com INVITE 5 sip:bob@biloxi.com 4 Location database Call routing via proxy s sip:bob@biloxi.com Registrar Biloxi.com Registrar: registers locations into database Proxy: informs s about locations, routes calls (both functions may be combined into the same machine) 6 sip:alice@atlanta.com INVITE 5 sip:bob@biloxi.com 4 Proxy atlanta.com Proxy biloxi.com REDIRECT 6 unibas.ch 7 Proxy unibas.ch Location database Call routing via proxy s sip:bob@biloxi.com Registrar Biloxi.com 8 REGISTER 7: Multimedia Networking 7-7: Multimedia Networking 7- Another Example Caller jim@umass.edu places a call to keith@upenn.edu () Jim sends INVITE message to umass proxy. () Proxy forwards request to upenn registrar. () upenn returns redirect response, indicating that it should try keith@eurecom.fr proxy umass.edu 8 Jim 7..56.89 registrar upenn.edu registrar eurecom.fr (4) umass proxy sends INVITE to eurecom registrar. (5) eurecom registrar forwards INVITE to 97.87.54., which is running keith s. (6-8) response sent back (9) media sent directly between s. Note: also a ack message, which is not shown. 7: Multimedia Networking 7- Keith (home) 4 7 9 Keith (now) 97.87.54. 6 5 H. in a Nutshell analogous to registrar, plus accounting and billing functions 7: Multimedia Networking 7-4 H. in a Nutshell ITU recommendation for real-time audio and video across IP networks Comprehensive suite of protocols including: Call set-up, (via RTP),, gatekeepers for accounting and billing, gateways to traditional phone networks,... Intensive support from industry and telecom industry (late 90 s), e.g. Microsoft NetMeeting then gained a large momentum: simple, web flavor, support from the IETF and Internet community Today battle still not decided; other players emerged (e.g. skype with proprietary protocols) 7: Multimedia Networking 7-5 Multimedia Networking: Protocol Map Interactive multimedia: Application location interactive session invite s call setup SDP statistics on media clock information information for accounting, billing agree on encoding schemes RTP/UDP H. RTCP gateway fixed or mobile phone networks 7: Multimedia Networking 7-6 6
Interactive, Real-Time Multimedia Mitigating the impact of loss and delay Example: Internet Phone Mitigating delay Adaptive playout delay adjustment Mitigating loss Forward error correction (FEC) Interleaving Loss concealment Mitigating delay: Adaptive Playout First packet received at time r First playout schedule: begins at p Second playout schedule: begins at p packets Adjust buffer size (hence playout delay) by compressing/elongating silent periods Valid only for voice! packets generated packets received loss playout schedule p' - r playout schedule p - r time 7: Multimedia Networking 7-7 r p p' 7: Multimedia Networking 7-8 Mitigating loss: Forward Error Correction (FEC) with Redundant Packets Mitigating loss: Forward Error Correction (FEC) with Redundant Streams A B C R Redundant packet (inserted one every n packets) Nominal stream (higher quality, e.g. PCM) R = A xor B xor C Lost packet A B C R Redundant stream (lower quality, e.g. GSM) Recovery: B = A xor C xor R + : simple, valid for any media -: consumes /n more bandwidth -: increases playout delay 7: Multimedia Networking 7-9 7: Multimedia Networking 7-40 Mitigating loss: Interleaving Interleaving Small gaps Used in GSM Pros: + : significantly improves perceived quality + : low overhead + : does not increase bandwidth usage + : can handle lost bursts Cons: - : increases playout delay 7: Multimedia Networking 7-4 7: Multimedia Networking 7-4 7
Loss concealment Contents: Multimedia, QoS, CDN, PP No concealment: sent received played A B C A B C A (silence) C Repetition: sent received played A B C A B C A A C packet repetition Interpolation: A C decoded B sent A B C signal, SDP, H. Peer-to-Peer (PP) received A B C played A B C B = f(a, C) time 7: Multimedia Networking 7-4 7: Multimedia Networking 7-44 Multimedia Networking Requirements Multimedia Quality of Service (QoS) File transfer elastic applications E-mail Web browsing delay-sensitive, loss tolerant Streaming multimedia Real-time multimedia Multimedia applications: network audio and video ( continuous media ) No loss (max loss = 0) Available transport protocols Max loss TCP Min bandwidth Max delay, jitter performance requirements?? UDP? IP Network Layer: Best Effort Delivery Service 7: Multimedia Networking 7-45 QoS network provides application with level of performance needed for application to function. 7: Multimedia Networking 7-46 Multimedia, Quality of Service Internet currently offers only best effort service proposals: IntServ: QoS guarantees possible but complex Diffserv: Loose guarantees (per class). Simpler, but still complex to deploy in practice. Future directions IETF Integrated Services (Intserv) Per flow QoS guarantees through resource reservation at all nodes along the path call set-up signaling (RSVP protocol, RFC 05) traffic, QoS declaration per-element admission control and reservation request/ reply -sensitive scheduling 7: Multimedia Networking 7-47 7: Multimedia Networking 7-48 8
IETF Differentiated Services Main concern with Intserv: Scalability: signaling, maintaining per-flow router state difficult with large number of flows in core routers Diffserv approach: simple functions in network core, relatively complex functions at edge routers (or hosts) achieved through differentiated service classes, as opposed to per-flow resource reservation 7: Multimedia Networking 7-49 Edge router: per-flow traffic management marks packets diffserv packet field: IPv4: Type of Service (ToS) IPv6: Traffic Class Classes: Expedited Forwarding (EF): 5 Assured Forwarding (AF) AF 4 AF AF AF Core router: Best Effort: 0 per class traffic management precedence Diffserv Architecture marking scheduling buffering and scheduling based on diffserv field 7: Multimedia Networking 7-50. How should the Internet evolve to better support multimedia? Summary Integrated services (IntServ) philosophy: guarantees through fine-grain end-to-end resource reservations (per flow) Requires new, more complex software in hosts & routers Differentiated services (DiffServ) philosophy: Fewer changes to Internet infrastructure Coarse grain service levels or classes of service "Laissez-faire" no major changes more bandwidth when needed, over-provisioning content distribution, application-layer solutions New architecture attempts Earlier Asynchronous Transfer Mode (ATM), MPLS Active/Programmable Networks More recent: NewArch (USA) Autonomic Communication 7: Multimedia Networking 7-5 Many limitations of current networked multimedia applications are due to network constraints Main problems with both Intserv and Diffserv Complexity, deployment & management cost/effort Crossing legacy best-effort portions of the Internet Socio-economic barriers: who uses who pays Current winner solution is laissez-faire with over-provisioning Research directions Autonomic, self-organizing networks: self-deploying services, including QoS Adaptive functionality, not only at application layer, also at transport and network layer 7: Multimedia Networking 7-5 Contents: Multimedia, QoS, CDN, PP Content Distribution, SDP, H. Peer-to-Peer (PP) Problem: Streaming stored multimedia: large and very popular files (e.g. video from TV broadcast) from single origin in real time Bottleneck at Delay, loss Not a solution: Multicast: not everyone asks content at the same time Web caching: content pulled on demand, fresh information (e.g. news) not cached Solution: CDN Content pushed in advance to s located as close as possible to potential s New business: CDN company Clients: large corporations, e.g. tv/movie industry 7: Multimedia Networking 7-5 7: Multimedia Networking 7-54 9
CDN: Proactive Content Replication An Akamaized Website (Wikipedia) CDN company (e.g., Akamai): customers are large content providers (e.g. CNN) CDN replicates customers content in CDN s. Content downloaded to CDN s ahead of time Placing content close to alleviates load at origin avoids impairments (loss, delay) of sending content over long paths CDN typically in edge/access network origin in North America CDN distribution node CDN in S. America CDN in Europe CDN in Asia origin Akamai CDN 7: Multimedia Networking 7-55 7: Multimedia Networking 7-56 CDN Redirection Mechanism CDN: (Research) Issues 4 Nearby CDN CDNs authoritative DNS www.cdn.com CDN network 4 Origin Content Provider s network www.ocpn.com HTTP request for www.ocpn.com/sports/sports.html HTML page containing link to www.cdn.com/www.ocpn.com/video.mpg DNS query for www.cdn.com Answer from map to determine nearest HTTP request for www.cdn.com/www.ocpn.com/video.mpg Content distribution: from origin to CDN s IP Multicast Application Layer Multicast (Overcast, Scattercast, Yoid) Dedicated Content Managers Replica placement: optimal placement of CDN s Minimum latency and overall bandwidth Discovery: Assign s to CDN s: locate that gives best performance Min latency and load Table (map) indicates distance (cost) from leaf ISP (range of IP addresses) to CDN Can be built based on BGP routing tables, RTT estimates, etc. Map lookup to chose CDN with minimum distance (cost) for an ISP 7: Multimedia Networking 7-57 7: Multimedia Networking 7-58 Contents: Multimedia, QoS, CDN, PP Client- architecture, SDP, H. Peer-to-Peer (PP) Server: content provider always-on host permanent IP address farms for scaling Clients: content consumer communicate with may be intermittently connected may have dynamic IP addresses do not communicate directly with each other 7: Multimedia Networking 7-59 7: Multimedia Networking 7-60 0
PP architecture Hybrid Peer: both content provider and consumer Motivation: share information, e.g. files End systems communicate directly (no rd party s) Decentralized, scalable Issues: content dissemination, search and retrieval intermittent connectivity dynamic IP addresses NAT/firewall traversal non-cooperative nodes security Centralized index e.g. original Napster centralized ) when peer connects, it informs index central : IP address content, e.g. song titles ) Alice queries index for song, index returns Bob s address ) Alice requests and obtains song file from Bob directly Centralized access control e.g. Skype: register/authenticate with central, rest PP (search for s, media exchange) Alice Bob peers 7: Multimedia Networking 7-6 7: Multimedia Networking 7-6 PP Case Study: Skype PP Case Study: Skype Supernode structure: login 7: Multimedia Networking 7-6 PP VoIP software, encrypted calls NAT and firewall traversal Gateways to mobile/fixed phone networks Proprietary protocols over TCP and UDP Supernode structure (precursor: KaZaA) Supernode peer must be fully reachable (not behind NAT) among other criteria: Firewall/NAT traversal via supernodes Media data: direct communication between peers when possible; when not (e.g. firewall) then relay via supernode Codecs in the range ~6kbps-8kbps with loss concealment 7: Multimedia Networking 7-64 CDN+PP Summary CDN increasingly popular Can PP win over CDN? Skype s new Joost service: TV on demand Bittorrent for sharing large files (including videos, software) Legal issues Copyright infringement Dissemination of offensive content, malicious programs (e.g. spyware, adware, viruses), spam (new: VoIP spam!) 7: Multimedia Networking 7-65