Department of Computer Science Institute for System Architecture, Chair for Computer Networks Internet Services & Protocols Multimedia Applications, Voice over IP Dr.-Ing. Stephan Groß Room: INF 3099 E-Mail: stephan.gross@tu-dresden.de
What are Multimedia Applications? Classes of networked multimedia applications Real time interactive Audio/ Video Internet telephony, video conferences Streaming live Audio/ Video digital Radio (DAB, DRM), digital TV Streaming stored Audio/ Video on Demand Video Business TV to the Desktop, Distance Learning Fundamental characteristics Typically delay sensitive end-to-end delay delay jitter But loss tolerant: infrequent losses cause minor glitches Antithesis of data, which are loss intolerant but delay tolerant. 2
Application Requirements application packet loss bandwidth time sensitive Data transfer E-mail Web documents Realtime audio/video Stored audio/video Interactive audio/video no no no tolerant tolerant tolerant elastic elastic elastic audio: 5kbps- 1Mbps video:10kbps- 5Mbps no no no yes, a few 100 ms yes, a few s Yes, a few 100 ms traditional applications multimedia applications 3
Multimedia Requirements for LANs / WANs If audio or video services are intended to be integrated in traditional data networks, the network must be capable for: Guaranteed transmissions Quality of Service! Priority of audio packet against data packets Standardised interfaces Availability and reliability of such communication systems Data security is getting more important for audio/voice transmissions over the Internet (tap-proof) 4
Multimedia over Today's Internet TCP/UDP/IP: best-effort service NO guarantees on delay, loss?????? But you said multimedia apps require QoS and level of performance to be effective!????? Today s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss 5
How to evolve the Internet for better multimedia Support? Integrated services philosophy: Fundamental changes in Internet so that apps can reserve end-to-end bandwidth Requires new, complex software in hosts & routers Laissez-faire no major changes more bandwidth when needed content distribution, application-layer multicast Differentiated services philosophy: Fewer changes to Internet infrastructure, yet provide 1st and 2nd class service. What s your opinion? 6
End-to-End Delay E2E-delay is the sum of delay experienced by packets due to the processing in end systems, interim systems (router, switches etc.) and on transmission line. End system End system delay Sound card Encapsulation (10-30 ms) Encoding (5-10 ms) Operating System IF Processing delay Route determ. Classification Buffering delay Output buffer load Transmission Output line Router Decoding Operating System Playout buffer Switching delay Propagation (5 µs/km) 7
Jitter A packet pair s jitter is the difference between the transmission time gap and the receive time gap Sender: Pkt i Pkt i+1 Receiver: Pkt i Pkt i+1 S i S i+1 jitter Time R i R i+1 Desired time-gap: S i+1 - S i Received time-gap: R i+1 - R i Jitter between packets i and i+1: (R i+1 - R i ) - (S i+1 - S i ) 8
Fixed Playout Delay for Jitter Compensation packets playout schedule p-r packets generated packets received loss playout schedule p'-r time r p p' 9
Packet Loss Problem: Internet might lose / excessively delay packets making them unusable for the session arrival time: Pkt i Pkt i+1 Pkt i+3 app deadline: i i+1 i+2 i+3 usage status:, i used, i+1 late, i+2 lost, i+3 used,... Solution step 1: Design app to tolerate some loss Solution step 2: Design techniques to recover some lost packets within application s time limits 10
Reducing Packet Loss w/in Time Bounds Problem: packets must be recovered prior to application deadline Retransmission unacceptable for real time apps Solution: Forward Error Correction (FEC) on packet level Example: Mixed Quality Streams (MQS) MQS: redundant duplicates of packets; despite of loss, playback possible through redundant packet, even with lower quality One redundant packet per packet single losses compensable Mixing (piggypack) stream with lower quality 11
Bursty Loss Many codecs can recover from short (1 or 2 packet) loss outages Bursty loss (loss of many pkts in a row) creates long outages: quality deterioration more noticeable FEC provides less benefit in a bursty loss scenario (e.g., consider 30% loss in bursts 3 packets long) 12
Interleaving To reduce effects of burstiness, reorder packet transmission Example: break down a 20 ms audio packet into smaller parts of 5 ms each and interleave them Despite of packet loss: partly filled audio parts, missing parts can be interpolated No redundancy, but induces buffering and additional delay; realtime requirements cannot be satisfied if necessary 13
IP Telephony aka Voice over IP (VoIP) Circuit switched telephony (ISDN) Public telephone network Circuit switched telephony 14
Session Initiation Protocol (SIP) Layer 7 protocol Application Layer Comes from IETF Purpose: Signalling of interactive sessions in the Internet Interactive Communication e.g. Multimedia-conferences Internet-Telephony (VoIP) Tele working and teaching Signalling aspects User localisation recognizing of user capabilities Test of user availability Connection establishment Connection negotiation 15
Some Facts about SIP SIP is a Signalling protocol, so it does not define how multimedia streams should be encoded and transported: Real-time Transport Protocol (RTP) is used for audio/video transmissions. SIP is usually implemented on top of UDP (or on top of TCP, too) Signalling messages are generally small and sent infrequently For every request sent, a response is to be received. The response can be used as acknowledgement. To avoid the need of a centralized directory server, SIP uses SIP proxies to route communication messages (comparable with H.323 gatekeepers) and A registration server is used to distribute and announce registrations Uses URL similar addresses/syntax and HTML-elements in messages, so it can be better combined with web technology. 16
The Architectural View of SIP Five main components: User Agent Client (UAC) End systems in SIP-based systems UAC sends a request to an UAS User Agent Server (UAS) Receives requests from clients Accepts and answers to a request, reject a request or routes it to another Proxy Server Primary task: transmitting SIP - protocol elements via routing Negotiator between client and server While routing SIP-messages, they can pass more than one SIP-Proxies 17
The Architectural View of SIP ff. Redirect Server Acts like an UAS Responses to requests of UACs with a redirect information, enabling the UAC to sent the message to an alternative address Do not sent messages automatically Registrar Receives REGISTER-messages and forwards these messages to a Location Service, which stores and returns possible locations for users 18
The Functional View SIP Services Setting up a call Provides mechanisms for caller to let callee know she wants to establish a call Provides mechanisms so that caller and callee can agree on media type and encoding. Provides mechanisms to end call. Locate a callee Maps mnemonic identifier to current IP address Call management Add new media streams during call Change encoding during call Invite others Transfer and hold calls 19
SIP Example: Setting up a Call to a known IP address A l i c e B o b Alice s SIP INVITE message includes her portnumber & IP address. Alice prefers to receive PCM (AVP 0). Bobs 200 OK message shows his port, IP address & GSM as his favourite codec 1 6 7. 1 8 0. 1 1 2. 2 4 INVITE bob@193.64.210.89 c=in IP4 167.180.112.24 m=audio 38060 RTP/AVP 0 port 5060 port 5060 200 OK c=in IP4 193.64.210.89 m=audio 48753 RTP/AVP 3 1 9 3. 6 4. 2 1 0. 8 9 B o b ' s t e r m i n a l r i n g s SIP messages are sent out of band. Audio data should be sent using RTP. p o r t 3 8 0 6 0 ACK port 5060 µ L a w a u d i o Default SIP port is 5060. G S M p o r t 4 8 7 5 3 t i m e t i m e 20
Setting up a call (more) Codec negotiation: Suppose Bob doesn t have PCM ulaw encoder. Bob will instead reply with 606 Not Acceptable Reply and list encoders he can use. Alice can then send a new INVITE message, advertising an appropriate encoder. Rejecting the call Bob can reject with replies busy, gone, payment required, forbidden. Media can be sent over RTP or some other protocol. 21
Example of a SIP message INVITE sip:bob@domain.com SIP/2.0 Via: SIP/2.0/UDP 167.180.112.24 From: sip:alice@hereway.com To: sip:bob@domain.com Call-ID: a2e3a@pigeon.hereway.com Content-Type: application/sdp Content-Length: 885 c=in IP4 167.180.112.24 m=audio 38060 RTP/AVP 0 Notes: HTTP message syntax sdp = session description protocol Here we don t know Bob s IP address. Intermediate SIP servers will be necessary. Alice sends and receives SIP messages using the SIP default port number 5060. Alice specifies in Via: header that SIP client sends and receives SIP messages over UDP Call-ID is unique for every call (like msg-id in email) 22
Name Translation and User Location Caller wants to call callee, but only has callee s name or e-mail address. Need to get IP address of callee s current host: user moves around DHCP protocol user has different IP devices (PC, PDA, car device) Result can be based on: time of day (work, home) caller (don t want boss to call you at home) status of callee (calls sent to voicemail when callee is already talking to someone) Service provided by SIP servers: SIP registrar server SIP proxy server 23
SIP Registrar When Bob starts SIP client, client sends SIP REGISTER message to Bob s registrar server (similar function needed by Instant Messaging) Register Message: REGISTER sip:domain.com SIP/2.0 Via: SIP/2.0/UDP 193.64.210.89 From: sip:bob@domain.com To: sip:bob@domain.com Expires: 3600 24
SIP Proxy Alice sends invite message to her proxy server contains address sip:bob@domain.com Proxy responsible for routing SIP messages to callee possibly through multiple proxies. Callee sends response back through the same set of proxies. Proxy returns SIP response message to Alice contains Bob s IP address Note: proxy is analogous to local DNS server 25
Example using SIP in VoIP jim@umass.edu calls keith@upenn.edu (1) Jim sends an INVITE message to Umass SIP proxy. (2) Proxy forwards request to upenn registrar server. (3) Upenn Server responses with a redirect (use keith@eurecom.fr!) (4) Umass proxy sends INVITE to Eurecom registrar. (5) Eurecom registrar routes INVITE to 197.87.54.21 (Keiths SIP client). (6-8) SIP response is sent back. (9) Audio data are exchanged directly between clients. S I P p r o x y u m a s s. e d u 1 8 2 3 S I P r e g i s t r a r u p e n n. e d u 4 7 S I P r e g i s t r a r e u r e c o m. f r 6 5 S I P c l i e n t 2 1 7. 1 2 3. 5 6. 8 9 jim@umass.edu 9 S I P c l i e n t 1 9 7. 8 7. 5 4. 2 1 keith@upenn.edu 26
VoIP uses RTP Real-time Transport Protocol (RTP) RFC 1889/1890 RTP provides: End-to-end transport, intended for application, which transmit real time traffic like audio, video or simulation data using unicast or multicast connections RTP usually uses UDP for an efficient and less delayed data transmission RTP does not provide Signalling Resource reservation, QoS, guaranteed delivery/ sequence (RSVP) 27
RTP Functions Payload type: Identification of Data audio/video stream encoding Sequence numbers Restoration of packet sequence Detection of packet loss Time stamps Intra-media synchronisation: to remove jitter, delay Inter-media synchronisation: lip synchronisation Connection monitoring (through RTCP) Quality feedback (QoS) Sender based rate adaption for RTP-sessions 28
Real-Time Control Protocol (RTCP) Works in conjunction with RTP. Each participant in RTP session periodically transmits RTCP control packets to all other participants. Each RTCP packet contains sender and/or receiver reports report statistics useful to application Statistics include number of packets sent, number of packets lost, interarrival jitter, etc. Feedback can be used to control performance Sender may modify its transmissions based on feedback 29
RTCP continued For an RTP session there is typically a single multicast address; all RTP and RTCP packets belonging to the session use the multicast address. RTP and RTCP packets are distinguished from each other through the use of distinct port numbers. To limit traffic, each participant reduces his RTCP traffic as the number of conference participants increases. 30
RTCP Packets Receiver report packets: fraction of packets lost, last sequence number, average interarrival jitter. Sender report packets: Synchronization source identifier (SSRC) of the RTP stream, the current time, the number of packets sent, and the number of bytes sent. Source description packets: e-mail address of sender, sender's name, SSRC of associated RTP stream. Provide mapping between the SSRC and the user/host name. 31
Synchronization of Streams RTCP can synchronize different media streams within a RTP session. Consider videoconferencing app for which each sender generates one RTP stream for video and one for audio. Timestamps in RTP packets tied to the video and audio sampling clocks not tied to the local wall-clock time Each RTCP sender-report packet contains (for the most recently generated packet in the associated RTP stream): timestamp of the RTP packet wall-clock time for when packet was created. Receivers can use this association to synchronize the playout of audio and video. 32
Conclusion Internet was build for data applications does not support multimedia applications very well There are several techniques to overcome drawbacks like packet delay, jitter or loss Fixed Playout Delay Mixed Quality Streams & Interleaving Example Application: VoIP uses SIP for signalling and RTP for data transmission Still missing: Quality of Service 33