Special Module on Media Processing and Communication

Transcription

1 Special Module on Media Processing and Communication Multimedia Communication Fundamentals Dayalbagh Educational Institute (DEI) Dayalbagh Agra PHM 961 Indian Institute of Technology Delhi (IITD) New Delhi SIV 864

2 Overview Review of last class History of streaming media Streaming performance requirements Protocol stack for multimedia services Real-time transport protocol (RTP) RTP control protocol (RTCP) Real-time streaming protocol (RTSP)

3 Audio Sampling and Quantization Sampling Quantization

4 Image Effect of spatial resolution

5 Image Spatial resolution and quantization level determine the size of the image y Gray scale (monochrome) image with 8 bit pixel x 256x256x8 bits = 256x256 bytes # of bits per pixel Color image (R, G, B) each color channel pixel is 8 bit 256x256x24 bits = 256x256x3 bytes

6 Video Video is a sequence of images in time Image (Frame) Time

7 Multimedia Communication Classes of multimedia applications 1.) Streaming stored audio and video 2.) Streaming live audio and video 3.) Real-time interactive audio and video Fundamental characteristics Typically delay sensitive end-to-end delay jitter But most times also loss tolerant: infrequent losses cause minor glitches Requirements to communication Transfer protocols with small delay but also weak reliability (TCP, UDP, or something else?) Control protocols for signaling between communicating applications (e.g. phone ringing ) Quality of Service guarantees within the network (IP layer): routers and resource reservations

8 Multimedia Comm (contd) 1. Transport protocols Deliver as much data as possible in short time audio and video data typically have a stream-like behavior (16 kbit/s for compressed audio, 64 kbit/s for PCM audio in telephony, 2 Mbit/s for MPEG-coded video) New transfer protocols needed RTP 2. Control protocols Deliver data with regard to negotiated policies (throughput, delay) and/or signaling information Control protocols needed H.323, SIP 3. Quality of Service (chapter 3.2) Deliver data as fast as possible and with low jitter real-time communication demands low end-to-end delays, typically less than 200 msec. End-to-end delay is limited by the routers, thus e.g. routing strategies have to be modified Network protocol enhancements needed scheduling, resource reservations, traffic shaping

9 Real-time multimedia streaming Real-time multimedia applications Video teleconferencing Internet Telephony (VoIP) Internet audio, video streaming (A-PDUs)

10 Overview: Multimedia Communication Protocols Internet Telephony Video conferencing Internet Radio/TV Messaging and Presence Interactive voice response Unified messaging Application layer Transport (TCP, UDP) Network (IPv4, IPv6) Link layer Physical layer H.323 SIP RTSP RSVP RTCP Signaling Quality of service Media G.711 MPEG RTP Media transport

11 Brief history of streaming media

12 Session Initiation Protocol (SIP) Instead of H.323, also the simpler, Internet-oriented SIP can be used: Defined by IETF SIP long-term vision All telephone calls and video conference calls take place over the Internet People are identified by names or addresses, rather than by phone numbers You can reach the callee, no matter where the callee roams, no matter what IP device the callee is currently using SIP is an application layer signaling protocol that defines initiation, modification and termination of interactive multimedia communication sessions between multiple users Call setup Agree on media type and encoding Maps logical address identifier to current IP address Call management: add new media streams during call, change encoding during call, invite others, transfer and hold calls Bases upon HTTP concepts (message syntax, SIP URLs, responses, )

13 Overview: SIP and sipd Address based on office.com Bob 1. DNS home.com pc1.home.com pc1.home.com INVITE 3. INVITE (proxy mode) Columbia.edu (2) Cisco.com sipd home.com Alice (3) m2.home.com

14 PSTN to IP Call PSTN External T1/CAS 1 Call PBX Internal T1/CAS (Ext: ) 2 Call 7134 Gateway 713x is called a part of Coordinated Dial Plan (CDP) in a Nortel PBX Regular phone (internal) 5 3 Ethernet Direct Inward Dial (DID) - direct and simple No-DID - dial extension, supports more users sipc Bob s phone SIP server sipd SQL database => bob

15 IP to PSTN Call PSTN External T1/CAS Call Note: In this direction there is no distinction between DID and non- DID calls. PBX Regular phone (internal, 7054) Internal T1/CAS Call sipc Bob calls Gateway ( ) 3 SIP server sipd Ethernet SQL database Use sip: @

16 RTP: Streaming performance requirements Sequencing to report PDU loss to report PDU reordering to perform out-of-order decoding Time stamping and Buffering for play out for jitter and delay calculation Payload type identification for media interpretation Error concealment covers up errors from lost PDU by using redundancy in most-adjacent-frame Quality of Service (QoS) feedback from receiver to sender for operation adjustment Rate control sender reduces sending rate adaptively to network congestion

17 Ideal Timing no jitter seconds First RTP-PDU Second RTP-PDU application Third RTP-PDU Send time Play time

18 Reality jitter First RTP-PDU delay Second RTP-PDU Send time Third RTP-PDU Fourth RTP-PDU Play time

19 Jitter (contd.) First RTP-PDU(0) Second RTP-PDU(10) Send time Third RTP-PDU(20) Fourth RTP-PDU (30) Play time

20 Jitter (contd.) Playback buffer At time 00:00:18 At time 00:00:28 At time 00:00:38