Communication Networks The University of Kansas EECS 780 Multimedia and Session Control James P.G. Sterbenz Department of Electrical Engineering and Computer Science Information Technology and Telecommunications Research Center The University of Kansas jpgs@eecs.ku.edu http://www.ittc.ku.edu/~jpgs/courses/nets 28 April 2010 rev. 10.0 2004 2009 James P.G. Sterbenz
Multimedia and Session Control Outline MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.3 Session Control 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-2
Multimedia and Session Control MS.1 Multimedia Applications MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.3 Session Control 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-3
Multimedia Applications Overview Multimedia applications involve audio and/or video perhaps in addition to conventional data Modes of operatoin streaming interactive 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-4
Multimedia Applications Streaming Streaming multimedia media is streamed from client to server not download and then play user may have back-channel to server to control playback Examples Youtube, Dailymotion, Hulu Lecture AL IPTV: television broadcast over IP Internet VOD (video on demand) 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-5
Multimedia Applications Interactive Interactive multimedia users communicate with using audio and/or video peer-to-peer interaction E2E or shared multicast with reflector may combine with collaborative data sharing document sharing or shared whiteboard distance learning with information access Lecture AL 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-6
Examples Multimedia Applications Interactive Examples VOIP (voice over IP) telephony and video conferencing e.g. skype shared collaboration environments virtual reality interactions, e.g Second Life multiuser games 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-7
Multimedia and Session Control MS.2 Multimedia Streaming and Transport MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.2.1 Multimedia streaming and RTSP MS.2.2 Multimedia transport: RTP and RTCP MS.3 Session Control 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-8
Multimedia Streaming and Transport MS.2.1 Multimedia Streaming and Transport MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.2.1 Multimedia streaming and RTSP MS.2.2 Multimedia transport: RTP and RTCP MS.3 Session Control 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-9
Multimedia Streaming Stream Transfer Mode Various mechanisms to start stream explicit client request server push may or may not establish connection state Data flow synchronisation and control embedded or out-of-band SETUP CONNECT REQUEST RELEASE 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-10
Server Multimedia Streaming Internet-Based Techniques source of media stream Client media player Transport end-to-end data transfer and control protocols is TCP appropriate? 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-11
Multimedia Streaming Delay and Jitter Cumulative data constant bit rate video transmission Assume CBR source note that some codecs are VBR time 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-12
Multimedia Streaming Delay and Jitter Cumulative data constant bit rate video transmission Assume CBR source Network imposes jitter why? time 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-13
Multimedia Streaming Delay and Jitter Cumulative data constant bit rate video transmission variable network delay Assume CBR source Network imposes jitter variable network delay Consequences? time 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-14
Multimedia Streaming Delay and Jitter Cumulative data constant bit rate video transmission variable network delay client video reception Assume CBR source Network imposes jitter Received flow no longer CBR implication? time 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-15
Cumulative data Multimedia Streaming Delay and Jitter constant bit rate video transmission client video reception constant bit rate video playout at client variable network delay buffered video Assume CBR source client playout delay Network imposes jitter Received flow no longer CBR receiver must buffer to compensate and playback CBR time 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-16
Multimedia Streaming Client Playback Buffer receiving end system 7 5 6 6 4 3 2 1 playout buffer application maximum playout point Playback buffer absorb jitter adds delay proportional to jitter may also be used to reorder if misordering permitted by TP 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-17
Server Multimedia Streaming Server Operation match stream rate to path bandwidth what is this? match stream rate to client capabilities what is this? 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-18
Server Multimedia Streaming Server Operation match stream rate to path bandwidth congestion avoidance match stream rate to client capabilities flow control 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-19
Client: media player Multimedia Streaming Client Operation adaptive playout to compensate for jitter decompression error concealment GUI (graphical user interface) controls for interactivity 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-20
Multimedia Streaming Internet Transport Transport: RTP over UDP avoid TCP control loop delays avoid TCP congestion control throttling UDP: datagram-based transport with loss tolerance RTP: synchronisation added 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-21
Multimedia Streaming Typical Web-Based Architecture [Kurose] Web browser GETs metafile launches player, passing metafile Media player contacts server Server streams audio/video to player 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-22
Multimedia Streaming Control RTSP Overview RTSP: real time streaming protocol [RFC 2326] client-server application layer protocol User to control of streaming rewind fast forward pause resume repositioning Out-of-band control port 554 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-23
Multimedia Streaming Control RTSP Operation Metafile communicated to web browser Web browser launches player Player sets up RTSP control connection data flow with streaming server [Kurose] 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-24
Multimedia Streaming Control RTSP Metafile Example <title>twister</title> <session> <group language=en lipsync> <switch> <track type=audio e="pcmu/8000/1" src="rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="dvi4/16000/2" pt="90 DVI4/8000/1 src="rtsp://audio.example.com/twister/audio.en/hifi"> </switch> <track type="video/jpeg" src="rtsp://video.example.com/twister/video"> </group> </session> 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-25
Multimedia Streaming Control RTSP Example C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=play S: RTSP/1.0 200 1 OK Session 4231 C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0 C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37 C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 S: 200 3 OK 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-26
Multimedia Streaming and Transport MS.2.1 Multimedia Transport: RTP and RTCP MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.2.1 Multimedia streaming and RTSP MS.2.2 Multimedia transport: RTP and RTCP MS.3 Session Control 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-27
Real-Time Protocol Overview and Transfer Mode RTP: real-time protocol [RFC 3550 / STD 0064] Streaming of data with real-time properties uses UDP for basic transport no connection establishment basic end-to-end multiplexing no reliability no flow or congestion control adds real-time support fields sequence number timestamp source identifiers 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-28
Real-Time Protocol Segment Format Encapsulated in UDP RTP header version = 02 P: padding bytes at end X: extension header CC: CSRC count [4b] PT: payload type [7b] sequence # [16b] timestamp [32b] resolution app dependent source identifiers SSRC CSRC (mixed in) source port# length 02 P X CC PT timestamp destination port # checksum SSRC: synchronisation source id CSRC list: contributing source ids... application payload optional padding sequence # #B pad 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-29
Real-Time Protocol RTP Payloads PT: payload type [www.iana.org/assignments/rtp parameters] Payload formats and encodings specified in various RFCs audio: RFC 3551, etc. video: RFC 2250, etc. 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-30
Real-Time Protocol Control Protocol Overview RTCP: real-time control protocol [RFC 3550 / STD 0064] Protocol to control RTP streams monitoring quality of service convey participant status information periodically transmits RTCP control packets to others RTCP packet contains sender and/or receiver reports report statistics useful to application # packets sent, # packets lost, interarrival jitter, Feedback can be used to control performance sender may modify transmission based on feedback 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-31
Stream synchronisation Bandwidth scaling Real-Time Protocol RTCP Functions todo: expand from Kurose foils 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-32
Multimedia Streaming and Transport MS.3 Session Control MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.3 Session Control MS.3.1 Session layer functions and services MS.3.2 Session control protocols: SIP MS.3.3 Session control protocols: H.323 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-33
Session Definition Session is association among users or application entities e.g. teleconference, distance learning session, game Participants end-system users or application programs network-embedded resources Topology e.g., transcoders, audio mixers set of end-to-end transport flows multipoint (or point-to-point if only 2 participants) 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-34
L8 L7 L5 L4 L3 L2.5 L2 L1.5 L1 Session Control Hybrid Layer/Plane Cube data plane management plane control plane social application session transport network virtual link link MAC physical Session layer (layer 5): control plane 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-35
Session Layer Session Control Protocol application application session session transport transport network network network network link link link link end system intermediate system network link intermediate system intermediate system network end system Session protocol is responsible for coordination/control of application sessions 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-36
Multimedia Streaming and Transport MS.3.1 Session Layer Functions and Services MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.3 Session Control MS.3.1 Session layer functions and services MS.3.2 Session control protocols: SIP MS.3.3 Session control protocols: H.323 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-37
Session Layer Service and Interfaces Session layer (L5) service to application layer (L7) establishes and maintains sessions among users/applications naming and addressing : to identify and locate participants signalling : messages to control application sessions may perform routing functions among session resources higher layer analogue of network layer services Session layer uses transport layer (L4) services a session consists of a coördinated set of end-to-end flows* * this is not the OSI definition, but the layer happens to be in the right place (L5) 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-38
Session Control Protocol Session Establishment Establish session: signalling assist in location and invitation of participants establish one or more transport-layer associations flows or connections discovery of needed resources e.g. transcoders routing among participants and resources establish session state distributed among participants (good scalability) centralised in a session controller (efficient coordination) combination 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-39
Maintain session Session Control Protocol Session Maintenance adjust to dynamic session group membership add and removal of participants merge and split of sessions initiate and terminate transport associations as needed add and remove resources as needed update session state 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-40
Session Control Protocol Session Termination Terminate session signal termination to all participants teardown all transport layer flows and connections release resources remove session state 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-41
Session Control Protocol Signalling Flow 1. Session signalling 2. Flow setup 3. Data transfer 4. Termination user initiation user session establishment end system network SESS-REQUEST SESS-ESTABLISH end system user user negotiation connection establishment SETUP CONNECT data transfer 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-42
Multimedia Streaming and Transport MS.3.2 Session Control Protocols: SIP MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.3 Session Control MS.3.1 Session layer functions and services MS.3.2 Session control protocols: SIP MS.3.3 Session control protocols: H.323 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-43
Session Control Protocols Example: SIP SIP (session initiation protocol) IETF session protocol Internet style signalling based on HTTP messages single component H.323 ITU session protocol telephony style signalling integrated protocol suite for multimedia conferencing: signaling, registration, admission control, transport, codecs 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-44
Session Initiation Protocol Overview SIP (session initiation protocol) IETF session protocol [RFC 3261] Internet style signalling based on HTTP-like messages SDP typically used to describe media characteristics (session description protocol) [RFC 2327] Single component works with RTP but does not mandate it 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-45
Session Initiation Protocol Assumptions and Design Goals Internet-based telephone video conference calls Addressing people identified by names or e-mail addresses Support for: roaming heterogeneous IP-based devices Simplicity 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-46
Call setup Session Initiation Protocol Call Services mapping of user identifier to IP address caller invitation of callee negotiation of media type and encoding unique call identifier mechanisms to end call Call management add new media streams during call change encoding during call invite others to join call transfer and hold calls 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-47
Session Initiation Protocol SIP Message INVITE sip:bob@domain.com SIP/2.0 Via: SIP/2.0/UDP 167.180.112.24 From: sip:alice@hereway.com To: sip:bob@domain.com Call ID: a2e3a@pigeon.hereway.com Content Type: application/sdp Content Length: 885 c=in IP4 167.180.112.24 m=audio 38060 RTP/AVP 0 Bob s IP address unknown SIP servers will resolve Alice specifies in Via: SIP over UDP SDP used for session description 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-48
Session Initiation Protocol Call Example Alice s SIP INVITE message: port number & IP address PCM preferred audio encoding Alice 167.180.112.24 193.64.210.89 Bob Bob s reply message 200 OK his port number & IP address GSM preferred audio encoding SIP messages HTTP message syntax sent over TCP or UDP here sent over RTP/UDP INVITE bob@193.64.210.89 c=in IP4 167.180.112.24 m=audio 38060 RTP/AVP 0 port 5060 port 5060 port 38060 200 OK c=in IP4 193.64.210.89 m=audio 48753 RTP/AVP 3 GSM ACK port 5060 μ Law audio port 48753 Bob's terminal rings time time default SIP port is 5060 [Kurose] 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-49
Session Initiation Protocol Call Negotiation Media can be sent over RTP or some other protocol Codec negotiation: suppose Bob doesn t have PCM μlaw encoder Bob will instead reply with 606 Not Acceptable Reply and list encoders he can use Alice can then send a new INVITE message advertising an appropriate encoder Call be rejected with replies: busy, gone, payment required, forbidden 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-50
Session Initiation Protocol Name Translation and User Location Caller has callee s e-mail address, not IP address Need to get IP address of callee s current host: user moves around DHCP protocol user has different IP devices (PC, PDA, car device) Result can be based on: time of day (work, home) caller (don t want boss to call you at home) status of callee e.g. sent to voicemail when callee is already talking 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-51
Session Initiation Protocol SIP Proxy SIP Proxy used to resolve to IP address Alice sends invite message to her proxy server contains address sip:bob@domain.com Proxy responsible for routing SIP messages to callee possibly through multiple proxies Callee sends response back through the same set of proxies. Proxy returns SIP response message to Alice contains Bob s IP address Note: proxy is analogous to local DNS server 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-52
Session Initiation Protocol SIP Registrar SIP Registrar keeps track of users queried by SIP proxy When SIP client started REGISTER message sent to registrar server Register message example REGISTER sip:domain.com SIP/2.0 Via: SIP/2.0/UDP 193.64.210.89 From: sip:bob@domain.com To: sip:bob@domain.com Expires: 3600 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-53
Session Initiation Protocol Example of Location and Name Resolution Example: jim@umass.edu calls keith@upenn.edu 1. Jim sends INVITE mssage to umass SIP proxy 2. Proxy forwards request to upenn registrar server 3. upenn server returns redirect response to keith@eurecom.fr 4. umass proxy sends INVITE to eurecom registrar 5. eurecom registrar forwards INVITE to 197.87.54.21, which runs Keith s SIP client 6 8. SIP response sent back 9. media sent between clients Note: ACK message not shown SIP proxy umass.edu SIP client 217.123.56.89 SIP registrar upenn.edu SIP registrar eurecom.fr SIP client 197.87.54.21 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-54 1 8 2 3 4 7 9 [Kurose] 6 5
Multimedia Streaming and Transport MS.3.3 Session Control Protocols: H.323 MS.1 Multimedia applications MS.2 Multimedia streaming and transport MS.3 Session Control MS.3.1 Session layer functions and services MS.3.2 Session control protocols: SIP MS.3.3 Session control protocols: H.323 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-55
Session Control Protocols Example: H.323 SIP (session initiation protocol) IETF session protocol Internet style signalling based on HTTP messages single component H.323 ITU standard session protocol telephony style signalling integrated protocol suite for multimedia conferencing: signaling, registration, admission control, transport, codecs 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-56
Multimedia and Session Control Additional References 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-57
Multimedia and Session Control Acknowledgements Some material in these foils comes from the textbook supplementary materials: Kurose & Ross, Computer Networking: A Top-Down Approach Featuring the Internet Sterbenz & Touch, High-Speed Networking: A Systematic Approach to High-Bandwidth Low-Latency Communication http://hsn book.sterbenz.org 28 April 2010 KU EECS 780 Comm Nets Multimedia and Sessions NET-MS-58