RTP Audio and Video for the Internet Colin Perkins TT rvaddison-wesley Boston San Francisco New York Toronto Montreal London Munich Paris Madrid Capetown Sydney 'lokyo Singapore Mexico City
CONTENTS PREFACE ACKNOWLEDGMENTS xi xvii Part I Introduction to Networked Multimedia Chapter 1 AN INTRODUCTION TO RTP 3 A Brief Histoiy of Audio/Video Networking 4 Early Packet Voice and Video Experiments 4 Audio and Video on the Internet 5 ITU Standards 7 Audio/Video Streaming 9 A Snapshot of RTP 9 Related Standards 10 Overview of an RTP Implementation 11 Behavior of an RTP Sender 11 Behavior of an RTP Receiver 13 Summary 14 Chapter 2 VOICE AND VIDEO COMMUNICATION OVER PACKET NETWORKS 15 TCP/IP and the OSI Reference Model 16 Performance Characteristics of an IP Network 23 Measuring IP Network Performance 25 iii
iv CONTENTS Average Packet Loss 26 Packet Loss Patterns 29 Packet Duplication 30 Packet Corruption 31 Network Transit Time 32 Acceptable Packet Sizes 37 Effects of Multicast 37 Effects of Network Technologies 39 Conclusions about Measured Characteristics... 40 Effects of Transport Protocols 42 UDP/IP 42 TCP/IP 43 Requirements for Audio/Video Transport in Packet Networks 45 Benefits of Packet-Based Audio/Video 47 Summary 48 Part II Media Transport Using RTP Chapter 3 THE REAL-TIME TRANSPORT PROTOCOL 51 Fundamental Design Philosophies of RTP 51 Application-Level Framing 52 The End-to-End Principle 53 Achieving Flexibility 54 Standard Elements of RTP 55 The RTP Specification 55 RTP Profiles 57 RTP Payload Formats 59 Optional Elements 60 Related Standards 61 Call Setup and Control 62 Session Description 63 Quality of Service 63
CONTENTS V Future Standards Development 65 Summary 66 Chapter 4 RTP DATA TRANSFER PROTOCOL 67 RTP Sessions 67 The RTP Data Transfer Packet 70 Header Elements 71 Header Extensions 85 Payload Headers 86 Payload Data 88 Packet Validation 89 Translators and Mixers 91 Translators 91 Mixers 92 Summary 94 Chapter 5 RTP CONTROL PROTOCOL 95 Components of RTCP 96 Transport of RTCP Packets 97 RTCP Packet Formats 97 RTCP RR: Receiver Reports 101 RTCP SR: Sender Reports 107 RTCP SDES: Source Description 110 RTCP BYE: Membership Control 116 RTCP APP: Application-Defined RTCP Packets.. 118 Packing Issues 118 Security and Privacy 120 Packet Validation 122 Participant Database 124 Timing Rules 128 Reporting Interval 129 Basic Transmission Rules 132 Forward Reconsideration 134
vi CONTENTS Reverse Reconsideration 138 BYE Reconsideration 139 Comments on Reconsideration 141 Common Implementation Problems 141 Summary 142 Chapter 6 MEDIA CAPTURE, PLAYOUT, AND TIMING 145 Behavior of a Sender 146 Media Capture and Compression 147 Audio Capture and Compression 147 Video Capture and Compression 150 Use of Prerecorded Content 152 Generating RTP Packets 152 Timestamps and the RTP Timing Model 154 Fragmentation 155 Payload Format-Specific Headers 157 Behavior of a Receiver 157 Packet Reception 159 Receiving Data Packets l60 Receiving Control Packets 163 The Playout Buffer 166 Basic Operation 167 Playout Time Calculation 170 Adapting the Playout Point 192 Playout Adaptation for Audio with Silence Suppression 193 Playout Adaptation for Audio without Silence Suppression 196 Playout Adaptation for Video 198 Decoding, Mixing, and Playout 200 Decoding 200 Audio Mixing 201 Audio Playout 204 Video Playout 206 Summary 207
CONTENTS VÜ Chapter 7 LIP SYNCHRONIZATION 209 Sender Behavior 211 Receiver Behavior 216 Synchronization Accuracy 222 Summary 223 Part III Robustness Chapter 8 ERROR CONCEALMENT 227 Techniques for Audio Loss Concealment 227 Measuring Audio Quality 228 Silence Substitution 229 Noise Substitution 231 Repetition 234 Other Techniques for Repairing Speech Signals.. 237 Techniques for Video Loss Concealment 242 Motion-Compensated Repetition 243 Other Techniques for Repairing Video Packet Loss 245 Dependency Reduction 245 Interleaving 247 Summary 250 Chapter 9 ERROR CORRECTION 251 Forward Error Correction 252 Parity FEC 254 Unequal Error Protection 263 Reed-Solomon Codes 266 Audio Redundancy Coding 266 Channel Coding 273 Partial Checksum 273
viii CONTENTS Reference Picture Selection 275 Retransmission 276 RTCP as a Framework for Retransmission... 277 Applicability 281 Implementation Considerations 283 At a Receiver 283 At the Sender 285 Summary 285 Chapter 10 CONGESTION CONTROL 287 The Need for Congestion Control 287 Congestion Control on the Internet 290 Implications for Multimedia 296 Congestion Control for Multimedia 298 TCP-Like Rate Control 298 TCP-Friendly Rate Control 300 Layered Coding 306 Summary 310 Part IV Advanced Topics Chapter 11 HEADER COMPRESSION 315 Introductory Concepts 316 Patterns, Robustness, and Local Implementation. 317 Standards 318 Compressed RTP 319 Operation of CRTP: Initialization and Context... 321 Operation of CRTP: Compression and Decompression 323 Effects of Packet Loss 326 Robust Header Compression 327 Operation of ROHC: States and Modes 329
CONTENTS ix Operation of ROHC: Robustness and Compression Efficiency 331 Considerations for RTP Applications 334 Summary 336 Chapter 12 MULTIPLEXING AND TUNNELING 339 The Motivation for Multiplexing 339 Tunneling Multiplexed Compressed RTP 342 Basic Concepts of TCRTP 343 Implementing TCRTP 346 Performance 347 Other Approaches to Multiplexing 349 GeRM 349 Application-Specific Multiplexing 354 Summary 357 Chapter 13 SECURITY CONSIDERATIONS 359 Privacy 36O Confidentiality 362 Confidentiality Features in the RTP Specification. 36З Confidentiality Using the Secure RTP Profile.. Зб7 Confidentiality Using IP Security 372 Other Considerations 375 Authentication 376 Authentication Using Standard RTP 377 Authentication Using the Secure RTP Profile.. 378 Authentication Using IP Security 378 Replay Protection 380 Denial of Service 381 Mixers and Translators 381 Active Content 382 Other Considerations 382 Summary 383
X CONTENTS REFERENCES 385 IETF RFC Standards 386 IETF Internet-Drafts 389 Other Standards 390 Conference and Journal Papers 391 Books 395 WebSites 396 Other References 396 INDEX 397