Unified Communications Group. Designing for Adoption: Real-time Audio in the Real World

Transcription

1 Unified Communications Group Designing for Adoption: Real-time Audio in the Real World

2 Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the companies, organizations, products, domain names, addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, address, logo, person, place, or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property Microsoft Corporation. All rights reserved. Microsoft and Active Directory are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners.

3 This document discusses media technologies used by Microsoft endpoints for Voice over IP applications. To serve numerous scenarios, the real-time media platform is employed in a diverse set of adaptive methods to provide high quality media experiences in the most hostile of environments. Architecture The architecture of the media stack was designed to allow for its use as a client or server component. To avoid complexity and duplication of functional components, it includes features that allow the stack to function as a single low complexity endpoint or a conferencing server that will support thousands of media channels. The following diagram shows an architecture of the software components organized roughly around the API abstractions and function groups. The abstraction is organized by Platform/Conference/Channel. This abstraction is used to logically separate features that may be client or server specific for the consumer of the stack. For example the conference entry point is associated with mixing which is a server function. Media Stack Feature Map Media API Platform Conference Channel Audio Video CODECS RTP Devices Endpoint Statistics UDP TCP Crossbar Video switching Dominant speaker Full mesh MP Audio Mixing Quality Score Quality Controller ICE Bitrate Control Color Converter Psycho -Visual Enhancement RMA Archive sink STUN TURN HTTP SOCKS Time Warping Jitter Buffer PLC FEC SRC NS AGC AEC VAD CNG DSOUND DSHOW Vista MicArray VMR RTAudio G SIREN G.726 G G.711 GSM CODECs RTVideo H.263 H.261 BW Estimation Multiplexing H/W abstraction SRTP DTMF 2/24/06 CIF/VGA/Pano

4 Common Components Quality Controller (QC) This module is a complex piece of software that uses all data inputs from the transport and payload layer of the stack to adapt the behavior of the endpoint to optimize the perceptible experience of the user. The QC considers a variety of parameters including: the number and types of media streams, the current estimated available bandwidth, the available negotiated codecs, and the network characteristics to optimize the experience. This optimization for the current conditions includes the selection of the highest quality codec, adjusting the nominal bit rates used and enabling redundancy in the form of FEC. Bandwidth Estimation The bandwidth estimator uses a method known as packet pair measurement to determine the effective throughput of the path that the media is traveling from source to destination. The measurement is done at the start and during the call to allow the stack to adapt to the changing network conditions. Sender Network Inter-packet gap Receiver packet #2 packet #1 packet #2 packet #1 As packet arrives, platform reads packet and timestamps it Audio General The audio engine is designed to provide high-quality wide and narrowband communications in a wide variety of devices and conditions. To do this, the audio engine employs DSP and network technologies. Packet Loss Concealment (PLC) When loss occurs across the network this module uses the information in the stream to make the loss imperceptible to the user and to identify the closest reentry point in the bitstream to resume normal speech codec operations. PLC is most beneficial for low loss rates.

5 Forward Error Connection (FEC) In conditions where the bandwidth is available and there is high packet loss in the network, the QC adds redundant information to the stream. This extra data is used to heal the stream. In general, FEC is most beneficial for high loss rates starting at 7 percent. Two forms of FEC encoding are supported by the Audio encoder. One is independent of the audio content while the second option changes the FEC structure based on the content. FEC scheme using minimum history information yields very good error resiliency at the cost of high transmission bit rate. Using partial history information still provides good error resiliency with a reduced influence on the bit rate. Acoustic Echo Cancellation (AEC) The Acoustic Echo Cancellation (AEC) module filters the audio sent to the user s speaker from the audio captured by the user s microphone. This filter prevents this echo from being sent to the other users in the call. Automatic Gain Control (AGC) This module detects the energy level and adjusts the level to provide a comfortable listening level for the user. This module also prevents clipping of the audio from the user s microphone. Voice Activity Detection (VAD) / Silence Suppression (SS) The Voice Activity Detection (VAD) module detects the presence or absence of speech from the user s microphone. This data is used by the Silence Suppression (SS) module to stop packets from being sent over the network if the user is not talking. VAD/SS reduces the total number of packets sent during a call which results in the use of less bandwidth. Noise Suppression (NS) Noise suppression isolates and removes non-speech signals from the signal captured by the microphone to create a cleaner rendition of the speech. It has the side effect of decreasing the amount of data that is being compressed and sent which saves on transmission bandwidth needed for speech signals. Comfort Noise Generation (CNG) The module adds artificially-generated noise during periods of silence to give the user the sense that the channel is still active.

6 Audio Codec Support The media stack supports a wide range of speech compression/decompression schemes (codecs). The codecs and the modes that they are supported in are listed in the following table. Audio Codec MSRTAudio@8kHz MSRTAudio@16kHz G SIREN G G.726 (AAL2) G.711 A-Law G.711 Mu-Law RTAudio Codec MSRT Audio stands for Microsoft Real-time Audio. It is a speech codec designed for real-time two-way Voice over IP (VoIP) applications. The encoder is capable of encoding single-channel (mono), 16 bit per sample audio signals. The encoder can be configured to run in fixed bit rate mode or variable bit rate mode. The decoder has a built-in Jitter-Control module, or can be used like traditional direct-mode decoders. The codec is multi-rate and the bit rate is driven by the logic of the quality controller (QC). Audio over TCP In some network deployments it not possible for User Datagram Protocol (UDP) traffic to be sent between the endpoints even if a media relay is used. In these cases, there is a need for real-time audio to be sent using Transmission Control Protocol (TCP) as the transport instead of UDP. TCP has loss recovery and windowing mechanisms which sometime delays the delivery of the data by several roundtrips. For real-time applications this sort of delay sometimes creates an unacceptable experience for the user. Due to the nature of TCP, special tuning is done to ensure a good experience using this alternate transport.