5.18. Audio for Digital Television. Digital Audio

Transcription

1 C H A P T E R 5.18 Audio for Digital Television TIM CARROLL Linear Acoustic Lancaster, Pennsylvania JEFFREY RIEDMILLER Dolby Laboratories San Francisco, California INTRODUCTION Audio is an area in television broadcasting and production that causes apprehension with some engineers because of its complexity and the care needed to get it right in design, installation, and operations. With the transition to digital television (DTV), there is a new level of uncertainty because the requirements for audio are far more advanced and complex than before. Yet digital audio provides significantly more sophistication, artistic creativity, greater processing power, and higher quality than its predecessor. This chapter provides a general overview of the key features and requirements of audio for DTV, a detailed review of one of the largest problems today (loudness), and some real-world suggestions on making it all work. By no means an exhaustive discussion of what is involved, the material here will at least bring to light what must be done, and a view of what is possible with television and digital audio beyond just getting a signal on-air. TELEVISION AUDIO SYSTEM OVERVIEW Analog Audio Television audio has been, until quite recently, a largely analog medium. It began as a single monaural frequency-modulated carrier at 4.5 MHz in the 6 MHz TV channel, as defined by the NTSC specifications. Stereo was superimposed on this carrier by the BTSC standard in much the same way as stereo was introduced to FM radio (see Chapter 6.3). However, while most FM radio stations transmit stereo, only about half of all television stations in the United States have implemented stereo transmission. Compared to methods of delivering audio to consumers by the many digital means that are available, even the most carefully aligned BTSC system is not comparable in quality. Some of the issues include the side effects of high-frequency preemphasis (far less than FM radio, but still present), somewhat limited frequency response, low channel separation, and elevated levels of noise and distortion. For subcarrier channels that carry auxiliary programs, such as second audio program (SAP) or descriptive video servers (DVS), the comparison is worse as these are monaural channels, limited to 7.5 khz frequency response, and noisy due to limits on modulation. Digital Audio When the process for standardizing a digital television system began, one of the requirements was that the audio system should be enhanced well beyond the capabilities of the BTSC system. It was decided that stereo, then later multichannel, audio would be delivered digitally. To conserve data bandwidth, audio data rate reduction, or data compression, would be employed, with the objective of maintaining performance that was as measurably and audibly as close to the original as practical. Multichannel Sound The first actions toward achieving this goal focused on MPEG-compressed delivery of two-channel audio. The need for supporting multichannel surround NAB ENGINEERING HANDBOOK Copyright 2007 Academic Press. All rights of reproduction in any form reserved. 309

2 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY sound was introduced because a substantial quantity of theatrical motion pictures had been mixed with four or more channels of audio: left front, right front, center, low-frequency effects (for driving the subwoofer), left surround, and right surround. With 70 mm films, the magnetic striped soundtracks were capable of carrying all of these channels, while 35 mm used matrix techniques such as Dolby Pro Logic to level- and phase-encode them into the standardized stereo variable area soundtrack. Matrix techniques were briefly also considered for digital television, as MPEG could simply carry the audio in the same way the stereo variable area (SVA) track does for 35 mm film. This approach was not pursued, as Dolby Laboratories introduced a new system called AC-3 (Audio Coder 3), also now called Dolby Digital, which delivered nonmatrixed 5.1 channels of audio via a composite bit stream that fit into precisely the same space as the stereo MPEG audio. Today, Dolby Digital (AC-3) is a part of the ATSC standard, the European DVB standard, the Open Cable standard, and various DVD standards. AC-3 Audio ATSC standard A/52B 1 describes in detail the AC-3 audio coding system, and readers are encouraged to download a copy for reference purposes. This document also describes E-AC-3, the enhanced version of the coder for use with E-VSB (enhanced VSB) mode of the ATSC system. Enhanced AC-3 offers increased efficiency, but broadcasters should be aware that transmission of a standard AC-3 stream is still required to maintain compatibility with the large installed base of receivers. AC-3 is an efficient audio coding system capable of carrying from 1 to 5.1 channels of audio at bit rates from 32 to 640 kbps with high quality 5.1 audio typically coded at data rates of 384 to 448 kbps (the maximum rate for ATSC emission). The sample rate for television uses is 48 khz, and the system is capable of 24-bit audio resolution. Importantly, the AC-3 stream carries a parallel audio control data path called metadata (described in detail below), which provides additional important features used by decoders to optimize reproduction of the audio program for a given receiver in a given listening environment. The system is designed such that a single encoded bit stream can be reproduced by any consumer decoder, regardless of whether the source content is encoded as 5.1 channels or as mono, and regardless of whether the decoder is capable of reproducing 5.1 channels or only a single channel. Programs of any number of channels can be reproduced by any AC-3 decoder. Figure shows an example of how a single bit stream can serve all types of decoders. Note that multichannel decoders also contain a matrix decoder of some type (usually Pro Logic or Pro Logic II) to handle surround sound content that is carried in two channels. This audio will be matrix decoded using level and phase information and will Available at FIGURE Single bit stream delivered to many different types of decoders; any bit stream can be decoded by any decoder under the guidance of metadata. then have the capability of providing audio to all speakers when appropriate. Unless care is taken, it is possible to cause audible clicks or pops when transitioning between 5.1- and two-channel material. A local television station will need at least one AC-3 encoder per audio program. These encoders are often integrated into a full ATSC encoder, which, in addition to audio encoding, contains video encoding and multiplexing capabilities. Until recently, most of these encoders contained one or more two-channel AC-3 encoders, but for 5.1-channel programming an external encoder was required. Newer encoders may also contain 5.1-channel encoding. Note that it is likely that several audio encoders will be required for each video program in order to carry all relevant program audio. Audio that is transmitted using the ATSC or Open Cable standards must be AC-3 encoded. Uncompressed audio is fed to the AC-3 encoder and an audio packetized elementary stream (PES) results. It is this PES data that will be multiplexed with video and other data into an MPEG-compliant transport stream for transmission to consumers. AUDIO METADATA An essential part of digital television audio is its metadata, or data about the audio data. Metadata conveys information such as the loudness of a program, how many channels have been encoded, how to downmix those channels if the bit stream is decoded by a two-channel decoder, and dynamic range control values to help match the audio to the listening environment. Dialog Loudness Based on research performed by Bell Laboratories, the BBC, Dolby Laboratories, and others, it has been determined that dialog provides the most common

3 CHAPTER 5.18: AUDIO FOR DIGITAL TELEVISION Digital Full Scale 0 dbfs -10 dbfs -20 dbfs -30 dbfs -40 dbfs ORIGINAL SIGNAL DIALNORM VALUE DIALOGUE LOUDNESS Digital Full Scale 0 dbfs -10 dbfs -20 dbfs -30 dbfs -40 dbfs SIGNAL WITH DIALOG NORMALIZATION PROGRAM LEVEL SHIFTED -11 db DIALOG LOUDNESS AT -31 dbfs -50 dbfs AVERAGE DIALOG FIGURE SIGNAL PEAKS (a) -50 dbfs AVERAGE DIALOG (a) Dialog loudness measurement and (b) application to signal. SIGNAL PEAKS (b) loudness anchor of a program. It is what most listeners will use to judge the relative loudness of one program versus another. The dialog loudness (also known as dialnorm) metadata parameter is used to indicate the long-term, A-weighted loudness of a given program with respect to 0 dbfs (full scale). Each program requires its own measurement and the assignment of a unique dialog loudness value. This value directly controls a 1 db per-step attenuator present in all AC-3 decoders, allowing programs to be scaled to the internal target for AC-3 of 31 dbfs. Figure shows how a typical program would be analyzed and dialog loudness parameter determined, and the results of applying this value after decoding. The usefulness of this approach is illustrated in Figure (a), where multiple programs having different dialog loudness values are applied to the encoder, and upon decoding, are then scaled by the proper amount at shown in Figure (b). It should be immediately apparent that while the average dialog loudness of each program in the figure is matched, the signal peaks have not been affected. Audio metadata delivers the unique ability to separate loudness matching from dynamic range control. This means that it is possible to more closely preserve content as originally produced, and leads to a discussion of the requirement for some sort of dynamic range control that supports this concept. Dynamic Range Control Television audio has traditionally kept dynamic range tightly controlled to ensure that loudness is consistent and that programs are intelligible in as many listening environments as possible. Unfortunately, this has meant that processing has been adjusted for the lowest common denominator, the 3 inch speaker on the side or front of a TV set, to the detriment of larger, higherquality reproduction systems. With digital audio, loudness can now be matched without the need for severely impacting a program s dynamic range, and it should be apparent that far less dynamic range control is necessary and it can be different than the traditional approach. Audio metadata contains Dynamic Range control (DRC) gain words that can be generated by the AC-3 encoder and applied to the audio at the time of decoding the original audio is not affected. Many Dolby Digital decoders offer the consumer the option of defeating the DRC metadata, but some do not. Decoders with six discrete channel outputs (full 5.1-channel capability) typically offer this option. Decoders with stereo, mono, or RF-remodulated outputs, such as those found on DVD players and set-top boxes, often do not. In these cases, the decoder automatically applies the DRC metadata associated with the decoder s selected operating mode. 0 dbfs Action Movie Drama Sports Sitcom Talkshow News 0 dbfs Action Movie Drama Sports Sitcom Talkshow News -10 dbfs dbfs -20 dbfs -30 dbfs dbfs -30 dbfs -40 dbfs -40 dbfs AVERAGE DIALOG SIGNAL PEAKS (a) AVERAGE DIALOG FIGURE (a) Dialog level assignments for each program prior to encoding and transmission and (b) results after reception and decoding. SIGNAL PEAKS (b) 311

4 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY The DRC parameters that are part of the AC-3 bit stream are called Line Mode and RF Mode, also known as dynrng and compr. Line Mode has ±24 db of range in 0.25 db steps, while RF Mode has ±48 db of range in 0.5 db steps. These modes are described in greater detail in the Loudness section below. The system relies completely on the dialog loudness parameter being measured and set correctly or inappropriate DRC values will be generated, which will result in level control problems. Also, while the control loop used for generating these values is frequency weighted, the gain control is wideband and is applied to all channels simultaneously. This places practical limits on the degree to which dynamic range can be controlled without causing objectionable audible artifacts. Figure illustrates the transfer function for the DRC subsystem of Dolby Digital (AC-3). It shows that although it is constrained by the fact that is it wideband, the sophisticated structure of the compressor can yield reasonable results, again assuming that the dialog level parameter has been correctly set. Due to inherent natural delays in the audio encoding process, look-ahead processing is possible that further improves the audible performance of the DRC system. No knowledge of transfer functions is required for users to generate dynamic range control parameters. Five presets available in all Dolby Digital (AC-3) encoders make the task straightforward. The available selections and corresponding key differences are listed below. Film Light Max Boost: 6 db (below 53 db) Boost Range: 53 db to 41 db (2:1 ratio) Null Zone Width: 20 db Early Range: 26 db to 11 db (2:1 ratio) Cut Range: 11 db to +4 db (20:1 ratio) Film Standard (Default) Max Boost: 6 db (below 43 db) Boost Range: 43 db to 31 db (2:1 ratio) Null Zone Width: 5 db Early Range: 26 db to 16 db (2:1 ratio) Cut Range: 16 db to +4 db (20:1 ratio) Music Light Max Boost: 12 db (below 65 db) Boost Range: 65 db to 41 db (2:1 ratio) Null Zone Width: 20 db Early Range: None Cut Range: 21 db to +9 db (2:1 ratio) Music Standard Max Boost: 12 db (below 53 db) Boost Range: 55 db to 31 db (2:1 ratio) Null Zone Width: 5 db Early Range: 26 db to 16 db (2:1 ratio) Cut Range: 16 db to +4 db (20:1 ratio) 312 FIGURE Transfer function of the DRC system in Dolby Digital (AC-3). Note that the minimal action null zone is roughly centered around the dialog level. Speech Max Boost: 15 db (below 50 db) Boost Range: 50 db to 31 db (5:1 ratio) Null Zone Width: 5 db ( 31 db to 26 db) Early Range: 26 db to 16 db (2:1 ratio) Cut Range: 16 db to +4 db (20:1 ratio) The default selection is Film Standard, and is applicable to most programming commonly found on television. There is also the capability for selecting a preset that will not generate any DRC values. This selection is appropriately called None, but note that while it will not act under most circumstances, it will do so in cases where dialog level has been set incorrectly and downmixing the audio channels in a decoder could cause overload. This so-called Protection DRC is automatic and cannot be bypassed. For example, consider a 5.1-channel program with signals at digital full scale on all channels being played through a stereo, downmixed line-level output. Without some form of attenuation or limiting, the output signal would obviously clip. Correct setting of the dialog level and DRC profiles normally prevents clipping and unnecessary application of overload protection. It is good engineering practice to avoid the None setting. Downmixing In order to handle any encoded bit stream, decoders must have some facility to match the decoded audio to the number of output channels available. This is particularly important in cases where the number of output channels is less than the number of encoded audio channels. Downmixing is the technique that makes this possible. Note that the LFE (low-frequency effects) channel is not included in any downmix but instead discarded.

5 CHAPTER 5.18: AUDIO FOR DIGITAL TELEVISION Decoders generally exist in two forms: two-channel and 5.1-channel. This means that the two-channel decoder must then be able to downmix 5.1 channels to two, and this is further subdivided into two types. The first is for a standard stereo output and is called Lo/ Ro (Left only/right only) and is created by the following formula: Lo = Lf + C*cmixlev + Ls*surmixlev Ro = Rf + C*cmixlev + Ls*surmixlev where Lf/Rf are left and right front channels, C is the center channel, and Ls/Rs are the left and right surround channels. Note the metadata parameters cmixlev and surmixlev, which determine the contribution of center and surround channels. The default setting is ( 3 db), but can be adjusted to 4.5 db or 6 db in all decoders, and in the newest decoders can also be adjusted to off and 0 db. The second type of downmix is by far the most common found in the field as it produces a matrix surround compatible output. The resulting signal is fully appropriate for matrix decoding by systems such as Dolby Pro Logic, Pro Logic II, DTS Neo:6, and others. Note the differences in the following formula, where Lt/Rt represent left total/right total: Lt = Lf + C*cmixlev (Ls+Rs)*surmixlev Rt = Rf + C*cmixlev + (Ls+Rs)*surmixlev This formula is important. Note that channels are being added and subtracted and, therefore, can cause cancellations and other undesired effects. For example, it would be improper to place dialog equally in the center and surround channels, as upon downmix it would cancel almost completely in the Lt signal. Proper downmixing relies on a 90-degree phase shift being applied to the left and right surround channels by the Dolby Digital (AC-3) encoder. Normally, it is enabled by default and it is necessary that it remains that way. It will produce little or no audible effect in a 5.1-channel environment, but is required for successful downmixing. PROGRAM PRODUCTION Now that the capabilities of the system have been defined, program producers can begin supplying audio content that will make it intact all the way to consumers. This is substantially true if certain rules that should be common to all delivery specifications (if not present already) are applied. Important aspects are monitoring, channel layout, synchronization (that is, lip sync), and audio upconversion also known as upmixing. Monitoring 2 ITU-R BT FIGURE ITU-R BT recommendation for multichannel speaker setup. Proper configuration of 5.1-channel monitoring systems has been well defined by the ITU, 2 and a representative speaker layout is shown in Figure The subwoofer is generally placed on the floor near the front channel speakers. Although beyond the scope of this book, room alignment is critical for creating a reference monitoring environment and a brief overview of the measurement and adjustment procedure will be presented. 3 It is impossible to know if a problem truly exists with the audio if the monitoring system is uncalibrated. Alignment is best done with a real-time analyzer (RTA) and calibrated microphone, but can also be accomplished with an inexpensive sound pressure level (SPL) meter. A suitable digital instrument is available from Radio Shack (see and an analog version from ATI (see Meters from other manufacturers can of course also be used. All channels should have an individual speaker. While some monitor systems allow for so-called phantom center operation, this is not an ideal scenario. In some cases, such as remote or outside broadcast vans, it cannot be avoided, but every effort should be made to have one speaker per channel, including a subwoofer for the LFE channel. Once the speakers have been physically aligned, electrical alignment is next. Set the real-time analyzer or SPL meter to apply a C-weighting curve and a slow response. A reference listening level of 79 db/c/slow 3 An excellent reference called 5.1 Channel Production Guidelines is available for download from and provides a comprehensive discussion of monitor setup and calibration. 313

6 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY Track 4: Low frequency effects (LFE) Track 5: Left surround (Ls) Track 6: Right surround Rs) Track 7: Left total or Left only (Lt or Lo) Track 8: Right total or Right only (Rt or Ro) It is helpful if the channels are specified in this order. However, other channel configurations exist in film and music, so a facility and its operators must be prepared to shuffle individual channels. FIGURE Real-time analyzer display of pink noise as reproduced by the center and subwoofer channels. is recommended as it most accurately matches the average listening level for most viewers. Generate pink noise at the reference level as shown on the meters of a console or other metering device used during mixing (i.e., set the level of the pink noise so that it averages around 0 VU). Route the pink noise to the center channel and adjust the monitor volume control on the console or the monitor controller until the SPL meter reads 79 db/c/slow. Mark the position of this volume control and keep it there for the remainder of the calibration. Apply pink noise individually to each of the remaining speakers except for the subwoofer and trim their gain (not the master gain that is set at reference) until each reads 79 db/c/slow. When pink noise is panned to each speaker, they should all reproduce at the same level of 79 db/c/slow with no need to adjust any levels. The subwoofer requires a slightly different alignment, as it needs an additional 10 db of gain as compared to the other main channels in order to match consumer reproduction standards. Figure shows how this would look on an RTA. Note that the figure shows the response of a typical film mixing stage. Due to the large distance from the speakers to the mixer, there is a natural roll-off above 2 khz (described in SMPTE 202M, and sometimes called the X-curve ). In small mixing and control rooms, this response would be flat. The important point of this figure is to show that the subwoofer has 10 db of additional gain as compared to the reference center channel. The net result of this extra subwoofer gain is that it will cause operators to mix the sounds to the subwoofer channel 10 db quieter. When this audio then reaches the consumer system, the subwoofer is again boosted by 10 db and the net of the process is unity. Channel Configuration SMPTE 320M specifies the following track layout for multichannel audio media: Track 1: Left front (L) Track 2: Right front (R) Track 3: Center (C) 314 DISTRIBUTION While ATSC specifications fully define how to send audio from the broadcaster to the consumer, an area subject to wide variability is network distribution, or routing the signal from the program distributor to the broadcaster. There are several approaches that can be used, and a logical combination of different techniques may yield the best results. Baseband Methodology Keeping audio in the noncompressed PCM domain allows straightforward access to the audio and perhaps easier control of audio/video synchronization. Audio can either be separate AES pairs or can be embedded along with video in an SD or HD SDI serial digital interface stream. 4 Note that a separate timealigned path for audio metadata storage and routing is required for baseband systems. One audio metadata distribution approach inserts it, among other things, into the vertical ancillary data (VANC) space of the SDI or HD-SDI signal. 5 Done properly, this method allows for video, audio, and all associated metadata to be stored, routed, and switched within one signal path, and synchronization between the audio and its metadata can be accurately maintained. Metadata is video synchronous so as to provide data gaps at video frame boundaries. This allows switching to take place without interruption of the metadata stream. When storing metadata in VANC, it is important to maintain video synchronization. Asynchronous insertion of metadata may cause problems at switch points that will be heard by consumers. Information about the structure of audio metadata is available from Dolby Laboratories. Compression Methodology Using audio compression systems, such as Dolby E 6 or the Linear Acoustic StreamStacker-HD e 2 system, 7 4 Described in SMPTE 259M/272M (SD) and 292M/299M (HD). 5 Described in SMPTE 334M. 6 Dolby E carries up to eight channels of PCM audio and metadata over a single 48 khz/20-bit AES pair (see 7 The e 2 format generated by StreamStacker-HD carries up to 16 channels of audio, metadata, and auxiliary data over a single 48 khz/ 20-bit AES pair or via TCP/IP (see

7 CHAPTER 5.18: AUDIO FOR DIGITAL TELEVISION allows multiple channels of audio to be carried along with metadata over a single bit-accurate path. Noting that the status bits of the AES frame are often not preserved in the recording process, the data rate is confined to the audio and aux portions of the frame, equaling approximately 1.92 Mbps. Although compressed systems impart some amount of latency during decoding, they also guarantee that audio and metadata remain tightly synchronized. Any VTR or server capable of storing uncompressed 20-bit 48 khz AES audio can carry these lightly compressed mezzanine compression formats. When dealing with compressed audio, it is important to choose equipment that has known timing and performance characteristics. While Dolby E is video frame-based, different equipment can cause timing shifts. Just like video, this timing must be known when designing a system. Dolby Laboratories tests and certifies equipment through its Dolby E partner program and maintains a list of those models on their website. StreamStacker-HD is timed to AES reference and is not video frame rate dependent. Transport Stream Several U.S. terrestrial broadcast networks have chosen to send a precompressed, ready-to-air ATSC transport stream to their affiliate or member stations. This was done initially as a cost-saving method in the early days of DTV, which allowed local stations to simply feed the transport stream direct to the transmitter, getting them on the air with no need to purchase an expensive local ATSC encoder. Of course, this meant a lack of local programming or branding, but it did help stations obey FCC rules while simultaneously getting SD and HD content on the air. A local station has two choices when local content must be intercut with a transport stream from the network: decode to baseband audio and video or splice in the compressed domain. Decoding to baseband audio and video produces signals that can be routed and switched with locally generated audio and video signals, then re-encoded for transmission. However, this results in some quality degradation due to the decodeencode cycle. For this reason the preferred method is usually splicing the transport stream, in which case original quality can then be maintained all the way to the consumer. Technologies now available for processing and splicing transport streams have overcome most of the early drawbacks of transport stream distribution and allow many operations, such as local logo insertion, to be performed in the compressed domain. They also allow the output of local ATSC encoders to be seamlessly spliced in place of the network transport stream. It is also possible to apply splicing techniques directly to audio that has been Dolby Digital (AC-3) encoded. The benefit of allowing preencoded content to pass through a facility and straight to transmission to the consumer is that it eliminates all other coding steps, and the ultimate quality is maintained with no need for local personnel to worry about metadata or other settings. This is useful also in situations that require the video to be decoded to baseband for local processing and logo insertion through traditional means. Using AC-3 bit stream splicing allows the encoded audio to pass through to consumers except when local content is switched in. The AC-3 bit stream is particularly well suited to this type of operation and responds well to splicing so long as timing constraints are obeyed. Techniques and products to support all types of local station needs, such as voiceover and program insertion, exist and are successfully being employed by many stations. AUDIO/VIDEO SYNCHRONIZATION Audio Synchronization Issues Most film editors are able to detect audio/visual (A/V) sync errors as short as ±1/2 film frame. At 24 fps this equates to approximately ±20 m. It is claimed that some editors can detect even smaller errors, but this might be more accurately attributed to their familiarity with the material being viewed. Other figures for A/V sync include ±1 video frame or ±33 40 m. Dolby Laboratories specifies that any Dolby Digital decoder must be within the range of +5 m audio leading video to 15 m audio lagging video. This is because human perception of A/V sync is weighted more in one direction than the other due to our experience in the real world. Light travels much faster than sound. For example, the sound of a baseball bat hitting the ball in a large stadium would appear relatively in sync to a viewer sitting in the first few rows of seats, but the further back a viewer gets, the more the sound lags behind the sight of the ball being hit. Because we are used to this common phenomenon, except in extreme cases over very long distances, it does not seem to be wrong. Now, imagine if the audio/video timing was reversed. If, while watching a baseball game, the sound of the bat hitting the ball arrives before the bat looks like it makes contact. This would be an unnatural sight and would seem incorrect even to those in the first few rows. The point is that the error is in the wrong direction. In summary, human perception is much more forgiving for sound lagging behind sight, probably because this is what we are used to naturally observing. The International Telecommunications Union (ITU) released ITU-R BT in It was based on research that showed the reliable detection of A/V sync errors fell between 45 m audio leading video and 125 m audio lagging behind video. That was just for detection, while the acceptability region defined by ITU, and therefore the recommended maximum, was quite a bit wider. In summary, the recommendation states that the tolerance from the point of capture to the viewer and or listener shall be no more than 90 m audio leading video to 185 m audio lagging behind video. This range is probably far too wide for truly acceptable performance. More recently, the ATSC has published documentation 8 stating a goal of 15 m audio 315

8 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY leading to 45 m audio lagging at the input to the DTV encoding devices. The ATSC document acknowledges that there are various sources of different delay throughout the broadcast system. It recommends that designers should correct these delays at each step in the chain and strive for 0 m offset as the goal for every step. Delays in the Television Plant While A/V sync issues within the TV plant are not new to digital television, they have become more noticeable in recent years. Some basic points to keep in mind are that in general, audio operations are very low latency. Compression, equalization, mixing, and processing can typically be accomplished in under 1 m in the digital domain, falling to microseconds in the analog domain. Generally, no compensating video delay needs to be added for these operations, as the latency is so low. Video processing, on the other hand, takes substantial amounts of time, usually no less than one video frame. Similar to audio, any time a video signal is digitized, operations upon that signal will take longer. As most video effects are unable to be performed in the analog domain, delay is inevitable. It is interesting to note that processing delay of audio and video signals has the opposite of the desired effect on each. As video processing takes longer, the video signals will be delayed with respect to the audio signals and A/V sync will seem incorrect much sooner. It is important that compensation is provided for any video device that has a delay in excess of a few milliseconds. An equal amount of delay should therefore be applied to the audio path. The ITU recommends in ITU-R BT.1377 that audio and video apparatus should be labeled to indicate the amount of processing delay. This delay should be indicated in milliseconds to avoid any frame rate discrepancies, and if the delay is variable the range should be stated. In the case of variable delay, a signal that can control an audio delay should also be provided. By following these recommendations, it is apparent that regardless of the actual delay, compensation can be made and A/V sync ensured. Some typical operations are presented below. Two are common to any video facility and have simple, logical solutions. The third is a somewhat surprising source of sync error that should be taken into account during facility design and troubleshooting. Video Frame Synchronizers A video frame synchronizer causes between one and two variable frames of delay. In this case, a special audio delay, able to track the variable delay of the frame synchronizer, is required. Most video frame synchronizers are available with matching and tracking audio delays and should always be purchased as a set as there is currently no standard interface that represents A/V sync values. 8 ATSC IS Finding IS-191, available at Digital Video Effects Digital video effects (DVE) can add from one to many video frames of delay. As the delay of a DVE is generally a fixed value, a fixed value audio delay can also be used. Devices with fixed delays are easier to deal with if they are always kept in line, or if they must be removed then a fixed video delay equal to that of the device is inserted in its place. This will prevent having to dynamically adjust audio delay and create an audible disturbance. Cameras and Displays Traditional tube-based television cameras generally have no intrinsic delay due to video processing because the timing of the video signal out of the camera is directly synchronized with the physical scanning of the image by the pickup tube. Because of the process in which the picture is scanned from top to bottom over a one-frame (or two-field) interval, there is some small variability of A/V timing depending on where the source of the sound is positioned in the video frame. Similarly, for traditional cathode ray tube (CRT) monitor displays or televisions, the displayed picture is directly synchronized with the input video, with no intrinsic delay. Also, because the CRT display will usually replicate the scan of the original camera image, the small variability due to the sound source position in the frame is removed. Modern cameras using solid-state image sensors and picture displays based on new technologies (such as LCD, plasma, and micro-mirror) are fundamentally different from scanned tubes, being based on pickup and display of complete images one whole frame at a time. These processes mean that the devices all have significant internal storage and processing delays. The video signal out of, or into, such devices is, therefore, no longer synchronous with the image being viewed by the camera or shown on the display. Delays will be different, depending on the particular technology and processing, and may be of the order of one frame up to several frames. Professional picture monitors for quality control purposes may also be used with external video processors that introduce further video delay. These video delays should be taken into account when designing or troubleshooting a television plant. Consumer television manufacturers and system designers also need to take them into account so that signals transmitted by broadcasters are correctly presented to viewers. A/V Sync in the MPEG-2 System The MPEG system provides the proper tools to make A/V sync correct through the transmission system. Each audio and video frame has a presentation time stamp (PTS) that allows the decoder to reconstruct the sound and picture in sync. These PTS values are assigned by the multiplexer in the MPEG encoder. The decoder receives the audio and video data ahead of the PTS values and can therefore use these values to

9 CHAPTER 5.18: AUDIO FOR DIGITAL TELEVISION SMPTE Timecode Video MPEG-2 Video Encoder Multiplexer Transport Stream Audio (One to 5.1 Ch) SMPTE Timecode Dolby Digital (AC-3) Audio Encoder (Int. or Ext.) FIGURE Simplified MPEG-2/Dolby Digital (AC-3) encoding system block diagram. properly present audio and video in sync. Figure shows a simplified block diagram of a typical MPEG-2 video encoder, Dolby Digital (AC-3) audio encoder, and a multiplexer. Note that the Dolby Digital (AC-3) encoder can be either internal or external to the video encoder and multiplexer. Aligning the MPEG-2 Encoding System Video and audio encoding take some time to accomplish, and the multiplexer must know exactly how long. This delay depends on the manufacturer of the equipment, but the value is necessary for the PTS values to be correctly assigned. Many of the A/V sync problems encountered in the field can be attributed to these delays not being properly accounted for or just not set at all. In practical terms, this simply means that if the transmission system uses an external Dolby Digital (AC-3) encoder, there is a known, fixed audio encoding latency that must be entered into the MPEG-2 encoding system. There is usually a setting called MPEG-2 Encoder Audio Delay, or possibly AC-3 Delay. Once set, it need not be changed unless either the audio or video encoder latency is reset. In many cases, SMPTE timecode can be applied to the audio and video encoders and can be used by the multiplexer to calculate exact PTS values, thereby removing encoder delay as a source of error. Testing the MPEG-2 Encoding System In its simplest terms, testing entails feeding typical A/V sync test material such as beep/flash (audio pip with simultaneous video flash) to the encoder, capturing the resulting transport stream from the multiplexer, and using analysis software to determine compliance. It is best to avoid using a consumer settop box to verify the performance of the MPEG-2 encoding system. There are commercially available tools that make the measurement easier and more accurate by directly analyzing the MPEG-2 transport stream. The latency of the Dolby Digital (AC-3) encoding algorithm is fixed regardless of the number of encoded audio channels. Therefore, a two-channel signal is adequate to test the A/V sync relationship of an MPEG-2 encoded video signal and a Dolby Digital (AC-3) encoded audio signal. This means that a test tape can be easily created with a video flash and an audio beep, verified with an oscilloscope, and used as the source for A/V sync testing. Testing the MPEG-2 Decoder Testing the MPEG-2 decoder is a straightforward process. It requires that a reference transport stream be applied to the decoder under test. This transport stream is a beep/flash-type signal that has been encoded and verified for proper synchronization as described above. The audio and video outputs are then displayed on a dual-trace oscilloscope and compared as shown in Figure It is necessary to perform this testing with different video scanning formats and at different frame rates. This is due to the use of video format converters after the MPEG-2 decoder that may respond differently to native rates than to rates that must be converted. It is also important to test the decoder response to bit stream discontinuities as errors and splices are handled differently from one decoder to the next. LOUDNESS This section briefly discusses and provides a highlevel overview of loudness estimation as it applies to digital broadcasting. It by no means attempts to give the reader a full treatise on this subject. There has been, and continues to be, extensive research and development in the area of loudness estimation and 317

10 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY Audio "Beep" Video "Flash" FIGURE Audio and video outputs of a decoder under test as displayed on a dual-trace oscilloscope. In the display, audio and video are exactly in sync. Note the ramp-up and ramp-down of the audio beep. This is due to the windowing function present in the Dolby Digital (AC-3) process. The measurement point is after the windowing, or when audio reaches maximum. devices to measure this subjective quantity. The reader is encouraged to follow the publications of several professional societies and standards bodies dedicated to work in this area, including the Audio Engineering Society (AES), Acoustical Society of America (ASA), and standards of the International Telecommunications Union (ITU). It should be noted that the following sections on loudness and the digital set-top box refer to Dolby Digital (AC-3) bit streams. However, all of the principles and background also apply to Dolby Digital Plus (E AC-3) bit streams. Background The term loudness is generally defined to be the attribute of auditory sensation in which sounds can be placed on some scale extending from quiet to loud (corresponding to a ratio of intensities of 1,000,000,000,000:1). Loudness itself is also a highly subjective quantity (and as such, cannot be measured directly) that involves psychoacoustic, physiological, and other factors still under investigation. Hence, this highly subjective quantity (loudness) often results in substantial differences in loudness perception between listeners, making a single measurement method that considers all of the above factors for all individuals a complex problem. This is proven by real-world experience, as there is often no single loudness level that will satisfy all listeners (or even a single listener) all of the time. At best, the loudness of sounds can only be approximated by artificial means. One study performed by Dolby Laboratories concluded that even when audio programming has been normalized by a group of people by ear, the normalized programs do not completely satisfy a different group of listeners 100% 318 INPUT A INPUT B Sync Point of the time. In fact, the different groups only agreed approximately 86% of the time. Given this level of uncertainty among groups of listeners, any loudness measure utilized in broadcast will not guarantee that we satisfy 100% of the listening audience. Loudness Perception A brief recap of what is known about the science behind loudness perception is in order at this point. First, the human auditory system is nonlinear with respect to frequency. Thus, perceived loudness is dependent on the frequency content of a sound. For example, a person with normal hearing would perceive a very low-frequency sound, such as a 20 Hz tone at 40 db SPL, to be quieter than a 1 khz tone at 40 db SPL. If this process is repeated for various frequencies (with the 1 khz tone still fixed at 40 db SPL), a 40 phon equal-loudness contour is created. (The term phon is defined as a unit of loudness level. For example, if a given sound is perceived to be as loud as a 40 db SPL sound at 1,000 Hz, then it is said to have a loudness of 40 phons.) Readers may also be familiar with the equal-loudness contours that were first developed by Fletcher and Munson in 1933; approximations of these contours have been utilized in sound level meters for several years and are commonly referred to as frequency weighting networks. In such a network, the intensity of each frequency is weighted according to the shape of the equal-loudness contour and for a particular loudness level in phons (e.g., A-weighting approximates the sensitivity of human hearing similar to the 30 phon loudness contour) before summing the energy across the entire frequency range, and devices of this type perform quite well at estimating the relative loudness of signals with similar spectra, such as dialog. However, calculating the loudness of more complex groupings of sounds (sounds with heterogeneous spectra) requires further thought, as something called the critical bandwidth comes into the picture. Critical bandwidth is a measure of the frequency resolution of the ear. For example, when two sounds of equal loudness, when sounded separately and are close together in pitch (narrowband), their combined loudness when sounded together will be perceived as only slightly louder than one of them alone. Hence, they are probably in the same critical band where they are competing for the same nerve endings on the basilar membrane of the inner ear. However, if the two sounds are widely separated in pitch (wideband), the perceived loudness of the combined tones will be considerably greater because they do not compete for the same nerve endings. Third-octave frequency bands can, and have been, used as an approximation to the critical bands in some standardized methods of calculating loudness (namely ISO Method B). As a side note, the critical band is about 90 Hz wide below 200 Hz, and increases to approximately 900 Hz for frequencies around 5 khz [1].

11 CHAPTER 5.18: AUDIO FOR DIGITAL TELEVISION Because loudness perception is partially dependent on whether the signal is wideband or narrowband, it is (and has been) challenging to design a measurement system that can detect, and subsequently apply a specific loudness measurement function for, each of these signal types on a continuous basis. Given this brief overview, it is understandable how the development of a measurement system that factors in even these few characteristics of human hearing (as well as numerous others not described here) would be quite complex, and yet it still wouldn t provide a measurement method perfect for every individual. Reference/Line-Up Levels There have been several attempts to standardize a common analog and digital reference level that is intended to facilitate the seamless exchange of programming among broadcasters or other program users. In the United States, SMPTE RP155 defines a reference or line-up level of 20 dbfs (at 1 khz) to be used for the calibration of audio level indicators and to be recorded on digital VTRs. It is common to find (but not always) that this 20 dbfs reference level represents +4 dbu in the analog domain. In contrast to RP155, EBU R68 defines a reference or line-up level of 18 dbfs (at 1 khz) that often represents 0 dbu in the analog domain. Thus, 0 dbfs is equivalent to +18 dbu analog within facilities that follow EBU R68, but 0 dbfs is equivalent to +24 dbu analog in facilities that follow SMPTE RP155. This discrepancy has caused problems and confusion for years, not only with the exchange of programming but with the manufacturers of broadcast equipment as well. It is not uncommon to find variations among different facilities (and, at times, within the same facility) as to what analog level indicators are adjusted to indicate with standardized reference signals. For example, some facilities in North America have calibrated their VU (volume unit) indicators (or similar devices) so that a 1 khz steady-state sine wave at 0 dbu indicates 0 VU instead of calibrating them so that +4 dbu indicates 0 VU. Given this condition, programming that has been adjusted to average 0 VU and then recorded onto a digital VTR will be recorded 4 db lower than expected (i.e., at 24 dbfs) since the recorder s gain controls at the default detent position will only line up a +4 dbu analog input signal with 20 dbfs (per SMPTE RP155). If this program is then passed through a contribution or distribution circuit that follows the EBU R68 recommendation it will emerge at the opposite end 6 db lower than expected. With situations like the one explained above it is quite clear that SMPTE RP155 and EBU R68 only work well for calibrating level indicators to a known reference and determining the amount of headroom through a given system. However, they assume every facility abides by these practices. Since this has not been proven to be true in all cases, measured level discrepancies will continue to exist. This can be summarized by acknowledging that reference or line-up levels can be arbitrary from facility to facility and measurements made relative to them (in the analog domain) are prone to significant errors. Fixed Loudness Reference To address the measurement problems experienced with these various, and often arbitrary, analog reference levels and calibration philosophies, there are recommendations that the industry should move toward the use of loudness measurement devices and methods using fixed references. For loudness measurements in the digital domain there is a convenient fixed reference of 0 dbfs. This fixed digital reference of 0 dbfs is universally agreed upon and is identical everywhere throughout the industry. Hence, a program produced in Europe with an average loudness of 18 dbfs can be verified (measured) in the United States to be at 18 dbfs. For analog television broadcast measurements there is also a fixed reference for measurement, 100% modulation. In the United States this equates to 25 khz peak deviation for monophonic audio and in most PAL and SECAM countries 100% modulation equates to 50 khz. The arguments between geographic regions, broadcast, and postproduction facilities over the proper line-up and level indicator calibrations may never be solved. Therefore, fixed measurement references that relate directly to innate channel-coding properties (analog or digital) must be utilized when controlling and verifying a program s loudness value. To summarize the arbitrary relationship between the loudness level and line-up level of broadcast programming, consider the following situations. If several different voices are adjusted in level so that they all deflect VU or PPM meters to the same mark, they may sound somewhat different in loudness to the listener. Both peak program meters (PPM) and VU meters are also frequently used to measure and/or align to a predetermined house reference level, and thus produce an arbitrary relationship to the dialog or speech loudness within a given program. For example, if a VU meter and a PPM meter are calibrated to display a reference tone equally, and speech that averages 0 VU is applied to both, the PPM meter will indicate levels considerably above its reference level and possibly above the maximum permitted level. On the other hand, speech that averages at the PPM reference will most likely indicate many db below 0 VU. This confirms the concept that the reference/line-up level is not the same as the dialog level of a program. AUDIO METERING AND MEASUREMENTS This section briefly discusses and provides a highlevel overview of a few audio level and loudness measurement devices and practices. It by no means attempts to give the reader a full treatise on this subject. As mentioned in the previous section, there has been and continues to be research in the area of loudness estimation and devices to measure this subjective quantity. 319