Expert-Caption. Users Manual. Version 5.2c. Copyright PixelTools Corporation

Transcription

1 Users Manual Version 5.2c Copyright

2

3 Contact Information: Wunderlich Drive Cupertino, CA USA Tel: +1 (408) Expert-Caption, MPEGRepair, ExpertHD, Expert1, Expert2, ExpertAudio, and ExpertMultiplex are trademarks of. All other company or product names are trademarks of their respective owners.

4 2

5 Congratulations on your purchase of PixelTools Expert-Caption You now have the ability to insert closed captioning into any exiting MPEG stream. You can also extract closed captioning or remove closed captioning from an MPEG file, analyze an MPEG file or transcode the captioning into a different format. Expert-Caption can also extract elementary video and audio tracks as well as individual frames from an MPEG file. You can update existing MPEG streams with new time code, aspect ratio and a variety of other options. The tool can add ADF, CGMS, APS and Content Advisory Flags (V-Chip) without re-encoding. 3

6 4

7 Table of Contents 1 What Expert-Caption Does General Description Features Benefits How this manual helps System Requirements Installation Tool Operation Overview Analyzer-Decoder Section Batch Operation Jump Start Tutorial Launching Expert-Caption Toolbar Drag and Drop Batch Mode Operation Detailed Decoding Operation File to Decode or Analyze IP to Decode or Analyze Decode AVC Decode Function Enable Decode Configure Extract Streams Extract Frames Fix Stream Audio Mux Decode Controls Run Pause Step Stop Analyze Decode Views Decode View Window

8 8.6.2 Decode View Window Overlays Decode Statistics Monitor Controls Display Window Status Line Progress Info Help About Batch Mode Operation Decoding MPEG File Decoding MPEG File Analyzing Closed Captioning Insertion SCC File Usage DDL File Usage HEXUSERDATA CLOSEDCAPTION PANSCANOFFSET SCC or CAP File Usage Initialization Files Default Initialization Files Loading and Saving Initialization Files Solutions Closed Captioning Insertion Adding Captioning During Decoding (Expert-Caption) Closed Captioning Format Extracting and Displaying closed captioning with Expert-Caption CGMS Insertion CGMS and APS flag Insertion During Encoding...95 CGMS and APS Flags Insertion to an Existing Stream (Expert-Caption) Source Material Included in the Install CD Recommended Reading MPEG Specifications MPEG Video Compression Standard Digital Video: An Introduction to MPEG

9 Glossary MPEG-2 Video Definitions Summarized from ISO/IEC MPEG-2 System Definitions Summarized from ISO/IEC

10 8

11 1 What Expert-Caption Does 1.1 General Description Expert-Caption is a Microsoft Windows based MPEG decoder application that demultiplexes MPEG Transport and Program streams, decodes video frames, and updates or inserts a wide variety of headers into the stream without re-encoding. The tool includes extensive closed captioning features that provide closed captioning analysis, display, extraction, removal, insertion, and transcoding. 1.2 Features Ultra simple Drag and Drop batch file captioning. Captioning File conversion between CEA-608 CEA-708 and SCTE formats Single step closed captioning addition and remultiplexing to existing MPEG file. Closed captioning insertion, extraction, and format conversion Extraction and Insertion of Analog Line 21 captioning data CEA-708 format analysis MPEG Stream Analysis, Splitting and Decoding Insertion of Meta data including CGMS, APS, Active Format Description, Content Advisory into existing streams Restriping of video time code on entire video Extraction of VANC and all user_data Support of dual language captioning Stream analysis General Runs on all MS Windows OS including Windows on Mac 9

12 Expert-Caption can insert captioning as defined in a.scc or a.cap file into an exiting MPEG stream without re-encoding. The comprehensive Closed Captioning features include captioning extraction, insertion, and transcoding. Closed Captioning formats supported include CEA-708, CEA-608, ATSC, SCTE, Cable Labs, and Analog (Extracted Line 21). The tool can also analyze streams to report the type(s) of captioning included in the stream. 1.3 Benefits Easy to use drag and drop captioning insertion of multiple files. Unattended batch mode operation Stream properties display Closed Captioning analysis MPEG stream compliance analysis Update of headers without re-encoding Support for CEA-708 and CEA-608 captions Cable Labs 2.0 compliant 10

13 2 How this manual helps This manual describes the functionality of the Expert-Caption Tool and provides operating hints. The brief Tool Operation Section should be looked at first if this is your first encounter with the tool. It describes the overall physical layout of the tool. The sections that make up the tool are illustrated. It provides you with enough background that you can start experimenting with the buttons on the tool. Many of the buttons bring up menus; each has own help button. The tool itself has a help button that accesses all of the dialog help descriptions. You can t break the tool. The sections following Tool Operation detail the operation of the specific buttons on each the sections of the tool. It should be used as a reference when using the tool. Detailed operations of user enhancement features (that are not directly tied into the buttons on the face of the tool) are next provided. These operations involve the usage of initial parameters and changeable parameters. The contents of the installation CD are described. The installation disk provides sample MPEG streams and associated.scc captioning files so you can easily test closed captioning insertion. 11

14 12

15 3 System Requirements Expert-Caption will function on any Intel 386 compatible processor under all Windows 7, Vista, XP, and 2000 operating systems. Expert-Caption will also function in a Macintosh computer running a Windows emulator. Expert-Caption requires approximately 10 Mbytes of system memory to process SIF size images, 40 Mbytes of memory to process Main Profile streams, and 80 Mbytes of memory to process High Profile streams. The MPEG decoding process is very computationally intensive. The faster the processors speed, the faster the decoding and processing. Expert-Caption will run faster on SSE3 machines and multiprocessor machines, but SSE3 and dual processor machines are not required. The tool should accomplish HD (1920x1080) MPEG decoding faster than real time. SD encoding and decoding should run even 3-4 times faster. PixelTools products include internal performance monitoring capability. Contact us if you would like assistance in optimizing the code in your particular environment. 13

16 14

17 4 Installation Simply double click on Expert-Caption Install program icon in the main directory of the installation CD or on your hard disk if you have received the program via . The Expert-Caption installer will prompt you for additional program loading options. The program can be easily un-installed by running Expert-Caption installer program again from the installation CD or you hard drive and selecting the Remove Program option in the Program Maintenance dialog that will come up if the program is already installed. The program can also be uninstalled using the Windows Control Panel Install and Remove Programs option and select Expert-Caption program. Select the remove program option from Expert-Caption maintenance dialog. If you upgrade your operating system, it is recommended that you run the new install.exe program located in your Expert-Caption install folder and select the Remove Program option. Remove the PixelTools Key Lock, re-run the new install program, and then re-insert the Key Lock. 15

18 16

19 5 Tool Operation Overview The Expert-Caption tool interface is comprised of two subsections. We ll take a quick tour of the tool to give you an ideal of the location of the subsections. Decode Section Monitor Section When looking at the tool, you see the monitor window displaying the PixelTools logo on the right and decode controls on the left. The decode controls enable you to view, decode, analyze, split, repair, and store the decoded frames from any legal MPEG-1 and MPEG-2 stream. The set of controls in the right side of the tool is the monitor section. The monitor window displays the most recent decoded frame or the most recent encoded input frame so you can monitor the video as it is being processed. A progress bar gives you an idea of how fast the decoding is processing. 5.1 Analyzer-Decoder Section The Expert-Caption analyzer decodes and analyzes MPEG1, MPEG2 and AVC elementary, system, program, transport, and DVD (.vob) streams. Decoded frames can be viewed in a color 17

20 window along detailed graphical overlays that indicate the encoding decisions (block types, quantization, and motion vectors) that went into the stream encoding. The window is opened using the Decode de Window button at the top of the GUI. MPEG compliance errors and user selectable video, audio, and system stream parameters or statistics are displayed in the Statistics display window opened using the st button at the top of the GUI. All MPEG compliance errors detected and any stream statistics selected is also written to the Decode.Log file located in the directory where your source file is located.. The elementary Video and Audio components of Transport, Program, and System streams can be saved. Closed Captioning from elementary user_data or from analog Line 21 can be extracted from the stream and saved as a.scc file. Each decoded frame can be stored as a YUV, a bitmap, or a TGA file. Elementary video and elementary audio streams can also be extracted from system streams and stored in new elementary streams. The Configure Fix Stream property pages allows you to update headers throughout an encoded MPEG stream. This property page also provides access to a Expert.Ddl parameter file that allow you to insert closed captioning and pan/scan offsets throughout an existing file. The Analyze Decode button displays the selected stream's properties in a dialog box and writes the properties into the Decode.Log file. 5.2 Batch Operation The powerful Expert-Caption can be controlled with the batch mode. Using this technique, you can easily process all files in a folder for example. The tool also can be run from drag and drop. In a MS Windows window, just select all of your source file(s) (for decoding, or captioning insertion) and drag them to the Expert-Caption icon on your desktop or in its install folder. The tool will automatically process the files drug onto the icon. The specific processing is specified in the ExpCap.ini file found in the tool s boot folder. 18

21 6 Jump Start Tutorial Expert-Caption Installation CD-ROM includes sample compressed MPEG streams for demo purposes. This tutorial will lead you through the insertion of closed captioning in a sample MPEG file and subsequent captioning verification. In this tutorial, we will analyze a sample MPEG stream to verify that is has no captioning, insert captioning contained in a.scc file, verify the type of captioning inserted, and finally verify the captioning over the decoded video. Analyze source video: First launch Expert-Caption and select the sample source MPEG file SczDrop.Mpg in the File to Decode or Analyze edit box. Depress the Analyze button. An analysis dialog should appear that indicates that the MPEG stream has no Closed Captioning identified in your source stream. The Closed Captioning format would appear in the right hand side of the Elementary Video Stream section of the dialog if it existed. Insert Closed Captioning Launch Expert-Caption and select the sample source MPEG file SczDrop.Mpg in the File to Decode or Analyze edit box. Depress the Configure button and select the Fix Stream Tab. Check 19

22 the Add Closed Captions or Commands from File check box. Browse to the appropriate.scc file called PtScc.scc (found on your sample CD). Select the Closed Captioning Format of SCTE20 & CEA708 NTSC. Finally, enter the name of your MPEG file that will hold your newly captioned video in the upper Save Fixed File edit box. (Make sure that the output file drive is not your install CD!). Your Fix Stream dialog should look like: Now depress the OK button to apply your settings. You are ready to insert the captioning. For verification that the captioning is being inserted, open the Decode Statistics window by depressing the St button at the top of the GUI. Depress the Control Decode Run button to start the process. You should see text in the Statistics Window like: 20

23 The sample video should be displayed in the mini-monitor during the process. You can greatly speed up the processing by disabling the Mini-Monitor (Depress the top right button) and by checking the Decode Configure Disable Decode button. Analyzing Captioned MPEG stream If all went well, your new Captioned.Mpg file should contain closed captioning. You can easily verify this by relaunching Expert-Caption, selecting the Captioned.Mpg file in the File to Decode or Analyze edit box, and depressing the Analyze button. The new analysis box should show the multiple closed captioning formats that have been inserted: 21

24 View the Captions Finally, you can view the captions over the decoded window. With your Captioned.Mpg file selected in the File to Decode or Analyze edit box, open the Decode window by depressing the de button at the top of the GUI. Verify that the windows menu items Text Overlays Closed Captioning Field 1 and CEA-708 have been checked. Depress the Control Decode Run button. You should see the CEA-708 captions in green and the CEA-608 captions in pink displayed over the decoded video: 22

25 Note that the captioning location and display color are for demonstration only. The caption locations and color will be controlled by the target TV. Multiplexed video In this tutorial, you have inserted captioning into an elementary video MPEG stream IE a stream without sound. Expert-Caption can insert captioning into Multiplexed streams as well. Closed captioning is inserted into the elementary video stream which increases the elementary stream length and necessitates re-multiplexing. Closed Captioning insertion into multiplexed video entails two passes through the stream. The first pass inserts captioning into the elementary video as above and also extracts the audio track using the Configure Extract Streams dialog. The second pass multiplexes the captioned elementary video with the extracted audio using the settings in the Configure Audio/Mux dialog. The Drag and Drop method of running Expert Caption (see below) will automatically run the above two steps to insert captioning into multiplexed streams. 23

26 24

27 7 Launching Expert-Caption Expert-Caption can be started in several ways. This provides flexibility in using Expert-Caption by itself or in the more automated batch mode. 7.1 Toolbar You can start Expert-Caption by selecting Expert-Caption program icon in the Windows toolbar, double-clicking on Expert-Caption shortcut on the desktop or selecting Expert-Caption icon in the Windows run toolbar dialog. 7.2 Drag and Drop Expert-Caption can process up to 64 user selected input files via drag and drop. Setting any of the output file names to will cause the tool to create the output file name based on the input file name. The ExpCap.ini file in the folder where Expert-Caption is stored must be set with the appropriate file names. Expert-Caption drag and drop of MPEG files can be used to insert closed captioning in the MPEG file(s). If the file extension of the dropped file ends in.mpg,.vbs,.mpt,.mpv, or.mp2 using the default ini parameter file, the tool will search for a.scc file that has the same file name as the MPEG file and is in the same folder as the MPEG file. If a corresponding.scc captioning file is found, the tool will proceed to automatically insert the captions defined in the.scc file into the MPEG file. The tools first analyzed the source file to determine the multiplex type and multiplexed bit rate. The tool then proceeds to extract the elementary video, extract the elementary audio, and insert the captions into the elementary video. Following this extraction pass, the tool will remultiplex the temporary audio and video files into the identical format as the original video. The tool defaults to the captioning style of SCTE20 / CEA 708 if no user captioning format has been selected. If you require a different captioning format, launch Expert-Caption, select your desired different captioning format, and same as the default ini. The drag and drop closed captioning insertion can process all selected files that are dropped onto the Expert-Caption icon. 7.3 Batch Mode Operation 25

28 The Expert-Caption can also be launched from a command line batch file. You just need to call Expert-Caption and give it the name of the video file to be multiplexed. For example: Call Expert-Caption Sample.mpg process Sample.mpg with the settings defined in the ExpCap.ini file. Expert-Caption will create output file names based on your source file name if the - character is used as the output file name. Refer to the Batch Mode Operations Chapter for more details on this feature. 26

29 8 Detailed Decoding Operation This section of the manual describes the decode section (left hand side) of the tool starting with the controls at the top left of the tool. The tool can decode streams from a disk based file or from an Internet Protocol bases stream. 8.1 File to Decode or Analyze The File to decode or analyze should be entered in this edit box or selected via the File to Decode or Analyze File button. Expert-Caption can decode MPEG -1 or MPEG -2 elementary, program, system, transport, or DVD (vob) streams. Expert-Caption can decode streams directly from DVD disks or over networked computers. Expert-Caption can also decode and analyze AVC (H.264) elementary or Transport Multiplexed streams. 27

30 Expert-Caption can also decode an elementary AVC or H.264 file. The AVC or H.264 file must have an extension of.264 or.avc. The decoded AVC file is displayed in the mini-monitor and the de decode window if opened. 8.2 IP to Decode or Analyze Expert-Caption can also decode an IP stream. Depress the File to Decode or Analyze IP button to bring up the IP selection dialog. 28

31 Enter the IP address, Port, and protocol of the source MPEG video you wish to decode or analyze. 8.3 Decode AVC Expert-Caption can decode elementary or transport multiplexed AVC (H.264) files. 8.4 Decode Function Enable The Configure button brings up a set of property pages that facilitate decoder configuration Decode Configure The Decode Configure property page allows you to customize the overall decoding process. 29

32 The following Decode Configure options are available: 30

33 Enter Number of Frames to Decode or ALL: The number of frames to decode can be entered in this box. Entering All will cause the decoder to decode the entire MPEG file. Why? This option will allow you to decode a specific segment within a large MPEG file Disable Decode Disable decode will cause the decoder to analyze the stream headers and repair the stream headers but not to decode the stream or save the frames to disk. Why? Expert-Caption, using the Disable Decode option, can be used as a stream header analyzer. The analyzer and repair function runs much faster when the decoder is disabled Loop Decode Checking the loop decode will cause the Expert-Caption decoder to decode the same MPEG stream over and over until the Stop button is depressed. Why? Loop Decode can be used to generate a large sequence of output files from a smaller MPEG sequence. It also can be used to demonstrate MPEG content and analysis Select PIDs to Decode This section contains two edit boxes in which you can enter the decimal Program Identification Number (PID) for the video channel you wish to decode, analyze or split into elementary video and for the audio channel you wish to split or analyze. If the transport channel has only one video stream, entering 0 in these edit boxes will instruct the Expert-Caption to extract the video PID number from the stream header. If you are unsure of the video PID(s) in the stream, entering 0 will cause Expert-Caption to decode the first video program it finds. If you enter a PID which is not found in the stream, the Expert-Caption will search for the selected PID for the first few seconds of video data and then print out a message listing all of the PIDs that it found. 31

34 The PID entry boxes have no effect if the input stream is not in MPEG2 Transport format. Why? Expert-Caption can be used for extracting a single video stream out of a bundle of video streams for analysis or repair Extract Streams The Extract Streams property page allows you to set up the decoder to extract the various elementary streams from a multiplexed stream and to save the extracted streams in disk files. 32

35 Save Input Stream as Elementary Video and Audio streams These options will cause Expert-Caption to save the elementary video MPEG and elementary audio MPEG into separate stream files when decoding a MPEG -1 system stream, a MPEG -2 Program Stream or a MPEG -2 transport stream. If the video and/or audio box is checked, the video and/or 33

36 audio components of the decoded stream will be stored in the file indicated in the associated edit box. The Video and/or Audio check boxes can be selected and de-selected during the decoding process. AVC (H.264) Transport streams can be split into elementary video and audio components using the Decode Extract Streams options. Why? Expert-Caption operates on elementary video streams and therefore the video streams must be extracted from the system or program stream for stream operations. There are occasions when it is desirable to replace video or audio streams within a MPEG system and the streams must be split and stored before re-multiplexing. It is also possible to store segments of the elementary video and audio streams by selecting and de-selecting the store elementary option during the decode process Elementary Video in File: Checking this option causes the elementary MPEG1 or MPEG2 video stream to be saved into the file you have entered or selected with the adjacent Browse button Elementary Audio (or AC3 in Transport) into file: Checking this option causes the elementary MPEG1 or MPEG2, audio stream, if present, to be saved into the file you have entered or selected with the adjacent Browse button. This option will cause an AC3 stream, if present in a transport stream, to be saved. (You should give the file an.ac3 suffix.) Save Private 1 Stream (AC3 or PCM in VOB): These options will cause Expert-Caption to save all data stored as the MPEG2 private_stream_1 into the file you have entered or selected with the adjacent Browse button. VOB files (often found on DVDs) use private_stream_1 to store Dolby AC3 or uncompressed PCM audio. VOB files store MPEG audio in the elementary audio stream not the private_stream_ Save Private 2 Stream: These options will cause Expert-Caption to save all data stored as the MPEG2 private_stream_2 into the file you have entered or selected with the adjacent Browse button. VOB files (often found on DVDs) use private_stream_2 to store DVD specific data Save Closed Captioning Field1 and Field2 34

37 Checking this option causes Expert-Caption to extract closed captioning data or other data from the source MPEG stream s user_data and save the data into the file designated in the adjacent edit box. This option allows you to store the user_data in a variety of formats. The suffix of the output file determines the output format. If the output file format is a closed captioning file format, you can designate a different file for Field 1 and Field 2. The captioning can be saved for re-insertion into this or another file. The CEA-708 captioning can be saved in several formats that aid in the debugging of CEA-708 displays. The following file suffixes/formats are currently recognized:.txt Selecting a.txt file extension will extract the CEA-608 data to your designated text file. The text file contains the field number (Ch1 or Ch2), the CEA-608 commands using the acronyms defined in the specification and the translated closed captioning text that is found. The following is an example of the text file output:.scc Selecting a.scc file extension will extract the CEA-608 data from your source video, reformat it and save it as an industry standard closed captioning.scc file. This file format is a popular method of defining closed captioning. Most PixelTools products accept captioning in a.scc file format. The.SCC file format consists of a time code followed by the actual CEA-608 data that will be inserted starting at that time code two bytes per frame until the entire line of data has been sent. The.SCC file can be of drop frame or non drop frame style. The SCC data includes CEA-608 commands followed by data. The 7 bit data fields have a MSB added to set odd parity and so the commands and data are not easily readable. The following is an example of the.scc file format: 35

38 .EDL Selecting an.edl (Encoding Decision List) or a.ddl (Decoding Decision List) file extension will extract all of the user_data data from your source video and save it exactly as it appeared in the stream. The user_data can contain CEA-708 or CEA-608 closed captioning or any other public or private data. The.EDL or.ddl file can be then used to insert this user_data into a stream during encoding or into an existing elementary stream. The EDL and DDL file format was developed by PixelTools to support their line of MPEG encoders. The frame number at which the associated user_data is to appear starts the command. The command HEXUSERDATA defines the type of EDL or DDL command. The actual user_data then appears after the commands. Note that the user_data start code of 0x000001b2 will automatically be added by the PixelTools encoder or decoder when it receives that HEXUSERDATA command. The following is an example of the.edl or.ddl format saved by this option: 36

39 .COD Selecting a.cod file extension will extract the CEA-708 DTV data for all 8 DTV windows. This option displays the CEA-708 DTV control commands and associated CEA-708 text as it is encountered in the MPEG stream being decoded. The SVC number represents the logical DTVCC sub-channel service that is being utilized. The CEA-708 commands are listed in their text form with the letters that for the CEA-708 command acronym capitalized. The specific window (out of the eight possible windows) that this command is being directed towards is listed after the command. Data associated with some commands is printed out in a hex format after the command. The following is an example of the.cod file format., RAW Selecting a.raw file extension will extract the DTV data from CEA-708 closed captioning found in an ATSC wrapper. 37

40 This option displays the CEA-708 DTV raw data after being extracted from the Service Level and Packet Level wrappers. The display frame number in which the data appears is printed for each packet followed by it s frame type (I,P or B) and then followed by the frame s transmission order (which may differ from the display order as frames are sometimes transmitted in a different order than they are displayed). The Inc number is the two digit packet count which repeats from 0 through 3. The PktSize is the Packet Size listed in the start of each packet. The DataBytes is the actual number of data bytes stored in this packet. The numbers following the DataBytes number are the exact hex numbers found in this particular packet in this frame. A single frame may have multiple packets. The first next number following the DataBytes number represents the PacketSize (lower fiver bits) and the Window Incrementor (upper three bits). The numbers following the PacketSize and Window Incrementor are the hex values in the packet that contain the DTV control sequences and the actual closed captioning text. The following is and example of the.raw format: Checking the Save Digital Data or the Save Analog Data buttons prior to transcoding will cause the Digital or Analog closed captioning data to be extracted from the source video and inserted in the re-encoded MPEG as specified in the Optimize Encode Configure Line 21 dialog Save Closed Captioning Analog Data Checking this option allows you to extract the analog closed captioning that appears on one of the lines of your source video and save it a variety of formats into the file you have designated in the adjacent edit box. The file extension of your designated output file determines the format of the file to be extracted. The following file formats are recognized:.scc All non NULL CEA-608 byte pairs will be reformatted and saved along their appropriate time code into an industry standard.scc file. This file can them be used to re-insert captioning in this or 38

41 another file. It can also be used to re-insert the old Analog captioning into user_data with CEA- 708, ATSC, and SCTE formatting..txt The ASCII text and control characters encountered in the analog captioning will be saved. This is useful for verifying the captioning that exists in a specific file. Checking the Save Digital Data or the Save Analog Data buttons prior to transcoding will cause the Digital or Analog closed captioning data to be extracted from the source video and inserted in the re-encoded MPEG as specified in the Optimize Encode Configure Line 21 dialog Select Specific line to extract Analog Closed Captioning Data The Save Analog Data option causes the Expert-Caption decoder to extract the analog closed captioning data from your designated line in each frame and save the two bytes in the file you have specified below the Save Analog Data check box. Why? Closed captioning data initially was stored in non-viewable lines in old analog TV broadcast signals. Analog bandwidth permitted two closed captioning characters (8 bits each) to be encoded in a non viewable line on each field. Some MPEG encoders encode this analog signal into extra lines at the top of each encoded frame. This Expert-Caption option allow you to extract the two bytes per line, store them in a text file, and/or encode them into user_data per the current DVD and HD specifications. If you save the analog captioning into the industry standard.scc file format, you can use Expert-Caption to insert CEA-608 or CEA-708 captioning into an existing MPEG stream or into a stream during encoding Extract Frames The Extract Frame property page provides controls for decoding and saving individual frames to disk files. These options can be changed during stream decoding. You can quickly extract only those frames you want by seeking to a point ahead of your extraction point in a MPEG stream, decoding individual frames up to the exact extraction point, checking the Store Decoded Frames option, and then running the decoder for the desired number of extracted frames. 39

42 Save Decoded Frames to Output Files Selecting this check-box will cause the decoder to save every decoded frame to a file. A valid file name must be entered in the associated edit box. The decoder can store frames in three popular formats: YUV, TGA, or BMP. YUV files will be the smallest size. TGA files are most common in 40

43 frame grabbing applications. The suffix of the Output Frame name will determine what format the frame will stored as. Valid suffixes are YUV for YUV files; TGA for TGA files; and BMP for BMP files. The output file name also must have a number embedded in the file name. This number is incremented by the decoder after every file write. Valid file names are: Frame0.YUV Frame023.Bmp or Frame3x.Tga The Store Decoded Frames check box can be selected and de-selected as desired throughout the decoding process. Why? Expert-Caption can re-encode these decoded frames and re-insert them into the original MPEG stream using the Replace or Insert functions. You can touch up the frames using your favorite image editor and re-insert the resultant files into the MPEG compressed bit-stream. Also, selected video frames can be "captured" from the MPEG file by enabling the Store Decoded Frames check box at appropriate points throughout the stream. The Decode Seek function provides additional capability to quickly retrieve files from anywhere within a large MPEG file RGB to YUV Conversion This option allows selection of the RGB to YUV conversion specifics. Selection of the Video mode causes the full RGB range to be converted to the full YUV range. Selection of the Graphics mode causes the full RGB range to be scaled to the standard graphics range. Why? Computer generated graphics are often in the range 0 to 255. Standard television video is in the range of 16 to 240. Selecting the Graphics mode causes the encoder to reduce the 0 to 255 RGB source to the 16 to 240 range to be compatible with standard signals. Note that selecting the Video mode does not limit the input source range. The video mode allows 0 to 255 values to be encoded as 0 to 255 values. It is often the case that video frames are themselves limited to 16 to 240 values. The graphics mode makes computer generated frames compatible with the video levels. 41

44 8.4.4 Fix Stream The Fix Stream control allows you to update all headers of the selected type throughout the entire encoded MPEG stream. This process occurs very fast as no transcoding takes place. You can select one or all of the options. To perform the fix process, depress the Decode Run button. The process can be sped up by disabling the coding via checking the Decode Configure Disable Decode control. Note that these options change the headers only and do not change the actual encoding. It is very easy therefore to produce an illegal MPEG stream where the headers do not correspond with the content. 42

45 Save Fixed File 43

46 The Save Fixed File edit box and associated Browse button are used to select the name of the file that will be used to save the fixed MPEG stream. These controls are enabled once one of the fix options is checked. If the Fixed File Name is entered as -, Expert-Caption will save the output to a file with the identical prefix as your source input file but with the suffix Fixed.vbs. This facilitates the batch mode functionality of Expert-Caption Save Multiplexed Format Checking the Save Multiplexed Format check box will just modify the requested bits within the existing Program or Transport stream if the Fix update does not change the length of the elementary video stream. Updating the Top First Flag or resetting the GOP time code are examples of fix processes that can be do not change the length of the elementary video stream and can output the original multiplexed format including all private data. The Save Multiplexed Format check box will cause automatic audio extraction and remultiplexing if the Fix update includes options that change the length of the video stream. Inserting closed captioning and ADF flags are examples of operations that require re-multiplexing. The source multiplexed file format is analyzed at the start of the Fix process. The audio is extracted coincident with the video Fix process. Once the elementary video has been Fixed and the audio has been extracted, Expert-Caption automatically re-multiplexes the video and audio to the same multiplexing type (Transport or Program stream) and identical bit-rate as the source video Update Top First Flag Checking this option causes Expert-Caption to update all of the top_first flags in the stream. Checking the top_first value will cause all top_first flags to be set to true. Unchecking the top_first value will cause all top_first flags to be cleared to false. This flag has no affect the MPEG encoding. This flag instructs the decoder which field to display first Update Repeat First Flag Checking this option causes Expert-Caption to update all of the repeat_first flags in the stream. Checking the repeat_first box will cause all repeat_first flags to be set to true. Unchecking the repeat_first box will cause all repeat_first flags to be cleared to false Update Low Delay Flag 44

47 Checking this option causes Expert-Caption to update all of the low_delay flags in the stream. Checking the low_delay box will cause all low_delay flags to be set to true. Unchecking the low_delay box will cause all low_delay flags to be cleared to false. This flag has no effect on the MPEG encoding. This flag indicates to decoders that there are no B frames in this stream. Note that if you set this flag true in streams that have B frames, the stream may not be decoded properly in some decoding situations Update Progressive Frame Flag Checking this option causes Expert-Caption to update all of the prog_frame flags in the stream. Checking the prog_frame box will cause all prog_frame flags to be set to true. Unchecking the prog_frame box will cause prog_frame flags to be cleared to false Update Progressive Sequence Flag Checking this option causes Expert-Caption to update all of the prog_seq flags in the stream. Checking the prog_seq box will cause all prog_seq flags to be set to true. Unchecking the prog_seq box will cause all prog_seq flags to be cleared to false Update Constrained Parameter Flag Checking this option causes Expert-Caption to update all of the constr_par flags in the stream. Checking the constr_par box will cause all constr_par flags to be set to true. Unchecking the constr_par box will cause all constr_par flags to be cleared to false. This flag has no effect on the MPEG encoding. The flag informs the decoder that the stream parameters do not exceed those specified in the constrained parameter set Update Closed GOP Flag Checking this option causes Expert-Caption to update all of the closed_gop flags in the stream. Checking the closed_gop box will cause all closed_gop flags to be set to true. Unchecking the closed_gop box will cause all closed_gop flags to be cleared to false. Note that this fix process does not actually change the GOPs and so if you set the closed_gop flag on a stream with open GOPs, the stream may not be decoded properly in some decoding situations Update Aspect Ratio Code 45

48 This option updates all of the aspect_ratio codes with the AspectRatio selected in the combo box. The aspect ratio has no effect on the MPEG encoding. The code tells the decoder the aspect ratio of the source Update Frame Rate Code This option updates all of the frame_rate codes with the FrameRatio selected in the combo box. This option does not change the actual encoded bit-rate. The MPEG encoding output bit-rate is very much a function of the input frame rate and so changing the frame_rate codes may produce un-intended playback issues Update Profile Code This option updates all of the profile codes with the Profile selected in the combo box. The Profile code has no effect on the MPEG encoding. The code informs the decoder of the maximum set of stream encoding values that this stream can contain Update Level Code This option updates all of the level codes with the Level selected in the combo box. The Level code has no effect on the MPEG encoding. The code informs the decoder of the maximum set of stream encoding values that this stream can contain Update Video Input Code This option updates all of the video_input codes with the Video Input selected in the combo box. The Video Input code has no effect on the MPEG encoding. The code tells the decoder original source of the video Update Matrix Coefficient Code This option updates all of the matrix_coefficient codes with the Matrix Coefficients selected in the combo box. The Matrix Coefficient code has no effect on the MPEG encoding. The code tells the decoder the color conversion used in generating the YUV for MPEG encoding.. 46

49 Reset MPEG Time Code This option causes the stream to re-write all time codes in the GOP headers with a new timecode sequence. The new time code will start with the values entered in the TimeCodeHour, TimeCodeMinute, TimeCodeSecond, TimeCodeFrame edit boxes and increment every GOP header as a function of the frame rate, number of frames and the DropFrame state Update Pan and Scan Frame Size This option causes sequence_display_extension headers to be updated or added with the horizontal_display_size and the vertical_display_size entered in the horizontal and vertical edit boxes. This is commonly known as the Pan and Scan size and defines an alternative display window size. The Pan and Scan frame size has no effect on the MPEG encoding. The headers tell the decoder what size it should make its alternate display window if one is desired Reset MPEG Time Code The Decode Fix Stream option will update a Program, System, or Transport multiplexed stream and only change the selected headers if the Save Multiplexed Format option is checked and selected change does not require a change of stream length. The Save Multiplexed Format option does not support changes requested via Expert.ddl file mechanism or the Insert XDS flags. If the Save Multiplexed Format is not checked, the Decode Fix will still update a Program, System, or Transport stream but will save the fixed stream as an elementary stream Insert XDS Flags Checking the Insert XDS Flags causes the selected add XDS flags to be added to an existing MPEG elementary stream without re-encoding. Checking the Insert XDS Flags check box will cause the decoder to add a set of Line 21 extended data at four second intervals throughout the stream. Included in the data are CGMS and APS flags The CGMS flags can have one of the following options: Copy Freely Copy Once Copy Never The APS flags can have one of the following options: 47

50 Off Type 1 Type 2 Type 3 48

51 The Content Advisory flags can have one of the following options: none MPA: G MPA: PG MPA: PG-13 MPA: R 49

52 MPA: NC-17 MPA: X USA TV-YG USA TV-Y7 USA TV-G USA TV-PG USA TV-14 USA TV-MA Can Eng E Can Eng C Can Eng C8 Can Eng G Can Eng PG Can Eng 14+ Can Eng 18+ Can Eng Fr E Can Eng Fr C Can Eng Fr 8 ans+ Can Eng Fr 13 ans+ Can Eng Fr16 ans+ Can Eng Fr 18 ans+ Selecting the none options disables the addition of the advisory beacons. The following Content Labels are also supported: V (Violence) This also adds the FV fantasy violence flag for children s programming. D (suggestive Dialog; usually about sex) L (coarse or crude Language) S (Sexual situations) Active Format Description Checking this option causes decoder to insert your selected Active Format Description (AFD) headers to your MPEG stream every frame. One of following AFD options can be selected: box 16:9 top box 14:9 top box > 16:9 center same as coded format 4:3 center 16:9 center 14:9 center 4:3 protect 14:9 center 16:9 protect 14:9 center 16:9 protect 4:3 center 50

53 AFD headers instruct decoders that display different dimensions than that of the encoded video how to display the content Line 21 Closed Captioning Checking the Add Closed Captions or Commands From File check box, causes the decoder to read and execute commands in a PixelTools.DDL file or a PixelTools EDL file or to insert closed captioning as defined in an industry standard.scc or.cap file. Refer to the DDL file section to review the DDL commands that are available Closed Captioning File Format Expert-Caption inserts closed captioning that is stored in a variety of file formats. The formats contain the closed captioning text and control codes along with the time code at which the text and commands are to be inserted in the stream. The captions can be stored in a PixelTools.edl [Encoding Decision List] command file, an a PixelTools.ddl [Decoding Decision List] command file, an industry standard.scc file or.cap. The.edl file or.ddl can contain the actual closed captioning text along with time code or frame numbers; The.edl or.ddl file can also contain the.scc codes or the actual hex data to be inserted in user data. The.scc file must be formatted per the industry standard Scenarist Closed Captioning file format. The PixelTools.edl and.ddl files are very useful in copying exact captioning from one file to another file. You can use the Decode section of the tool to Extract the caption stream as a.ddl file. This file contains the actual binary data and the associated time code. This file can then be used as the encoding caption file source to insert the exact same captions at the exact same time code where there were extracted Dual Language Captioning Expert-Caption can insert two captioning files simultaneously into your MPEG stream. These two captioning files can be two different language version of the captioning. The alternate language captioning file must be in the same folder as the primary folder and must have the same name and prefix as the primary captioning file but must have a _A at the end of the file prefix. For example, if your primary file is named CheezeCake.Scc your alternate language file must be named CheezeCake_A.Scc and stored in the same folder. 51

54 Closed Captioning Format The closed captioning can be inserted per several format specifications: CEA 608 Selecting this option causes your closed captioning text data to be reformatted as CEA 608 data including control characters as Rollup 3 style captioning. The captions are added two characters per picture user_data header per frame in transmission order. The closed captioning data in this format starts with Selecting this option causes your SCC commands to be inserted as two characters per picture user data in transmission order and embedded in the appropriate 608 headers. Selecting this option causes HEXUSERDATA to be inserted as is in transmission order into picture user_data headers. The HEXUSERDATA must contain the appropriate CEA 608 wrapper ATSC/ CEA 608 Selecting this option causes your closed captioning text data to be reformatted as CEA 608 data including control characters as Rollup 3 style captioning. The captions are added two characters per picture user_data header per frame in display order. The 608 data is embedded in SCTE-21 and ATSC data. The closed captioning data in this format starts with Selecting this option causes your SCC commands to be inserted as two character per frame in display order and embedded in the appropriate 608, SCTE-21 and ATSC headers. Selecting this option causes HEXUSERDATA to be inserted as is in display order into picture user_data headers. The HEXUSERDATA must contain the appropriate ATSC/CEA 608 wrapper DVD Selecting this option causes your closed captioning text data to be reformatted as CEA 608 data including DVD control characters. The captions are added at up to 15 characters per GOP user_data header. The closed captioning data in this format starts with f8. Selecting this option causes your SCC commands to be inserted as up to 15 characters per GOP user_data header. Selecting this option causes HEXUSERDATA to be inserted into GOP user_data. The HEXUSERDATA must contain the appropriate DVD format. 52

55 CEA-708 DTV Up to 18 closed captioning bytes added as a CEA-708 Window 0. The data is added to Picture user_data in transmission order. The first set of user_data defines and enables Window 0. The remaining picture user_data headers contain the closed captioning bytes which contain the text. If no text is included, the bytes are NULL. Both the Text and SCC captioning data is converted to CEA-708 commands and data CEA-708 NTSC Up to 2 closed captioning bytes added as a CEA-708 NTSC field 1 or 2 (determined by the Closed Captioning Style selection). The data is added to Picture user_data in transmission order. The user data also includes NULL DTV data. This mode is designed to facilitate legacy support of CEA 608 captioning in CEA 708 systems CEA-708 DTV & NTSC Up to 18 CEA 708 captioning bytes and 2 CEA-708 NTSC field 1 or 2 captioning bytes are added per frame. The data is added to Picture user_data in transmission order SCTE 20 Selecting this option causes your closed captioning text data to be reformatted as ANSI/SCTE20 data including control characters as Rollup 3 style captioning. The captions are added two characters per picture user_data header per frame in display order. The closed captioning data in this format starts with Selecting this option causes your SCC commands to be inserted as two character per frame in display order and embedded in the appropriate ANSI/SCTE20 headers. Selecting this option causes HEXUSERDATA to be inserted as is in display order into picture user_data headers. The HEXUSERDATA must contain the appropriate ANSI/SCTE20 wrapper SCTE 20 & CEA-708 NTSC Selecting this option causes your Text or SCC data to be inserted three times into the stream. The data will be inserted as two characters per line in an SCTE20 packet as described above. The Text or SCC data will be inserted as two NTSC characters in the 708 NTSC bytes as described above. The Text or SCC data will be converted to DTV control and data and will be inserted as a multi byte pairs of data as described in the 708 DTV section above. 53

56 C-Cube Selecting this option causes your closed captioning text data to be reformatted as CEA 608 data including control characters as Rollup 3 style captioning. The captions are added two characters per picture user_data header per frame in display order. The closed captioning data in this format starts with Selecting this option causes your SCC commands to be inserted as two characters per picture user data in display order and embedded in the appropriate 608 headers. Selecting this option causes HEXUSERDATA to be inserted as is in display order into picture user_data headers. The HEXUSERDATA must contain the appropriate CEA 608 wrapper Closed Captioning Style The older closed captioning formats can be inserted in one of two field styles. The user_data is placed in each digital frame and not in separate fields. Each older captioning format utilizes a different mechanism to define the field to which the captioning data is placed. The CEA-708 DTV format does not utilize a captioning style and the settings are ignored for this format Field 1 The captioning data is placed in the field 1 section of user data for the selected captioning format Field 2 The captioning data is placed in the field 2 section of user data for the selected captioning format Xcode Analog Captioning from line: Checking this option causes the tool to decode the analog closed captioning from the selected line and then add the extracted two bytes in each frame to the user_data field per the selected Closed Captioning Format and Style. Why? One of the original methods of adding line 21 data to MPEG streams involved adding 32 additional scan lines to the source frame IE 720 x 512 instead of 720 x 480 and including the line 21 analog 54

57 captioning in one of these new scan lines. Different capture equipment inserted the analog line 21 data in different scan lines. The edit box allows you to select the appropriate line that contains the analog line 21 data. Modern specifications require that the closed captioning be inserted in user_data. This feature allows you to decode the two analog bytes per frame and insert them into the user data in the appropriate format. If image resizing is desired IE removing the extra 32 lines of each frame, the video must be reencoded. The Encode section of Expert-Caption can extract the analog captioning, remove the 32 extra lines, and re-encode the resized frames adding the two bytes per frame in the user_data Offset (secs) If the time code embedded in your elementary video does not correspond with the time code in your closed captioning file, you can correct for that in this edit box. Entire the number of seconds that you want added to or subtracted from the timecode in your captioning file before it is inserted into the stream. Often, for example, your MPEG video time code starts at 00:00:00:01 but your captioning file starts at 01:00:00:00. To correct for this hour difference, enter a (60sec per minute * 60 minutes per hour) in the edit box. Your can easily verify the proper captioning timing before processing your entire file by opening the Decode Statistics (St) window and Depressing Decode Step to start the decoding process. The Statistics window should display the frame number and time code of the first captioning that will be added to the stream Remove Existing Captioning Checking this option causes all recognized digital user_data closed captioning to be removed prior to adding new captioning or EDL commanded data. The recognized captioning data must be one of the closed captioning formats. (You do not have to select the Closed Captioning Format to remove. If the closed captioning format is not one of the formats supported by this tool, it will not be removed with this option. The Remove All User Data will remove un-recognized captioning.) Remove All User Data Checking this option causes all existing user_data to be removed prior to adding new captioning or EDL commanded data. 55

58 8.4.5 Audio Mux Expert Caption includes a Program Stream and a Transport Stream multiplexer so your fixed video files can be multiplexed with your audio into a format identical with the original multiplexed video. The multiplexer operation is independent from the rest of the tool. To multiplex video and audio files, open the Decode Configure Audio/Mux dialog, Browse or enter your elementary source files and multiplexed output file, set up the multiplexing format using the Mux button, and depress the Run button. The audio files can be compressed MPEG, AC3 or DTS files. The multiplexing format can be Program, System or Transport. 56

59 Configure Mux This dialog controls the transport, program, or system multiplexing of the requested files Multiplexed Bit-rate This dialog controls the output bit-rate of the multiplexed file. You can select the Custom radio button and enter a specific bit-rate or select the Minimum to cause the multiplexer to use the lowest bit-rate that will accommodate the video, audio and multiplexing overhead data. Note that if Custom is selected, you need to assure that your entered bit-rate is high enough to handle the multiplexing overhead, the elementary video, and the elementary audio Multiplex Type This dialog selects the specification that is used to multiplex the files. VideoCD 57

60 Video CD multiplexing is used for creating Video CDs. Video CDs use 352 x 240 sized video encoded at a bit rate of 1.159mbps. MPEG1 System MPEG-1 System multiplexing is usually (but not always) used for multiplexing MPEG-1 elementary streams. MPEG2 Program MPEG-2 Program multiplexing is usually (but not always) used for multiplexing MPEG-2 elementary streams. MPEG2 Program multiplexing can be used for DVD creation. MPEG2 Transport MPEG-2 Transport multiplexing is usually (but not always) used for multiplexing MPEG-2 elementary streams. MPEG2 Transport multiplexing is usually used for High Definition streams. Variable Bit-Rate The Variable Bit-Rate switch, if checked, prevents the multiplexer from adding stuffing bytes during the multiplexing process. Stuffing is used to maintain a constant bit-rate. 58

61 Stream Terminations These options allow you to make the output multiplexed stream duration the same as the video or audio components. Note that the stream terminations option is not enabled in the Transport type. The Transport multiplexer multiplexes until the entire video and audio streams have been multiplexed. Terminate stream at end of video Checking the Terminate Stream at end of Video will cause the multiplexer to add the end of sequence header to the output stream and close the output stream after the last video data has been multiplexed into the system or program stream. Remaining audio data will not be included in the multiplexed stream. Note that if both the Terminate Stream at end of Video and the Terminate Stream at end of Audio options are checked, the Multiplexer will add the end of sequence header to the output stream and close the stream at the end of the shortest (in time) video or audio stream. Terminate stream at end of audio Checking the Terminate Stream at end of Audio will cause the multiplexer to add the end of sequence header to the output stream and close the output stream after the last audio data has been multiplexed into the system or program stream. Remaining video data will not be included in the multiplexed stream. Note that if both the Terminate Stream at end of Video and the Terminate Stream at end of Audio options are checked, the Multiplexer will add the end of sequence header to the output stream and close the stream at the end of the shortest (in time) video or audio stream Starting Delay These edit boxes allow you to add a starting delay to the video, audio, or system stream time stamps. This supports decoders that have different video or audio startup delays. Note that the System delay is not supported in the Transport Mode. The starting delay numbers affect the offsets for the MPEG-1 System Clock Reference or the MPEG-2 Program Clock Reference. This Clock Reference starts at time 0 and increments every 1/90,000 of a second for MPEG-1 or 1/27,000,000 of a second for MPEG-2. The number entered in this box is converted into units of 1/90,000 for MPEG-1 or 1/27,000,000 for MPEG-2 and added to each system time stamp as it is written to the stream 59

62 Transport Settings These edit boxes allow you to enter the desired PIDs (Program Identifications) for the video, audio, and system components as well as the PCR interval. Valid PID values are from 16 through The PCR (program clock reference or timing signal) PID is usually the same as the Video PID. The Audio PID should be different than the video PID. The PMT (Program Map Table) PID is usually one less than the video PID. And the PCR interval is usually 90 ms. These transport settings are only enabled with transport multiplex type Pack Structure These edit boxes are used to configure the multiplexing structures for Program or System stream multiplexing. The Bytes per Packet controls the number of bytes that will be stuffed into each packet. Often the Bytes per Packet corresponds to the disk sector size, IE 2048 for hard drives and 2324 for VideoCDs. The Packets per Pack controls the number of packets that follow each pack header. This number is usually set to 1. The System Header Each Packet switch causes the multiplexer to include a system header at the start of every packet. This box is normally unchecked as most decoders ignore the System Header information. These options are disabled when the transport multiplex type is selected Timing These check boxes allow you to disable the video and/or audio time stamps. These options are disabled when the transport multiplex type is selected. 60

63 8.5 Decode Controls The decode control section controls the decoding process. The following buttons are located in the control section: Run Pause Step Stop Analyze Run The Run button starts or resumes decoding of the MPEG stream and the storing the decoded frames onto the disk (if enabled). The decoder will decode the file entered in the MPEG File to Decode or Analyze Edit box. The decoder will decode until the number of frames selected (in the Decode Configure menu) has been reached, the file has been completely decoded, or the Pause or Stop buttons are depressed Pause The Pause button temporarily pauses the decoding process Step The Step button causes the decoder to decode the next frame from the MPEG file and to enter the Pause state. When decoding video consisting of field encoding, the step button needs to be depressed three times at the start of the video for the proper field to make it through the decoding pipeline and into the decode view windows Stop The Stop button stops the decode operation. All files used for the storage of the elementary elements and statistics are closed Analyze The Analyze button causes the decoder to analyze the selected file and displayed the identified stream properties in a dialog and to a log file. Depressing the Analyze button causes the decoder 61

64 to examine the first dozen or so frames in the stream and then the final frames in the stream. The starting and ending time code is used for calculating the stream length. The following is a sample Analysis dialog: The analysis is also saved into a text file called Decode.Log located in the directory where the source is found. The following is an example of the Decode.Log file: ******************************************* Decoding MPEG File:J:\ABatchTest\FlwTranFixed.Mpg PixelTools Analysis of: J:\ABatchTest\FlwTran.mpg 62

65 ********************************** Transport Stream PMT PID:16 ********************************** Elementary Video Stream MPEG2 Closed Captioning Types Detected: CEA-708 DTV CEA-708 NTSC CEA-708 DTV and NTSC Bitrate: m Frame Size: 720 x 480 Frame Rate: File Length: 0.10 secs Starting Time Code: 00:00:00:01 Ending Time Code: 00:00:00:01 Drop Frame: 0 Starting Vbv Delay: Ending Vbv Delay :24079 AspectRatio: 4:3 Buffer: 112 Sequence Extension Profile: Main Level: Main ProgressiveSequence:Yes Chroma Format: 4:2:0 Low Delay: No Picture Extension DC Precision: 8 Bits Picture Structure: Frame 63

66 Top Field First: 0.00 Repeat Field: 0.00 Alternate Scan: 0 Progressive Frame: 0.33 Picture Rate: Display Sequence Extension Format: NTSC Colors: SMPTE 170M Transfer: SMPTE 170M Matrix: SMPTE 170M Pan and Scan HorizontalDisplay: 720 VerticalDisplay: 480 ********************************** Elementary Audio Stream Bitrate: 256 Sample Frequency: 44.1 khz Emphasis: None Layer: Two Mode: Stereo Original: No Copyright: No Error Protection: No Private Bit: No ********************************** The file analysis process can also be run in the batch mode using the z argument. Refer to the Decode Batch section for details. Why? When debugging MPEG systems, it is often extremely useful to compare the analysis of a working stream with the analysis of a non-working stream. The analysis process takes about a second and often saves hour of work. Also, there are often cases where a set of MPEG files need to be verified for proper encoding parameters. The decode process can be run in the batch mode to analyze the set of files by comparing the resulting Decode.Log files. 64

67 8.6 Decode Views The Decode View Window can be opened by depressing the de speed button at the top of the tool Decode View Window This button opens a window that displays the decoded frames in 24-bit color. This button exists in both the Frame View and Field View Property Pages The following options are available in this menu: Display Motion Vectors Block Info Text Overlays Why? 65

68 The decode window permits detailed visual analysis, frame by frame, of the source MPEG. Expert- Caption decodes each frame with full floating point accuracy assuring that any artifacts in the decode window are due to the encoding and not the decoding Decode View Window Overlays These options control the display options and overlay capabilities of all of the decode windows Display The Display menu enables zoom options of the decode view Motion Vectors The Motion Vector menu can be selected to cause the color coded motion vectors to be overlaid on the display image. The color coding is identical to that of the Motion Vectors in the encode section Block Info The Block Info menu can be selected to cause the color coded encoding block decision types and the quantization values to be overlaid on the display image The block color coding is identical to that of the Motion Vectors in the encode section. The quantization values, if enabled, are red if the value is new IE included in the stream or black if it is the same value as before IE not included in the stream Text Overlays The Text Overlays menu enables Closed Captioning and Watermarking overlays to be added over the display image. The Closed Captioning option causes any closed captioning text detected to be overlaid over the display image as it is decoded from the video user_data. 66

69 The Watermarking option causes any PixelTools Escort watermarking text messages to be overlaid on the display image. The CEA-608 and CEA-708 closed captioning text can be added to the decoded window display using the Text Overlays Closed Captioning Options. The Field1 option displays all Field1 captions that have been added via the CEA-608, ATSC, DVD, or CEA-708 NTSC standards. The text appears with a greenish color above the Field2 text. The Field2 option displays all Field2 captions that have been added via the CEA-608, ATSC, DVD, or CEA-708 NTSC standards. The text appears with a reddish color below the Field1 text. The CEA-708 option displays all of the CEA-708 captions that are written to windows 1 through 8.. The text appears with a bluish color below the Field2 text. 67

70 8.6.3 Decode Statistics The Decode Statistics window displays a text output of items selected in the View menu. Decoding errors or problems are reported in this window no matter what view options are enabled. Menu items allow the display of MPEG stream items from the top system layer to the video macroblock layer. All data displayed in the Decode Statistics window is also stored in the Decode.log file. Digital Closed Captioning This option causes the decoder to examine the user_data found in the video and translates any closed captioning data into ASCII text and CEA-608 control codes and displayed in the Statistics window. Analog Closed Captioning This option causes the decoder to decode the closed captioning trace stored in the extra 32 black lines on the top of each frame. The two non zero characters, if found in each frame, are translated into ASCII text and CEA-608 control codes and displayed in the Statistics window. 68

71 The Decode Statistics can output information regarding the following stream headers: System Statistics Audio Lengths Displays the length of the audio packets as they are encountered in the stream. Transport PCRs Displays the Program Clock References found in the Transport stream (which is a count of the 27 MHz clock) and converts the clock values to actual seconds. Pack Headers Displays the pack headers, their contents, and their location as they are encountered in the stream. Pack headers are used in Program and System streams to indicate the multiplexed bit-rate of the stream. System Headers Displays the system headers, their contents, and their location as they are encountered in the stream. System headers are used in Program and System streams to indicate constrained parameters, video lock, and other system related parameters. Packet Headers Displays the packet headers, their contents, and their location as they are encountered in the stream. The packet headers are located at the start of each video and audio packet. They contain the time codes (Decode Time Stamps and Presentation Time Stamps) for the start of the video frame or audio chunk contained within. The packet header display also prints out the theoretical offset time computed by the specific file offset and the stream bit-rate. This is useful for debugging program stream timing or buffer issues Video Statistics Start Codes Displays all video start codes as they are encountered in the stream and outputs their stream byte offset. 69

72 Sequence Extensions Extracts all Sequence Extensions and displays their types and the values of the contents. Sequence Extensions are used to contain MPEG-2 specific data. Telecine Flags Extracts the MPEG-2 flags from sequence extensions that indicate the telecine sequencing for the current MPEG. The format of the output is: Frame: 6 prog_seq:0 prog_frame:0 topfirst:1 repeatfirst:0 Frame: 8 prog_seq:0 prog_frame:0 topfirst:0 repeatfirst:1 Frame: 1 prog_seq:0 prog_frame:0 topfirst:1 repeatfirst:1 User Data Displays the actual user data in hex bytes that is found after the user_data headers. Also displays the type of user data IE Gop, Picture, or Sequence user_data Closed Captioning This cause the decoder to examine the user_data found in the video and translates any closed captioning data into ASCII text and CEA-608 or CEA-708 control codes. 70

73 The following data can be selected individually or all at once: CEA Field1 CEA Field2 This format includes two CC bytes per frame in display order. The data consists of the basic CEA- 608 format which starts with The Field1 and/or Field2 options result in the selected field(s) being extracted and translated into the statistics window. This selection also decodes the two NTSC CC bytes of CEA-708 data or the ATSC/CEA-608 CC bytes in transmission order. The CEA-708 and CEA-608/ATSCdata starts with DVD Field1 DVD Field2 This format includes 20 CC bytes per GOP in transmission order. The data consists of a sequence of CEA-608 data byte pairs which starts with f8. The Field1 and/or Field2 options result in the selected field(s) being extracted and translated into the statistics window. CEA-708 All This format includes 18 CC bytes per picture in transmission order. The data displayed via this option is the 708 control codes and ASCII text for all 8 possible 708 windows. This option does not display the two 708 NTSC byte pairs per frame. (These are displayed using the CEA Field1 and CEA Field2 options.) 71

74 CEA-708 Window1 ->Window8 These options display only the control codes and ASCII text for the selected CEA-708 window. CEA-708 Raw Frames This option extracts and prints out the 708 packets in each frame along with the Frame number, Frame type [I P or B], packet increment number, packet length, and number of data bytes. This print out is extremely useful in debugging CEA-708 issues. As sample print out is listed below: Frame:16 B[2] Inc:0 PktSize:6 DataBytes:10 2a 8c fe f Frame:20 P[3] Inc:2 PktSize:3 DataBytes: Frame:20 P[3] Inc:3 PktSize:5 DataBytes: a Frame:19 B[5] Inc:1 PktSize:2 DataBytes:2 22 0d 03 Sequence Headers Displays the contents of sequence headers as they are encounters. This information contains information essential for proper decoding such as frame width, height, bit-rate and buffer. Group of Pictures Displays the contents of GOP headers as they are encounters. This information includes the time code of the I frame within the GOP VBV Buffer Displays the reported vbv_buffer delay at the start of each frame. Also displays the actual buffer fullness of the target decoder. This is useful for identifying and locating buffer underflows and overflows. Pictures Displays the contents of picture headers as they are encounters. This information includes the picture type (I, P of B), temporal reference (number of frames from the start of the GOP) and the vbv_delay. AvgMQuant Displays the average MQuant of the current frame in addition to the running average of the entire video to this point. This very useful when optimizing encoding quality. The format of the display is: 72

75 Avg MQuant per frame: per sequence: Avg MQuant per frame: per sequence: Decode AVC The contents of certain AVC (H.264) stream headers can be displayed using the Decode Statistics View AVC options. The following headers can be decoded: Network Adaptation Layer (NAL) Supplemental Enhancement Info (SEI) Sequence Parameter Set (SPS) Picture Parameter Set (PPS) Audio Statistics Start Codes Displays the MPEG audio start codes and their offsets within the steam as they are encountered Header 73

76 Displays the MPEG audio frame headers and their offsets within the steam as they are encountered. The frame headers contain the bit-rate, layer, and encoding flags of the audio stream 74

77 75

78 9 Monitor Controls The monitor section, located in the center of Expert-Caption, provides information about the current encoding or decoding process and information about Expert-Caption itself. 9.1 Display Window The display window displays the latest decoded frame or the latest frame that is input to the encoder. The frames are displayed in full 24-bit color. Closing the encoder monitor display (using the Encode Views Property Sheet) will speed up the decoding or encoding process. 9.2 Status Line The text status line, located below the monitor window, lists the status of the current encoding and/or decoding process. 9.3 Progress Time Code / Percent Done The window below the status line displays either the time code of the frame being encoded or the percent completion of the current encoding process. The time code or percent completion mode is selected using the TC or % buttons. It should be noted that the percent completion of the encoder process will remain at 0 if the number of frames to be encoded is set to ALL. This is because the encoder does not know how many frames are to be encoded from this and other directories. 76

79 77

80 10 Info The Information region contains buttons that display information about Expert-Caption Help The Help button brings up an overall table of contents of the on line help document About The About button brings up information about the specific product and its manufacturer. Depressing the About button first brings up a dialog that lists properties detected in your CPU. Of particular interest are the number of processors and the SSE2 capability. Both SSE2 capability and number of processors greater than one will accelerate the encoding speed. Pressing the OK button on this system information dialog will cause a product information dialog to come up. Of particular interest is the release version number. 78

81 79

82 11 Batch Mode Operation Expert-Caption can be operated from the DOS command line or from within a batch file Decoding MPEG File Decoding Expert-Caption can be caused to decode a file using the d option. To decode a file, copy a file containing the appropriate control parameters into the ExpCap.ini file in the directory where Expert-Caption is launched. Run the decoder using the syntax Merge d Source.mpg Where Source.mpg is the source file to be decoded MPEG File Analyzing Expert-Caption can be caused to analyze a file using the z option. decoder using the syntax To analyze a file, run the ExpCap z Source.mpg Where Source.mpg is the source file to be analyzed. Upon completion, Expert-Caption will display a dialog box containing the MPEG properties of the source file. Also, Expert-Caption will store a plain text version of the analysis in the Decode.Log file located in the directory where Source.mpg is located Closed Captioning Insertion Expert-Caption can be used insert captioning into one or more files using the batch mode. Set up the boot ExpCap.ini file with your desired captioning format. You can use - as the.scc file name to cause Expert-Caption to look for a.scc file with the same file prefix as your source file. You can use - as the Fixed File output to cause Expert-Caption to use the elementary output stream with the same file prefix as your source file but with Fixed.vbs as the output elementary file suffix. The multiplexed output stream will have the same file prefix as your source file but with the Fixed.mpg suffix. Use the standard decode file batch call to execute this function. ExpCap d Source.mpg 80

83 81

84 82

85 11.2 SCC File Usage A Scenarist Closed Captioning file (.SCC) can be used to cause closed captioning data to be inserted during encoding. The file can be used as is but it must be renamed Expert.edl and must be located in the directory where you encoding output is being stored. The.SCC file includes time codes and the formatted closed captioning data that will be inserted at these time codes. A typical.scc file looks like: Scenarist_SCC V1.0 00:00:08:22 94ae 00:00:08: ae 94d0 94d0 94ce 94ce d9 4fd5 204c 4fd e 946e c84f cd45 20c1 d0d0 4c49 c1ce 4345 d32c 942c 942c 942f 942f 00:00:09:27 94ae 94ae 83

86 12 DDL File Usage Expert-Caption encoder can make changes throughout an existing MPEG file during decoding through the use of a Decoding Decision List (DDL) File. The DDL file must be named Expert.ddl and must be located in the directory where the fixed MPEG file is being stored. The supported DDL commands are functionally identical to the corresponding EDL commands. The syntax of the commands is identical to that of the EDL file. The following commands are supported: 12.1 HEXUSERDATA This DDL command enables up to 128 bytes of user data to be added at the start of the selected frame. The data can be of any type. The user is responsible for making sure that the data is of the proper format. The specific location of the user_data ie picture_headers or GOP_headers is controlled via the Optimize Encode Configure Line 21 Closed Captioning Format selection. Refer to the Closed Captioning Format section for more details. Example: FRAME 1038 HEXUSERDATA 04 ff 29 d0 55 9f FRAME 1039 HEXUSERDATA 05 ff 39 d4 22 9f CLOSEDCAPTION This DDL command causes the encoder to reformat the very next line of text into closed caption data user_data and add the text, two characters at a time, to the picture user_data starting with the selected frame. The user is responsible to keep the closed caption command enough frames apart so that the next closed caption command does not occur before the previous closed command has finished parsing the characters in the stream. The specific location of the user_data ie. picture_headers or GOP_headers and its formatting is controlled via the Optimize Encode Configure Line 21 Closed Captioning Format selection. Refer to the Closed Captioning Format section for more details. Example: 84

87 FRAME 1038 CLOSEDCAPTION The little brown cow jumped over FRAME 1146 CLOSEDCAPTION the moon! 12.3 PANSCANOFFSET This DDL command causes the encoder to add the following horizontal and vertical offsets to the Pan and Scan frame offset associated with the selected frame. This can be used to move the smaller Pan and Scan frame within the larger video display window on a frame by frame basis. This assumes that the Pan and Scan Frame size has been correctly set. This also assumes that the offsets do not cause the Pan and Scan frame to move off of the larger video display window. The first number after the command represents the horizontal offset for that frame. The second number represents the vertical offset for that frame. Example: FRAME 1038 PANSCANOFFSET FRAME 1039 PANSCANOFFSET FRAME 1040 PANSCANOFFSET The following comments are provided with the default ddl file: # DDL File Template # The DDL file must be named EXPERT.DDL # EXPERT.DDL Must be in the directory where the fixed MPEG is stored # All DDL Items are Preceded by the frame number # at which the option starts or ends # The Frame Number is on a line of its own # Use no commas to separate items # # DDL Options: 85

88 # HEXUSERDATA 1b f3 02 {adds up to 128 bytes to be added to frame USER_DATA} # CLOSEDCAPTION TextString on next line {Converts Text to Closed Captioning User Data} # PANSCANOFFSET HorizontalOffset VerticalOffset {Adds signed offset to frame PanScan offset} # # The Expert.ddl can be a closed caption.scc file # The.scc file is formatted with one closed caption display line in one # line in the.ddl file. The line starts with the time code that the cc line # is displayed followed by the formatted closed captioning characters such as: # 00:00:03: a2 94a c861 76e ce e9e3 e520 c461 79a1 # # 12.4 SCC or CAP File Usage Any legal closed captioning.scc file or.cap file can be used to cause closed captioning data to be inserted into an existing MPEG stream. The specific location of the user_data ie picture_headers or GOP_headers and its formatting is controlled via the Optimize Encode Configure Line 21 Closed Captioning Format selection. Refer to the Closed Captioning Format section for more details. 86

89 87

90 13 Initialization Files Initialization files contain parameters that are used for encoding. Careful management of these parameter files can save days of work by assuring that the appropriate settings have been used for the process and documenting the exact settings. The initialization files are text files and can be edited with any text editor. This practice however, is not recommended as the structure of the initialization file is not well organized or well documented. Many ini parameters are not used in current products but are preserved to support backward compatibility. Ini files are best created and updated via the Parameter Load and Save functions Default Initialization Files Expert-Caption, when it is launched, attempts to read an initialization file called ExpCap.Ini from the directory where it was launched. If that file cannot be located, it loads its built in MPEG-2 defaults and issues a warning message. It is possible to streamline repetitive encoding operations by copying the appropriate parameter file to MpgeRep.Ini prior to launching the encoder. The application will be configured per your desired settings upon launch using this technique Loading and Saving Initialization Files Encoding parameters for the current encode operation can be saved with the Encoding Parameter Save button and used in the future for similar encoding situations. 88

91 89

92 14 Solutions The versatility of Expert-Caption provides efficient solutions to many MPEG requirements. It is not always obvious how certain solutions can be accomplished using the above described buttons and controls. We will detail below specific settings and controls that can accomplish popular MPEG tasks Closed Captioning Insertion PixelTools products can add closed captioning during MPEG encoding or to an existing MPEG stream without re-encoding. PixelTools products also extract closed captioning from an MPEG stream and overly the appropriate text string over the video. The closed captioning source can be the actual closed captioning text strings along with their associated time code, formatted captioning codes (as in a.scc file) or formatted user data. PixelTools products will convert the text strings to CEA-608A, ATSC or DVD formatted data in the appropriate user data field. PixelTools products can also add any pre-formatted data to user data fields of existing or new MPEG files. With this approach, all closed captioning specifications that use video user data can be performed. (You need to convert your closed captioning text strings to the appropriate format.) Adding Captioning During Decoding (Expert-Caption) Closed captioning can be added to an existing elementary stream. This is accomplished using the PixelTools decode functionality in conjunction with a Decision List (ddl) text file. The DDL file contains the closed captioning text string and the first frame number or time code where the text string will appear. The DDL file must be named Expert.ddl and must be located in the directory where your output fixed MPEG file is being stored. To enable processing of the Expert.ddl file, check the Decode Function Enable Configure Fix Stream Use Commands in DDL file option and enter the name of the file the will include the closed captions in the Save Fixed File edit box. The CLOSEDCAPTION command in the Expert.ddl file specifies the closed captioning text string. An example of the format of the closed captioning is presented in the top of the sample Expert.ddl provided with the product. The formatted closed captioning data is added to the appropriate frame user_data fields as requested in the Expert.ddl file after depressing the Control Decode Run button with your source file in the Decode MPEG File to Decode or Analyze edit box. You can speed up the process by depressing the Speed button with the slashed monitor to disable decode display. 90

93 The ddl file format is similar to that of the edl file format presented above. In addition to the text strings supported with the CLOSEDCAPTION command, any formatted user data can be inserted via the HEXUSERDATA command. Also, the.scc closed caption format is supported. (Be sure to rename the.scc file as Expert.ddl) Text to Closed Captioning The following is an example of the Edl or Ddl closed captioning file commands: FRAME 6 CLOSEDCAPTION This is a demo of the FRAME 36 CLOSEDCAPTION Closed Captioning Insertion FRAME 66 CLOSEDCAPTION From PixelTools SCC File to Closed Captioning The closed captioning EDL file can also be of popular closed caption file format (.scc) which consists of the time code followed on the same line by a sequence of sets of 4 HEX codes representing the actual closed captioning data that will be added to the video user_data. The following is an example of the Edl or ddl with formatted closed captioning command: Scenarist_SCC V1.0 00:00:00: a2 94a e973 20e e5 6def 20ef e620 00:00:01: a2 94a ec ef73 e f4e9 ef6e e96e :00:02: a2 94a f2 ef6d 20d0 e9f8 e5ec 54ef efec HEX User Data to Closed Captioning Also, any pre-formatted closed captioning data (using any closed captioning standard) can be added via the HEXUSERDATA edl command. Just add the frame number or time code of the frame user data and the actual user data, up to 72 characters long, after the HEXUSERDATA command.. An example of the format of the hex user data is presented in the top of the sample Expert.edl provided 91

94 with the product. The following is an example of the Edl or ddl raw user data file commands: Frame 10 HEXUSERDATA a aa aa aa 80 Frame 11 HEXUSERDATA a aa aa aa 80 Frame 12 HEXUSERDATA a aa aa aa 80 Frame 13 HEXUSERDATA e9 02 0a aa aa aa Closed Captioning Format The Closed Captioning Format selection controls the actual formatting of the text or SCC EDL and DDL commands into the MPEG user data and controls the specific user data locations. The Format selection control is located in the Encode section of Expert-Caption or ExpertHD in the Optimize Encode Configure Line 21 property page. Note that the insertion of closed captioning data into an existing stream using the Decode Fix process utilizes the Closed Captioning Format set in the Encode configuration. The following Closed Captioning format options are available: CEA 608 (default) This option causes up to two closed captioning characters to be inserted into picture header user data fields in transmission order. Analyzing the actual user data using the Expert-Caption Decode Statistics window will show Picture user data that will start with the Hex values: ATSC / CEA 608 This option causes up to two closed captioning characters to be inserted into picture header user data fields in display order. The data formatted per CEA-608 and is wrapped in a SCTE-21 and ATSC header. Analyzing the actual user data using the Expert-Caption Decode Statistics window will show Picture user data that will start with the Hex values:

95 DVD This option causes up to 30 closed captioning characters to be inserted into GOP header user data fields. The first set of user data will be inserted at the first GOP at or after the Frame number or time code set in the EDL or DDL file. The remaining user data (over the 30 character per GOP limit) will be inserted in subsequent GOP headers. Analyzing the actual user data using the Expert-Caption Decode Statistics window will show user data at EVERY GOP header that will start with the Hex values: f8 9e. The GOP headers will contain the closed captioning data or will contain NULL data Extracting and Displaying closed captioning with Expert-Caption The Expert-Caption Decoder can extract and display closed captioning information in several formats. Hex User Data Opening the Decode Statistics window (by depressing the St speed button) and selecting the View Video Statistics User Data will display the hex values of all user data as it is encountered in the stream. Note that the data will be encountered in transmission order not display order. All of the text displayed in this file will be stored in the Decode.Log file. Closed Captioning Data Opening the Decode Statistics window (by depressing the St speed button) and selecting the View Video Statistics Closed Captioning will display the extracted closed captioning data encountered in the stream. The data will be the ASCII closed captioning text and the closed captioning control codes. Note that the data will be encountered in transmission order not display order. All of the text displayed in this file will be stored in the Decode.Log file. 93

96 Closed Captioning Overlay Opening the Decode Display window using the De speed button will bring up a window that will display all of the frames in display order. Selecting the Text Overlays Closed Captioning option will cause all closed captioning text detected in the stream to be over laid on the decoded video. The CEA-608 and CEA-708 closed captioning text can be added to the decoded window display using the Text Overlays Closed Captioning Options. The Field1 option displays all Field1 captions that have been added via the CEA-608, ATSC, DVD, or CEA-708 NTSC standards. The text appears with a greenish color above the Field2 text. The Field2 option displays all Field2 captions that have been added via the CEA-608, ATSC, DVD, or CEA-708 NTSC standards. The text appears with a reddish color below the Field1 text. 94

97 The CEA-708 option displays all of the CEA-708 captions that are written to windows 1 through 8. The text appears with a bluish color below the Field2 text CGMS Insertion PixelTools products can add CGMS and APS copy protection flags during encoding or to an existing stream without re-encoding. CGMS and APS flags, embedded in an MPEG stream, instruct recording equipment whether to permit copying of the MPEG or not. These flags are inserted in the MPEG elementary user data CGMS and APS flag Insertion During Encoding To enable the addition of CGMS and APS data while encoding a file, depress the Optimize Encode Configure Line 21 tab. Check the Add XDS Flags option. Then select the appropriate CGMS flag option: Copy Freely, Copy Once, or Copy never. Select the appropriate APS flag: off, Type 1, Type 2, or Type 3. Depress the Apply button. Click on image to see full screen. During encoding of your source video, your selected flags will be added to the elementary stream at appropriate intervals CGMS and APS Flags Insertion to an Existing Stream (Expert-Caption) To enable the addition of CGMS and APS to an existing file, depress the Decode Function Enable Configure Fix Stream tab. Check the Insert XDS flags. Then select the appropriate CGMS flag option: Copy Freely, Copy Once, or Copy never. Select the appropriate APS flag: off, Type 1, Type 2, or Type 3. Enter or Browse the resultant file (containing the flags) in the Save Fixed File edit box. Depress the Apply and OK button. Select the elementary MPEG file you wish to add the flags to in the MPEG File to Decode or Analyze Edit box using the Browse button. Depress the Control Decode Run process to start the insertion process. You can speed up the insertion process by depressing the Speed Button with the line through the monitor to turn off the display during encoding. 95

98 15 Source Material Included in the Install CD The Source directory contains sample image sequences so you can verify the MPEG encoder over a wide range of image sizes and formats. The frames were taken from public domain and some are corrupted, so be sure to monitor the Encoding Input View window... to verify that the problem is with the source and not the encoder! The frames are not of high quality and are here for encoding practice and encoding compliance and not to demonstrate high quality encoding. (If you have better quality content that you are willing and able to release for our demo, let us know. We will use Expert-Caption to brand your logo on each of the frames). The source frames are arranged in directories according to the frame type (BMP, TGA, etc). Inside each frame type directory, there are one or more directories arranged by frame size. And inside each frame size directory, there are one or more frame sequences (totaling 3-5 seconds in length). The YUV frame type contains the majority of frame sequences. Within each directory containing a frame sequence, there exists an ini file that can be used to setup suitable encoding parameters to encode the sequence. The ini file will work for all PixelTools encoders. The MPEG compressed files, produced using these ini files, are stored in the MPEG directory on this CD. So to test the encoder on a frame type or frame size, select the parameter files from that directory, select the lowest sequence number of the first frame to encode, and depress the start button. The following frame sizes are included in the source subdirectories: D1_ x 480 pixel frames (both fields) for encoding into High Profile MPEG-2 or sub sampling into lower resolutions or chromas D1_ x 480 pixel frames (both fields) for encoding into Main Profile MPEG-2. The frames contain half as many chrominance as D1_422 HD1_ x 240 pixel frames (both fields) for encoding into Main Profile MPEG-2. A very popular format with half the pixels as D1_420 96

99 but still containing two fields SIF 352 x 240 pixel frames for encoding into MPEG-1. QSIF 176 x 120 pixel frames for encoding into low bit-rate MPEG-1. Very useful for Internet applications especially where the frame rate is decreased The Install CD also includes the MPEG Directory. The MPEG directory contains encoded samples of the source files encoded using various frame sizes and encoding parameters. The ini file used to create each MPEG stream is included in the MPEG directory. The MPEG directory also includes MPEG-1 system and MPEG-2 program multiplexed versions of each of the elementary streams. 97

100 16 Recommended Reading 16.1 MPEG Specifications The most accurate and complete information about the MPEG encoding and decoding process is listed in the ISO international specification. MPEG-1 is described ISO/IEC Systems ISO/IEC Video ISO/IEC Audio MPEG-2 is described in ISO/IEC Systems ISO/IEC Video ISO/IEC Audio The specifications can be purchased in the United States from American National Standards Institute (ANSI) in New York at You need about $500 available on you credit card for each of the sets of three volumes MPEG Video Compression Standard ISBN This is an excellent book that describes the theory and implementation of MPEG encoding. Although this text was written before the MPEG-2 standard was approved, it provides accurate information on both MPEG-1 and MPEG Digital Video: An Introduction to MPEG-2 ISBN Is another good supplemental book that fills in a lot of details not highlighted in MPEG Video Compression Standard Our home web site includes a highly acclaimed tutorial on how MPEG works. We also will include encoding tips for those difficult encoding situations. 98

101 This is the most complete resource for MPEG encoding information, specifications, and MPEG tools and products. 99

102 17 Glossary 17.1 MPEG-2 Video Definitions Summarized from ISO/IEC AC coefficient: Any DCT coefficient for which the frequency in one or both dimensions is nonzero. Big picture: A coded picture that would cause VBV buffer undertow. Big pictures can only occur in sequences where low delay is equal to I. "Skipped picture" is a term that is sometimes used to describe the same concept. B-field picture: A field structure B-Picture B-frame picture: A frame structure B-Picture B-picture; bidirectional predictive-coded picture: A picture that is coded using motion compensated prediction from past and/or future reference fields or frames. Backward motion vector. A motion vector that is used for motion compensation from a reference frame or reference field at a later time in display order Backward prediction: Prediction from the future reference frame (field). Bit-stream: An ordered series of bits that forms the coded representation of the data. Bitrate: The rate at which the coded bit-stream is delivered from the storage medium to the input of a decoder. Block: An 8-row by 8-column matrix of samples, or 64 - coefficients (source, quantized). Bottom field: One of two fields that comprise a frame. Each line of a bottom field is spatially located immediately below the corresponding line of the top field. Byte. Sequence of 8 bits Chrominance format. Defines the number of chrominance blocks in a macro block. Chrominance component: A matrix, block or single sample representing one of the two color signals related to the primary colors in the manner defined in the bit-stream. The symbols used for the chrominance signals arc Cr and Cb. Coded B-frame: A B-frame picture or a pair of B-field pictures. Coded frame: A coded frame is a coded 1-frame, a coded P-frame or a coded B-frame. Coded I-frame: An 1-frame picture or a pair of field pictures, where the first field picture is an (picture and the second field picture is an I-picture or a P-picture. Coded P-frame: A P-frame picture or a pair of P-field pictures. Coded picture: A coded picture is made of a picture header, the optional extensions unmediated following it and the following picture data. A coded picture may be a coded frame or a coded field. 100

103 Coded order: The order in which the pictures are transmitted and decoded. This order is not necessarily the same as the display order. Coding parameters: The set of user-definable parameters that characterize a coded video bit stream. Bit-streams are characterized by coding parameters. Decoders are characterized by the bit-streams that they arc capable of decoding. Component. A matrix, block or single sample from one of the three matrices (luminance and two chrominance) that make up a picture Compression. Reduction in the number of bits used to represent an item of data. Constant bit-rate coded video: A coded video bit-stream with a constant bit-rate. Constant bit-rate: Operation where the bit-rate is constant from start to finish of the coded bit stream. DC coefficient: The DCT coefficient for which the frequency is zero in both dimensions. DCT coefficient. The amplitude of a specific cosine basis function Decoder input buffer: The First-In First-Out (FIFO) buffer specified in the video buffering verifier. Decoder: An embodiment of a decoding process. Decoding (process): The process defined in ISOAEC that reads an input coded bit-stream and produces decoded pictures. Dequantisation: The process of re-scaling the quantized DCT coefficients after their representation in the hit-stream has been decoded and before they are presented to the inverse DCT. Digital storage media: DSM: A digital storage or transmission device or system. Discrete cosine transform: DCT is either the forward discrete cosine transform or the inverse discrete cosine transform. DCT is an invertible, discrete orthogonal transformation. The inverse DCT is defined in Annex A ISOIEC Display aspect ratio: The ratio height/width (in SI units) of the intended display. Display order. The order in which the decoded pictures are displayed. Normally this is the same order in which they were presented at the input of the encoder. Display process: The (non-normative) process by which reconstructed frames are displayed. Dual-prime prediction: A prediction mode in which two forward field-based predictions are averaged. The predicted block size is 16 x 16 luminance samples. Dual-prime prediction is only used in interlaced P-pictures. Editing: The process by which one or more coded bit-streams arc manipulated to produce a new coded bit-stream. Conforming edited bit-streams must meet the requirements defined in ISOIIEC Encoder. An embodiment of an encoding process 101

104 Encoding (process): A process, not specified in ISO/IEC , which reads a stream of input pictures and produces a valid coded bit-stream as defined in ISO/IEC Field: For an interlaced video signal, a "field" is the assembly of alternate lines of a frame. Therefore an interlaced frame is composed of two fields, a top field and a bottom field. Field-based prediction: A prediction mode using only one field of the reference frame. The predicted block size is 16 x 16 luminance samples. Field-based prediction is not used in progressive frames. Field period: The reciprocal of twice the frame rate. Field picture: field structure picture A field structure picture is a coded picture with picture structure is equal to "Top field" or "Bottom field. Flag. A one-bit integer variable that may take one of only two values (hero and one) Forbidden: The term "forbidden" when used in the clauses defining the coded bit-stream indicates that the value shall never be used. This is usually to avoid emulation of start codes. Forced updating: The process by which macroblocks are intra-coded from time to time to ensure that mismatch errors between the inverse DCT processes in encoders and decoders cannot build up excessively. Forward motion vector: A motion vector that is used for motion compensation from a reference frame or reference field at an earlier time in display order. Forward prediction. Prediction from the post reference frame (field) Frame: A frame contains lines of spatial information of a video signal. For progressive video, these lines contain samples starting from one time instant and continuing throw h successive lines to the bottom of the frame. For interlaced video, a frame consists of two fields, a top field and a bottom field. One of these fields will commence one field period later than the other. Frame-based prediction: A prediction mode using both fields of the reference frame. Frame period: The reciprocal of the frame rate. Frame picture; frame structure picture: A frame structure picture is a coded picture with picture structure is equal to' Frame". Frame rate: The rate at which frames are output from the decoding process. Future reference frame (field): A future reference frame (field) is a reference frame (field) that occurs at a later time than the current picture in display order. Frame reordering: The process of reordering the reconstructed frames when the coded order is different from the display order. Frame re-ordering occurs when 13-frames are present in a bit-stream. There is no frame reordering when decoding low delay bit-streams. Grow of pictures: A notion defined only in ISO/IEC It (MPEG-I Video). In ISO/IEC , a similar functionality can be achieved by the mean of inserting group of pictures headers. Header: A block of data in the coded bit-stream containing the coded representation of a number of data elements pertaining to the coded data that follow the header in the bit-stream. 102

105 Interlace: The property of conventional television frames where alternating lines of the frame represent different instances in time. In an interlaced frame, one of the fields is meant to be displayed first; this field is called the first field. The first field can be the top field or the bottom field of the frame. I-field picture: A field structure I-Picture. I-frame picture: A frame structure I-Picture. 1-picture; intra-coded picture: A picture coded using information only from itself. Infra coding: Coding of a macro block or picture that uses information only from that macro block or picture. Level: A defined set of constraints on the values that may be taken by the parameters of ISO/IEC within a particular profile. A profile may contain one or more levels. In a different context, level is the absolute value of a nonzero coefficient (see "run"). Luminance component: A matrix, block or single sample representing a monochrome representation of the signal and related to the primary colors in the manner defined in the bit stream. The symbol used for luminance is Y. Mbit. 1,000,000 bits. Macro block: The four S by 8 blocks of luminance data and the two (for 420 chrominance format), four (for 42:2 chrominance format) or eight (for 4:4:4 chrominance format) corresponding 8 by 8 blocks of chrominance data coming from a 16 by 16 section of the luminance component of the picture. Macro block is sometimes used to refer to the sample data and sometimes to the coded representation of the sample values and other data elements defined in the macro block header of the syntax defined in ISO/IEC The usage is clear from the context. Motion compensation: The use of motion vectors to improve the efficiency of the prediction of sample values. The prediction uses motion vectors to provide offsets into the past and/or future reference frames or reference fields containing previously decoded sample values that are used to form the prediction error Motion estimation: The process of estimating motion vectors during the encoding process. Motion vector: A two-dimensional vector used for motion compensation that provides an offset from the coordinate position in the current picture or field to the coordinates in a reference frame or reference field. Non-intra coding: Coding of a macro block or picture that uses information both from itself and from macroblocks and pictures occurring at other times. Opposite parity: The opposite panty of top is bottom, and vice versa. P-field picture: A field structure P-Picture. P-frame picture: A frame structure P-Picture. P-picture; predictive-coded picture: A picture that is coded using motion compensated prediction from past reference fields or frame Parameter. A variable within the syntax of ISO/IEC that may take one of a range of values. A variable that can take one of only two values is called a flag. Parity (of field): 'I he panty of a field can be top or bottom 103

106 Past reference free (field): A past reference frame (field) is a reference frame (field) that occurs at an earlier time than the current picture in display order. Picture: Source, coded or reconstructed image data A source or reconstructed picture consists of three rectangular matrices of "it numbers representing the luminance and two chrominance =s A 'coded picture' is defined in ISQ/IEC _ For use video, a is ides cal to a frame, while for interlaced video, a picture can refer to a frame, or the top field or the bottom field of the frame depending on the context. Picture data: In the VBV operations, picture data is defined as all the hits of the coded picture, all the headers) and user data immediately preceding it if an 'including any stuffing between them) and all the stuffing following it, up to (but not including the next start code, except in the case where the next start code is an end of sequence code, in which case it is included in the picture data. Prediction: The use of a predictor to provide an estimate of the sample value or data element currently being decoded. Prediction error: The difference between the actual value of a sample or data element and its predictor. Predictor: A linear combination of previously decoded sample values or data elements. Profile: A defined subset of the syntax of ISO/IEC Progressive: The property of film frames where all the samples of the frame represent the same instances in time. Quantisation matrix: A set of sixty-four 8-bit values used by the debutanizer. Quantised DCT coefficients: DCT coefficients before dehumanization. A variable length coded representation of quantized DCT coefficients is transmitted as pan of the coded video bit-stream. Quantiser scale: A scale factor coded in the bit-stream and used by the decoding process to scale the dequantisation. Reconstructed frame: A reconstructed frame consists of three rectangular matrices of 8-bit numbers representing the luminance and two chrominance signals A reconstructed frame is obtained by decoding a coded frame. Reconstructed picture: A reconstructed picture is obtained by decoding a coded picture A reconstructed picture is either a reconstructed frame when decoding a frame picture), or one field of a reconstructed frame (when decoding a field picture) If the coded picture is a field picture, then the reconstructed picture is the top field or the bottom field of the reconstructed frame. Reference field: A reference field is one field of a reconstructed Ironic. Reference fields arc used for forward and backward prediction when P-pictures and B-pictures are decoded. Note that when field P-pictures arc decoded prediction of the second field P-picture of a coded frame uses the first reconstructed field of the same coded frame as a reference field. Reference frame: A reference fume is a reconstructed frame that was coded in the form of a coded I-frame or a coded P-frame Reference frames are used for forward and backward prediction when P-pictures ad B-pictures are decoded. Reordering delay: A delay in the decoding process that is caused by frame re-ordering. Reserved: The term "reserved" when used in the clauses defining the coded bit-stream, indicates that the value may be used in the future for ISO/IEC defined extensions. 16 x 8 prediction: A prediction mode similar to field-based prediction but where the predicted block size is 16 x S luminance samples. 104

107 Run: The number of zero coefficients preceding a non-zero coefficient, in the scan order. The absolute value of the non-zero coefficient is called "level". Saturation: Limiting a value that exceeds a defined range by setting its value to the maximum or minimum of the range as appropriate. Skipped macro block: A macro block for which no data is encoded. Slice: A consecutive series of macroblocks which arc all located in the same horizontal row of macroblocks. Source; input: Term used to describe the video material or some of its attributes before encoding. Start codes (system and video): 32-bit codes embedded in that coded bit-stream that arc unique. They are used for several purposes including identifying some of the structures in the coding syntax. Stuffing (bits); stuffing (bytes): Code-words that may be inserted into the coded bit-stream that are discarded in the decoding process Their purpose is to increase the bit-rate of the stream which would otherwise be lower than the desired bit-rate. Temporal prediction. Prediction derived from reference frames or fields other than those defined as spatial prediction. Top field: One of two fields that comprise a frame. Each line of a top field is spatially located immediately above the corresponding line of the bottom field. Variable bit-rate: operation where the bit-rate varies with time during the decoding of a coded hit-stream Variable length coding (VLC): A reversible procedure for coding that assigns shorter code words to frequent events and longer code words to less frequent events. Video buffering verifier (VBV): A hypothetical decoder that is conceptually connected to the output of the encoder. Its purpose is to provide a constraint on the variability of the data rate that an encoder or editing process may produce. Video sequence: 'The highest syntactic structure of coded video bit-streams. It contains a series of one or more coded frames. Zigzag scanning order: A specific sequential ordering & of the DCT coefficients from (approximately) the lowest spatial frequency to the highest. 105

108 17.2 MPEG-2 System Definitions Summarized from ISO/IEC Access unit (system): A coded representation of a presentation unit, In the case of audio, an access unit is the coded representation of an audio frame In the case of video, an access unit includes all the coded data for a picture, and any stuffing that follows it, up to but not including the start of the next access unit. If a picture is not preceded by a group start code or a sequence header code the access unit begins with the picture start code If a picture is preceded by a group start code and/or a sequence header code, the access unit begins with the first byte of the first of these start codes If it is the last picture preceding a sequence end code in the bit-stream, all bytes between the last byte of the coded picture and the sequence end code (including the sequence end code) belong to the access unit. Bit-rate: The rate at which the compressed bit stream is delivered from the channel to the input of a decoder. Byte aligned: A bit in a coded bit stream is byte-aligned if its position is a multiple of 8-bits from the first bit in the stream. Channel: A digital medium that stores or transports an ISO/IEC I stream. Coded representation: A data element as represented in its encoded form. CRC: The Cyclic Redundancy Check to verify the correctness of data. Decoding time-stamp (DTS) system: A field that may be present in a PES packet header that indicates the time that an access unit is decoded in the system target decoder. Entitlement control message (ECM): Entitlement Control Messages are private conditional access information, which specify control words and possibly other, typically stream-specific, scrambling and/or control parameters. Entitlement management message (EMM): Entitlement Management Messages are private conditional access information, which specify the authorization levels or the services of specific decoders. They may be addressed to single decoders or groups of decoders. Elementary stream (ES) system: A generic term for one of the coded video, coded audio or other coded bit streams in PES packets. One elementary stream is carried in a sequence of PES packets with one and only one stream_id. Elementary Stream Clock Reference (ESCR) system: A time stamp in the PES Stream from which decoders of PFS streams may derive timing. Multiplexed stream system: A bit stream composed of 0 or more elementary streams combined in a manner that conforms to ISO/IEC Pack system: A pack consists of a pack header followed by zero or more packets. It is a layer in the system coding syntax described in of IISO\EC I. Packet data system: Contiguous bytes of data from an elementary stream present in a packet. Padding audio: A method to adjust the average length of an audio frame in time to the duration of the corresponding PCM samples, by conditionally adding a slot to the audio frame. 106

109 Payload: Payload refers to the bytes that follow the header bytes in a packet. For example, the payload of some Transport Stream packets includes a PES packet header and its PES packet data bytes, (it pointer field and PSI sections. or private data, but a PES packet payload consists of only PI'S packet data bytes. The Transport Stream packet header and adaptation fields arc not payload PES system: An abbreviation for Packetized Elementary Stream. PES packet system: The data structure used to carry elementary stream data, A PES packet consists of a PES packet header followed by a number of contiguous bytes from an elementary data stream. It is a layer in the system coding syntax described in 2 of ISO/IEC I PES packet header system: The leading fields in a PES packet up to and not including the PES packet data byte fields, where the stream is not a padding stream. In the case of a padding stream the PES packet header is similarly defined as the leading fields in a PES packet up to and not including padding byte fields, PES Stream system: A PES Stream consists of PES packets, all of whose payloads consist of data from a single elementary stream, and all of which have the same stream id. Specific semantic constraints apply. Presentation time-stamp (PTS) system: A field that may be present in a PES packet header that indicates the time that a presentation unit is presented in the system target decoder. Presentation unit (PU) system: A decoded Audio Access Unit or a decoded picture. Program system: A program is a collection of program elements. Program elements may be elementary streams. Program elements need not have any defined time base; those that do, have a common time base and are intended for synchronized presentation. Program Clock Reference (PCR) system: A time stamp in the Transport Stream from which decoder timing is derived. Program element system: A generic term for one of the elementary streams or other data streams that may be included in a program Program Specific Information (PSI) system: PSI consists of normative data which is necessary for the demultiplexing of Transport Streams and the successful regeneration of programs and is described in of ISO/IEC , An example of privately defined PSI data is the non-mandatory network information table. Scrambling system: The alteration of the characteristics of a video, audio or coded data stream in order to prevent unauthorized reception of the information in a clear form. This alteration is a specified process under the control of a conditional access system. Splicing system: The concatenation, performed on the system level, of two different elementary streams. The resulting system stream conforms totally to ISO/IEC I. The splice may result in discontinuities in tune-base, continuity counter, PSI, and decoding. Start codes system: 32-hit codes embedded in the coded bit stream. They are used for several purposes including identifying some of the layers in the coding syntax. Start codes consist of a 4-bit prefix (0x ) and an 8-bit stream_id as shown in Table 2-18 of I ISO/IEC Still picture: A coded still picture consists of a video sequence containing exactly one coded picture that is intracoded. This picture has an associated PTS and the presentation time of succeeding pictures, if any, is later than that of the still picture by at least two picture periods. 107

110 System header system: The system header is a data structure defined in ISO/IEC I that carries information summarizing the system characteristics of ISO/IEC I Program Stream. System Clock Reference (SCR) system: A time stamp in the Program Stream from which decoder timing is derived. System target decoder (STD) system: A hypothetical reference model of a decoding process used to define the semantics of an ISO/IEC I multiplexed bit stream. Time-stamp system: A term that indicates the time of a specific action such as the arrival of a byte or the presentation of a Presentation Unit. Transport Stream packet header system: The leading fields in a Transport Stream packet, up to and including the continuity counter field. Variable bitrate: An attribute of Transport Streams or Program Streams wherein the rate of arrival of bytes at the input to a decoder vanes with time. 108