GUIDE 10 challenges facing the broadcast subtitle community and how new technologies can help resolve them Subtitles are a valuable piece of the TV broadcast and streaming ecosystem, and whether they add to the enjoyment of viewing by the hearing impaired, or through language translation, they contribute in some way to the operators business success. The industry is enjoying new opportunities to monetise content on new platforms, and to new audiences, but in reaching digital viewers media operators have to adopt new practices and standards while, at the same time, reducing per channel production and transmission costs.
This guide highlights what Miranda sees as the top 10 challenges that operators and subtitling service companies face in the emerging digital media and proposes technology-assisted solutions which reduce the time and cost of subtitle production and distribution. The 10 challenges we will be focusing on are: 1 Reducing the time and cost of subtitle content creation and quality control 2 Converting broadcast subtitles for web and mobile streaming 3 Re-timing subtitle files for edited video content 4 Reducing the per-channel cost of dynamic time of air subtitle output 5 Encoding closed subtitles in HD video 6 Simplifying DVB subtitling across multiple channels 7 Treating subtitles as part of a file-based media workflow 8 Subtitle creation for live and news programming 9 Creating and managing Audio / Video Description content 10 Monitoring subtitle streams in a multi-channel broadcast facility Note: In this document, the term subtitles means both closed captions and subtitles.
1 Reducing the time and cost of subtitle content creation and quality control The upward trend in the requirements for both hard of hearing and language translated captioned and subtitled programing comes at a cost to the content production and delivery supply chain. Any technology or workflow efficiencies that reduce the time and costs of subtitle production are thus very welcome. The processes behind subtitle content creation and editing can be considered in two distinct areas subtitle editing and alignment, and subtitle quality control (QC). The first of these areas, the subtitle creation, editing, and alignment to the spoken dialogue is the most resource intensive and thus the most critical when looking for efficiencies. Traditional craft subtitling requires skilled staff who understand the subtleties of subtitle positioning, formatting and how to best précis the spoken words into the most elegant and informative of subtitles. The opposite end of the subtitle creation spectrum produces verbatim or as-spoken subtitles these are quicker to produce using fast typists or voice to text software to allow a re-speaker to listen to the program audio and re-speak the subtitles into a subtitle workstation application. If program scripts are available, and are accurate, these may be imported into the workstation to more quickly produce a verbatim subtitle file. The process of aligning or timecoding each subtitle to the program manually is both time-consuming and tedious and this is an area where technology is well placed to reduce time and effort. Speech analysis software provides the means to auto synchronise subtitles to the program audio either as each subtitle is created ( TiGo or time as you go ) or as a post-process once the program subtitle text has been prepared. Although new to the market place, and limited initially to a defined set of languages, such text to audio alignment technology is already proving to be very effective in reducing the time to produce audio aligned subtitle files. Traditional workflow Craft subtitling using time intensive dialogue précis and manual subtitle positioning. Manual subtitle alignment to video Visual QC checking of final subtitle file Optimised workflow Fast semi-skilled verbatim subtitle input using speedy typist, script import or voice to text re-speaking Solution: Miranda Softel Swift Create subtitle workstation with the respeaking option Deployment of speech analysis technology to auto-align subtitles to video as they are created (TiGo mode) or as a post process (auto sync mode) Solution: Miranda Softel Swift ReSync subtitle alignment tools Background subtitle integrity checking of subtitles during the creation process, and automated subtitle file QC checking at point of subtitle file deployment Solution: Miranda Softel Swift Create subtitle workstation (with integrity checking) and Miranda Softel Swift subtitle QC application
Efficiencies in ensuring the quality of program subtitle files come from the application of automated QC tests both during and after the preparation of each subtitle file. During the subtitle creation process, a background integrity checking application can provide prompt and effective feedback to the workstation user where quality indicators such as subtitle reading speed, presentation and inter-subtitle separation may be used to signal quality issues. A full workstation review mode is an essential feature of the subtitle workstation this allows the program and subtitles to be played together as they would be seen by the viewer. There is scope for time-saving in this review process (which otherwise would take at least the duration of the program) by allowing the user to skip quickly through the captioned dialogue thus reducing the time to complete the program review process. When subtitle files are transferred from the subtitling team to a broadcast operations facility there is potential to replace the traditional manual task of re-checking the subtitles against the program video with an automated subtitle QC tool. It is not uncommon to find a mis-match between subtitle files and program episodes or between program timecode and subtitle file timecode. This tool analyses indicators such as audio speech to subtitle correlation and alignment (do the subtitles align to the spoken dialogue?), subtitle to scene change correlation (subtitles typically come on at the start of a new scene, and rarely cross over scene changes), and subtitle display regularity (it is rare for subtitles to be absent for long periods of time). Visual subtitle checking is time-intensive and an automated QC tool can quickly pay its way by reliably identifying the subtitle files which actually require re-checking. 2 Converting broadcast subtitles for web and mobile streaming Captions and subtitles are as important to viewers watching programs on web and mobile devices as they are to TV viewers, and in some cases even more so (watching content in a noisy environment for example). The challenge comes in knowing how and what subtitle file format is needed for each new media platform. In all cases there is a shared requirement for some sort of timed text file but the format and file types vary widely. In the USA, there is a legal mandate for content owners and broadcasters to make available caption files in the new SMPTE 2052 IP caption file format. Many web and mobile streamer platforms require a caption file which is close in specification to a DFXP timed text file, and in Europe the EBU are promoting the EBU-TT subtitle file format. The challenge to media operators is to decide where and when in the subtitle production and management workflow to produce these web and mobile derivative subtitle files. Professional subtitle workstations (such as Miranda Softel Swift Create) can export subtitle files in many different formats, and this is the easiest way for subtitle service companies to deliver multi-format subtitle files to their media distribution customers.
Content distribution facilities have the challenge of transcoding the master video into the wide range of compressed video streaming file formats required for the various formats. In this case it makes sense to produce the IP subtitle files as part of the video transcoding process. Transcoder suppliers frequently license subtitle plug-ins from subtitle specialists (such as Miranda Softel Swift vtx file-based subtitle processor) which offer the means to extract subtitles from the source video and create IP format subtitle files, or they can offer transcoding from one subtitle file format to another. Broadcast facilities often use powerful digital media asset management (MAM) platforms to store and publish content. MAM suppliers also typically offer Miranda Softel Swift vtx subtitle plug-ins to facilitate subtitle browsing, QC and multi-format subtitle file export. As a final option, a stand-alone Swift vtx subtitle file format transcoder offers the ultimate flexibility to be deployed as an independent tool in the media publishing process.
3 Re-timing subtitle files for edited video content One of the biggest subtitling challenges is dealing with video and subtitle files which no longer match often video has scenes cut for online streaming, or the frame rate has changed, or content such as advertisements are added, or simply the video timecode no longer matches the subtitle file. No one wants to contemplate the costs of fully re-subtitling their video, and everyone hopes for a tool to fix their caption files. Looking at these scenarios in order of complexity, the first being re-alignment - the case where timecode on the video is not the same as when originally subtitled, or where the video frame rate has changed. Subtitle re-alignment can be achieved using advanced subtitle file processing, which deploys a variety of techniques to lock each subtitle back to the matching spoken dialogue. Typically, subtitle re-alignment is run as a standalone task, or integrated into a production workflow or MAM system. The second scenario is where the video has been edited since it was originally subtitled, but where an EDL file (edit decision list) is available ( EDL assisted re-alignment ). The valuable metadata in the EDL file can be used by a subtitle processor to re-align each subtitle, after the first edit point, to the matching dialogue in a video file. EDL assisted re-alignment is typically offered as part of a subtitle workstation (so that the user may verify the amended subtitle file manually), or as a standalone automated process integrated into production workflow. The most complex of scenarios ( fix-up realignment ) where there have been video edits, but no EDL file available, is naturally the most challenging to solve. Technology available today is able to get very close to working out which subtitles are no longer needed, and to re-aligning the remaining captions, but the resulting fix-up subtitle file still requires manual checking. The challenge Re-alignment the video no longer carries the original timecode, or the video frame rate has changed EDL-assisted re-alignment video has been edited, but an edit decision list file is available Fix-up re-alignment video has been edited and all subtitle alignment lost The solution Post-process the subtitle file to produce a re-timecoded, re-aligned subtitle file. Run the task manually or as part of a media workflow process Solution: Miranda Softel Swift ReSync Post-process the subtitle file using the EDL file to re-align existing subtitle timings after the first edit point in the video. Run the task in a subtitle workstation or as an independent process as part of the wider media production workflow Solution: Miranda Softel Swift Create with EDL processing, or Swift vtx EDL Post-process the subtitle file to produce >90% accurate re-aligned subtitle file ready for manual checking in subtitle workstation Solution: Miranda Softel Swift ReSync
4 Reducing the per-channel cost of dynamic time of air subtitle output Time of air broadcast of subtitle files, aligned to the channel s event playlist and to video timecode, is a popular solution for single or multi-language closed (VBI or VANC encoding) or open (burnt onto the video) broadcast subtitling. Teletext subtitle encoding into the ancillary space of SD or HD video serves two purposes; the transmission of WST Teletext subtitles as part of a digital TV service, or as a means of tunnelling subtitles in the video signal to a DVB bitmap digital subtitle renderer downstream at the video encoding facility. Cost savings can be achieved by choosing a subtitle play-out product, such as Miranda Softel Swift TX, which uses commodity IT server hardware and which scales from one to multi-channel in the same server chassis. Multi-channel open subtitle rendering is the most cost efficient when deployed, using high channel density SDI video keyer PC cards. In the case of Teletext and closed caption encoding into video, external single or multi-channel encoder hardware is generally the most cost effective choice of hardware. Operational costs for time-of-air subtitling may be minimized by ensuring tight integration between main and failover subtitle processors and with channel automation. This automation integration provides essential look-ahead schedule metadata to the subtitle processor so that subtitle files can be pre-loaded and so that missing subtitle files can be flagged at the earliest moment. Tight integration simplifies operator supervision and ensures that operational issues are diagnosed and solved using effective built-in monitoring and service logging tools. Capital costs are therefore most effectively reduced through the deployment of high-channel-density subtitle processors, and optimum operation cost reduction is achieved through the selection of a high-availability and tightly integrated subtitle play-out platform. Tight integration simplifies operator supervision and ensures that operational issues are diagnosed and solved using effective built-in monitoring and service logging tools. Automation Systems Media servers systems Digital asset management systems Harris (inc Louth) Grass Valley Dalet Miranda s Softel Subtitling and Captioning Solutions Partners Miranda Pebble Beach Snell (ProBell) Harris Harmonic (Omneon) Pinnacle (Avid) MassTech Evertz (Pharos) Arvato Systems XOR Media TMD
5 Encoding closed subtitles in HD video Whether encoding closed subtitles or captions into HD SDI video signals, or into HD video files, it is important to understand the difference between the various closed subtitle and caption standards. Closed subtitle and caption format CEA 708 Use case The North American standard for the carriage of closed captions in HD SDI video. Also used to carry closed captions in video files in MXF SMPTE436M track or in the user bit space of an MPEG program stream. CEA708 also supports the carriage of international Unicode character sets and is deployed in countries outside the US such as Korea. OP47 Teletext OP47 carries up to the equivalent of 4 SD VBI lines of Teletext subtitle data in the VANC ancillary space of an HD SDI video signal, or in an MXF SMPTE436m track within a video file. Solution: Miranda Softel vflex HD (encoding into HD SDI), and Swift vtx (encoding into video files) SMPTE 2031 2031 defines the way in which a full SD 16 VBI line Teletext service is carried in the VANC ancillary space of an HD SDI video signal, or in an MXF SMPTE436m track within a video file. Solution: Miranda Softel vflex HD (encoding into HD SDI), and Swift vtx (encoding into video files) Miranda Softel MultiText Teletext subtitles The Teletext character set is limited to European and Arabic languages. Miranda s Softel MultiText format allows any international subtitle stream to be formatted as Teletext data and carried as OP47 or SMPTE2031 data in HD SDI video signals or in video files. MultiText subtitles may then be transcoded downstream into DVB digital bitmap subtitle streams. Miranda s Softel ancillary data processor, vflex HD, enables the bridging, processing, encoding, and regenerating of Teletext (OP47) and other ancillary data in HD video signals.
6 Simplifying DVB subtitling across multiple channels In the case of DVB subtitle processing, less is more. There are significant power usage, rack space and operational advantages in the deployment of a single server, multi-channel DVB subtitle system. The Miranda Softel Swift TX DVB subtitle processor can generate over fifty individual language subtitle streams, split across any number of program channels, from one standard IT rack server. A multi-channel subtitle processor often has to support three very different modes of operation; playlist synchronized play-out of subtitle files, transcoding of input video-embedded subtitles, and locally input subtitle streams for live programs. Play-list synchronization is most effective where the third party channel play-out platform supports an industry standard subtitle control protocol such as Miranda Softel XAP. Tight integration between channel play-out and subtitle systems ensures effective pre-loading of subtitle files, missing file alerts, and real-time subtitle event play and stop messages with embedded start at timecode controls (this removing the need for complex routing of timecode to the subtitle system). High channel density DVB subtitle processors, such as Miranda s Softel Swift TX system, offer proven operational simplicity through proven performance, integration and easy to follow services system health monitoring tools. Miranda Softel Swift TX DVB subtitle processor can generate over fifty individual language subtitle streams from one standard IT rack server. HD/SD OPEN CG itx Playout Systems Automation Control + Timecode Video Single Channel Dual Channel Quad Channel DVB Compression System Video with open subtitles itx Content Store Subtitle Files Master Standby DVB Subtitles DVB Compression System Video with DVB subtitles SWIFT TX PLAYOUT
7 Treating subtitles as part of a file-based media workflow There are strong and positive arguments for binding subtitles to their matching video files at some point in the media publishing workflow in advance of transmission. However, given the variety of media and subtitle file formats in use, this is a real challenge. Early binding, as Miranda refers to this process, brings advantages to the workflow such as the ability to QC-check subtitled video files, and can also reduce the need for subtitle live binding transmission hardware and the operational resources required to oversee and manage time of air broadcast systems. Miranda s Softel Swift vtx file-based subtitle encoding application has been designed and developed to accommodate the almost exponential combination of subtitle file formats, video file formats and standards, video file wrappers, and subtitle encoding practices which are in daily use at broadcast and streaming facilities around the world. Swift vtx is designed to operate in a number of use cases including as a standalone subtitle processor, embedded within third party MAM and video INPUT OUTPUT Video Video transcoding products, or as Files Files a slave to a wider file based workflow, using Swift vtx s Subtitle Encoding Subtitle software API. Extraction INPUT OUTPUT Subtitle files Task Processes Subtitle files Core competencies required of a file-based subtitle encoding tool Subtitle file handling The ability to read all commonly used subtitle file formats The ability to transcode source to destination subtitle formats (e.g. closed caption to Teletext) The ability to re-time a subtitle file frame rate between source and destination The ability to create new subtitle files, in any format, from subtitles extracted from a source video file Subtitle extraction The ability to extract, frame by frame, subtitles or closed captions from a source video file The ability to pass extracted subtitles to either the subtitle encode processor or the file writing and export process Subtitle encoding The ability to generate standards-compliant subtitle video payloads for all common subtitle and closed caption formats The ability to open and encode subtitles into all common video and video wrapper file formats The ability to encode subtitles in MPEG program stream format for VOD applications
8 Subtitle creation for live and news programming Live programing creates one of the most significant and resource intensive subtitling challenges of all. By the very nature of live TV, most of the subtitle creation and output has to be handled dynamically during the original broadcast. There is no speech analysis technology available today which is able to automate the real-time subtitling process to the level of accuracy that is needed for broadcast TV. So, while we wait, the challenge is to find ways of generating live subtitles with the minimum of cost and effort. The two most common live subtitle input methods are stenography and re-speaking. Trained stenographers are common in the USA but less so elsewhere, so their use is limited. Re-speaking mixes the power of the latest voice to text applications with the industry-specific features of the subtitle workstation to offer a relatively simple way to produce live subtitles using only semi-skilled operators. The operator, who has previously trained the application to his or her voice, listens to program audio and either repeats or summarizes the dialogue or commentary through a microphone to the subtitle workstation. The Miranda Softel Swift Create Live subtitle workstation supports subtitle re-speaking, but also a set of advanced features designed to simplify the process of live subtitle creation as far as possible. These include profanity filtering (black book of words never to be broadcasted), work sharing (the ability for multiple subtitlers to collaborate on the same content at the same time), and the ability to pre-load scripted elements of a live broadcast from a third party newsroom system. Swift vtx (early binding) and Swift TX (live binding) solutions may also be deployed to subtitle any pre-prepared elements of a news program. If the news programs, or any stories within the news programs, are to be repeated, then Miranda s Softel Re-Sync re-alignment tools may be used to autocorrect the subtitle timings. Re-speaking mixes the power of the latest voice to text applications with the industry-specific features of the subtitle workstation to offer a relatively simple way to produce live subtitles using only semi-skilled operators.
9 Creating and managing Audio/Video Description content There is a slow but sure rise in the adoption of audio description (AD) services, which should not be a surprise given the potential number of visually impaired viewers. The challenges involved in creating AD content break down into three areas, scripting, recording and audio encoding. AD scripting is very similar to the process of subtitle creation and editing except that the text is aligned to the quiet rather than the conversational parts of each scene. The Miranda Softel Swift ADePT workstation software simplifies the process of creating timecoded passages of AD script for each scene in a program, along with audio pan and fade metadata in each case. A rehearsal mode allows the describer to practice the narration and to check the timings. When it comes to the AD script recordings, there are three ways to go; there are professional describers who are skilled at both script editing and narration, or facilities may have their own resident talent, or it is possible to use text to speech software to automate the process. Miranda s Softel Swift ADePT record workstation takes care of the recording process and offers features to aid the narrator such as visual count downs, before and during each passage, and the ability to choose the best take from multiple recordings. The AD audio is typically broadcast or streamed in one of two ways. Where bitrate is expensive, such as for digital terrestrial TV, the AD audio is often transmitted on its own and mixed into program audio by the consumer s receiver using a dedicated low bit rate control signal. Otherwise, an audio mix of AD and program audio is broadcast as an alternate stream. The Miranda Softel Swift ADePT Replay and Swift vtx offer tools to create or broadcast these audio streams. 00:00: 10:00 Audio Video DESCRIPTION SPEECH DESCRIPTION SPEECH AD Script Audio Description Audio ADePT Script ADePT Record & Replay Video The Miranda Softel Swift ADePT record workstation takes care of the recording process and offers features to aid the narrator such as visual count downs, before and during each passage.
10 Monitoring subtitle streams in a multi-channel broadcast facility Multiple channels and multiple language subtitle streams make it quite a challenge to visually monitor subtitles at a digital TV play-out or turnaround facility. Viewers are fast to complain when subtitles go missing or are poorly aligned to the video; thus, operators are wise to invest in a tool which offers a mixture of both automated and visual subtitle stream monitoring. For visual subtitle monitoring, the choices are to leverage existing multi-viewer displays, or to create a dedicated display to show one or more realtime video streams along with all subtitle language streams. The Miranda Softel Swift Subtitle Monitor is a DVB subtitle monitor that addresses both these modes and more. Using a choice of PC or HD SDI display the operator can configure a set of display profiles which display a grid of X by Y video streams, each showing all or a subset of the language subtitles overlaid onto the video. The Miranda monitor has enhanced options to create automated loss of subtitle alerts, and to output the DVB subtitles streams for display on third party video multi-viewer displays. Example of Swift Subtitle Monitor Visual Display
The Miranda Softel Swift family of subtitling and closed captioning solutions The Swift range of subtitling and closed captioning solutions enables the creation, repurposing, encoding, transcoding, insertion, monitoring and transmission of subtitles and closed captions in a wide array of file formats while increasing reliability and productivity. Optimized for next-generation file-based workflows, Swift products support subtitling and closed captioning for multi-platform delivery, across multi-language feeds and in a complete variety of file formats. MANAGE CREATE DESCRIBE INSERT TRANSMIT MONITOR Quality Control Task management Bind-point control Caption/Subtitle file Create Edit Import Export Repurpose Browse-clip digitizing for non-linear creation/ re-purposing Audio/Video description Script Record Track preparation Linearizer File based Encode Transcode Extract Repurpose Open subtitle generation VBI/ANC data encode & decode Multi-format time-of-air transcode and transmission Live subtitle input & capture At a glance multi-channel and multilanguage monitoring of subtitle match and alignment to video Over 25 video channels supported Workflow CG CG About Miranda Technologies For over 23 years, Miranda Technologies (www.miranda.com) has been a leading worldwide provider of hardware and software solutions for the television broadcast, cable, satellite and IPTV industry. Its solutions span the full breadth of television operations, including production, playout and delivery. With a wealth of experience in delivering IT-based and traditional television systems, Miranda leverages its expertise to create the highest quality solutions available and delivers, deploys and supports them with a companywide commitment to keeping the customer first. Miranda was acquired in July 2012 by St. Louis-based Belden Inc. and is the cornerstone of its broadcast division. Belden is a worldwide leader in the design, manufacture and marketing of cables, connectivity and networking products for many industries including broadcast, entertainment, consumer electronics, education, transportation and industrial automation. For more information, visit www.belden.com. 3499 Douglas-B.-Floreani, Montreal, Quebec, Canada H4S 2C6. Tel: +1 800 224 7882 Fax: +1 514 333 9828 Abbey Gate, 57-75 Kings Road, Reading, Berkshire, United Kingdom RG1 3AB. Tel: +44 118 952 3400 Fax: +44 118 952 3401 Belden is a trademark or registered trademark of Belden Inc. or its affiliated companies in the United States and other jurisdictions. Miranda, Softel and Swift are trademarks or registered trademarks of Miranda Technologies Partnership. Belden Inc., Miranda Technologies Partnership and other parties may also have trademark rights in other terms used herein. Copyright 2013 Miranda Technologies Partnership. All rights reserved.