Sample Cities for Multilingual Live Subtitling 2013

Transcription

1 Carlo Aliprandi, SAVAS Dissemination Manager Live Subtitling 2013 Barcelona,

2 SAVAS - Rationale SAVAS is a FP7 project co-funded by the EU 2 years project: R&D companies and 5 user companies are working at European level to develop a brand new generation of products and services for better production of Multilingual Subtitles Target: the Broadcasting and New Media Industry and its stakeholders

3 Objectives To exploit audiovisual resources produced by Broadcasters and Subtitling Companies, making it possible their acquisition and reuse To deploy a platform for audiovisual language resources sharing through the most suitable legal and business data trading approaches To develop a brand new ASR (Automatic Speech recognition) technology for Multilingual Live Subtitling 3 3

4 Core Objective To collect audiovisual resources to build a brand new ASR technology for Multilingual Live Subtitling Why? current ASR commercial systems have several limitations, because they have been designed for dictation applications, thus performing too poorly in specific tasks, like Live Subtitling language coverage may be limited, as commercial systems have been developed to target the main language markets (i.e. English, Spanish, French, German,..) and they are not available for many languages and dialects domain coverage may be limited, as commercial systems have been developed to target general and generic topics 4 4

5 New Languages An initial set of 6 languages is under developement: Basque, Spanish, Italian, French, German, Portuguese A set of 3 more languages will be developed: Swiss_Italian, Swiss_French, Swiss_German A wide set of new Dictionaries will be created for the following domains: Broadcast News Sports Interviews and Debates 5 5

6 Minor Languages Languages that are: traditionally used within a given territory of a State by nationals of that State who form a group numerically smaller than the rest of the State's population; different from the official language(s) of a State 6 6

7 Automatic Speech Recognition A LVCSR (Large Vocabulary Continuos Speech Recognition) system involves a comprehensive set of resources Acoustic Modeling Language Modeling Large Vocabulary Pronunciation Lexicon Huge amount of data are usually necessary Training is an iterative process: TRAIN TEST REFINE - RETRAIN 7 7

8 Minor Languages - challenges Spoken by few people Difficult to find language expertise Less language resources available Strong regional accents Cross-fertilisation between languages la meteo vs il meteo rolladen for roll-up shutters and chifer for croissant Higher number of foreign (OOV) words Native speakers often speak another language 8 8

9 Minor Languages - workaround More training for selected people Make (not acquire) LR Spoken by few people Difficult to find language expertise Less language resources available Build a baseline system (ex FR_FR), then exploit adaptation techniques: Adapt it to Swiss lexicon Adapt it to Swiss accent (FR_CH) Adapt it to regional accents Adapt it to non-native speakers Strong regional accents Cross-fertilisation btw languages la meteo vs il meteo rolladen for roll-up shutters and chifer for croissant Higher number of foreign (OOV) words Native speakers often speak another language 9 9

10 New Products and services Based on ASR Engines, now available new subtitling systems and services: 1. S.SCRIBE!, a Speaker Independent Transcription System for batch production of multilingual subtitles 2. S.RESPEAK!, a collaborative Respeaking System, for live and batch production of multilingual subtitles Forthcoming: S.LIVE!, a Speaker Independent Transcription System for live production of multilingual subtitles 10 10

11 Live Subtitling: more then good ASR Adapt ASR to spontaneous speech Speaker independent ASR Normalization rules, in terms of definition of dates, numbers, punctuation, capitalisation Formatting rules : Subtitles generation in several modes: word by word, line blocks, scrolling blocks 2 rows of text at maximum 37 characters per line at maximum Change of colors according to speaker gender or speaker clustering Present the name of the speaker if that information is available Change colors for words with confidence below a settable threshold The airing time of the subtitle should be the same as the input audio 11 11

12 Subtitling systems - architecture REC / SEMI-LIVE PROGRAM BATCH TRANSCRIPTION SUBTITLE EDITOR LIVE PROGRAM+ RESPEAK LIVE RESPEAKING TELETEXT WEBTV INTERNET LIVE PROGRAM LIVE TRANSCRIPTION TELETEXT INSERTER 12 12

13 Subtitling systems mix architecture SEMI-LIVE PROGRAM+ RESPEAK LIVE RESPEAKING SUBTITLE EDITOR LIVE PROGRAM LIVE TRANSCRIPTION LIVE RESPEAKING TELETEXT WEBTV INTERNET 13 13

14 S.SCRIBE! Batch Transcription XML file SRT/STL file 14 14

15 S.SCRIBE! Transcription Service 15 15

16 S.SCRIBE! Transcription Service DEMO 16 16

17 S.RESPEAK!: Live Respeaking Push forward live-subtitling as a more collaborative process, Split the cognitive load among respeaker and corrector The respeaker can check the respeaking status The editor can dinamically create new words (OVV) The respeaker can blend into speech OVV and prefixed scripts The editor can correct errors Status=onAir Speech Recognition Input Keyborad/stenotype input Buffered subtitle On-Air subtitle Selected (onair) speaker Assigned speakers/colors Dinamic OWV list 17 17

18 Opportunities for stakeholders Share your experience with us Share your resources (audio, video, subtitles, scripts ): give value to your assets and exploit them at European level take the opportunity to fit the technology to you Collaborate with the consortium, in the End-User Panel 18 18

19 End-User panel The SAVAS End-User panel brings together Broadcasters, Subtitling Companies, Universities and companies involved in the industries of Media, Accessibility and LVCSR The role of participants is to provide recommendations and feedback, to give advice to development, to take part in testing and evaluation Are you interested in participating? Contact us for any detail

20 Thanks! Q&A