Media Fingerprinting. Applications in Broadcast Television



Similar documents
Datasheet EdgeVision

How To Test Video Quality With Real Time Monitor

Content Management Playout Encryption Broadcast Internet. SmartPlayout

Audio/Video Synchronization Standards and Solutions A Status Report. Patrick Waddell/Graham Jones/Adam Goldberg

Case Study: Real-Time Video Quality Monitoring Explored

WHITE PAPER. Ad Insertion within a statistical multiplexing pool: Monetizing your content with no compromise on picture quality

COPYRIGHT 2011 COPYRIGHT 2012 AXON DIGITAL DESIGN B.V. ALL RIGHTS RESERVED

DENSITÉ SERIES XVP-3901

Proactive Video Assurance through QoE and QoS Correlation

TECHNICAL OPERATING SPECIFICATIONS

Video Signal Path Management and Monitoring For Multiservice Operators

Tresent Technologies IPQ User report

AVP SW v2

Selenio Media Convergence Platform

Cable TV Headend Solutions

Clearing the Way for VoIP

Implementing Closed Captioning for DTV

Applications that Benefit from IPv6

Observer RPM. Remote Program Monitor. Post STB-Monitoring, Logging and Troubleshooting

Case Study Monitoring for Reliability

Implementing Closed Captioning for DTV

Video compression: Performance of available codec software

Teleport Services: NOC the State of the Art

An architecture for the delivery. of DVB services over IP networks Rennes, January 2007 INTRODUCTION DIGITAL VIDEO TRANSPORT

reach a younger audience and to attract the next-generation PEG broadcasters.

The Life of a Commercial

New Features for Remote Monitoring & Analysis using the StreamScope (RM-40)

Dual Channel Universal HD/SD Audio/Video Frame Synchronizer and Format Converter

Fundamentals of MPLS for Broadcast Applications

Images move the world We move images

Dolby DP600 and DP600-C Program Optimizer Overview for Cable and IPTV Operators

HMP-1801 HD/SD Solid-State Media Server

Smart LNB. White Paper. May 2014

IP Video Surveillance

1. Central Monitoring System Software

Video Conferencing Glossary of Terms

1. Central Monitoring System Software

HIGH-DEFINITION: THE EVOLUTION OF VIDEO CONFERENCING

How To Set Up & Manage an IPTV System WHITE PAPER

Market Drivers Influencing Broadcast

9! Multimedia Content! Production and Management

Terminal, Software Technologies

Audio and Video Synchronization:

Classes of multimedia Applications

ATM Video Surveillance

Application Note How To Determine Bandwidth Requirements

Understanding Video Latency What is video latency and why do we care about it?

IPC series IP camera user manual

MPEG-H Audio System for Broadcasting

Monitoring Conditional Access Systems

Live Webcasting & Video Streaming Made Easy with VidOstreamTM. Family

Digital Audio and Video Data

Broadcast Automation Without Boundaries

Cisco ROSA Video Service Manager (VSM) Version 05.03

Video Compression Trends & OUTLOOK. Andreas Pentrelli Solution Consultant

Course 12 Synchronous transmission multiplexing systems used in digital telephone networks

PRELIMINARY USER MANUAL

INTRODUCTION. The Challenges

DIGIMARC CORPORATION 9405 SW Gemini Drive Beaverton, Oregon

Combining Voice over IP with Policy-Based Quality of Service

Technical specifications

Voice over IP. Demonstration 1: VoIP Protocols. Network Environment

Live and VOD OTT Streaming Practical South African Technology Considerations

Central Management System (CMS) USER MANUAL

Completely Integrated and Customizable Media Services

LINE IN, LINE OUT AUDIO IN, AUDIO OUT FIXED, VARIABLE TO TV, VIDEO IN, VIDEO OUT Sony Electronics Inc. All rights reserved.

AT&T Connect Video conferencing functional and architectural overview

LINE IN, LINE OUT TO TV, VIDEO IN, VIDEO OUT

For Articulation Purpose Only

Radio TV Forum 2014 Rohde & Schwarz Solutions

Physical Layer, Part 2 Digital Transmissions and Multiplexing

DELIVERING CAPTIONS IN DTV An NCAM DTV Access Brief

(Refer Slide Time: 4:45)

DTS-HD Audio. Consumer White Paper for Blu-ray Disc and HD DVD Applications

SUBTITLING AND CLOSED CAPTION MANAGEMENT

Acterna DSL Services Tester TPI 350+ Application Highlights

Chapter 3 ATM and Multimedia Traffic

Workflow Comparison: Time Tailor versus Non-Linear Editing Systems WHITE PAPER

Voice over IP (VoIP) and QoS/QoE

1. Central Monitoring System Software

Conference interpreting with information and communication technologies experiences from the European Commission DG Interpretation

Next Generation DTV: ATSC 3.0

Bosch IP An introduction to IP technology and the future of CCTV. Bosch IP Network Video Product Guide

Voice---is analog in character and moves in the form of waves. 3-important wave-characteristics:

Comux UK Ltd. Fixed Services Offer. 12 September Birmingham Science Park Aston Faraday Wharf Holt Street Birmingham B7 4BB UK

K2 System Ready for Nonlinear Media Production: QOS, Bandwidth and File Compatibility in File-based Workflows

ADVANTAGES OF AV OVER IP. EMCORE Corporation

Contents. Connection Guide. What is Dante? Connections Network Set Up System Examples Copyright 2015 ROLAND CORPORATION

2K Processor AJ-HDP2000

Network administrators must be aware that delay exists, and then design their network to bring end-to-end delay within acceptable limits.

How To Understand The Technical Specifications Of Videoconferencing

DVB Subtitling. The purpose of this document is to explain the principles behind DVB subtitling and to introduce Softel digital subtitling products.

TV 2 AS HD DELIVERY FOR TV 2 AS

VoIP Bandwidth Considerations - design decisions

Dolby DP600 and DP600-C Program Optimizer Overview for Postproduction Facilities

Integrated Playout and Media Asset Management

Video Conferencing. Femi Alabi UNC-CH - Comp 523 November 22, 2010

Streamit s audio streaming solutions for dummies

ivms-4200 Client Software Technical Specification v1.02

Transcription:

Media Fingerprinting Applications in Broadcast Television

Points of Discussion Next What is a media fingerprint? Fingerprint generation schemes How do fingerprints compare to Watermarks? Applications in Broadcast TV The case for a Fingerprint Standard

What is a Media Fingerprint? A form of identification for a piece of audio and video media Can be used to identify or recognize a specific piece media later in time or downstream in a system

Key Attributes of a Media Fingerprint Does not alter the media itself Compact, ideally much, much smaller than the media itself Robust and Resistant: Able to survive normal processing of the media Efficient: Economical to generate as well as compare or search in a database

Finger Prints Also Known As Digital Signatures Digital Reference Files Feature Vectors A/V Signatures Data Correlations Robust Hash

Points of Discussion What is a media fingerprint? Next Fingerprint generation schemes How do fingerprints compare to Watermarks? Applications in Broadcast? The case for a Fingerprint Standard?

Basic Fingerprinting Concept V a1 a2 an Change Based Video Fingerprint Algorithm Audio FP Algo Baseband Audio and Video Program Payload: Signal 156 Mbytes /Sec 1080i60 signal w 6 PCM audio ch Optional Timestamp multiplex Audio and Video Fingerprint Stream ~700 Bytes / sec 13 bytes / fld-frm for Video w 6 Ch Audio

Basic Fingerprinting Concept Next Change Based Video Fingerprint Algorithm Audio FP Algo V a1 a2 an Baseband Audio and Video Program Signal 156 Mbytes /Sec 1080i60 signal w 6 PCM audio ch Optional Timestamp multiplex Audio and Video Fingerprint Stream ~700 Bytes / sec 13 bytes / fld-frm for Video w 6 Ch Audio

A Sample Video Fingerprint Generation Algorithm Consider Sample N two points consecutive in each of fields the consecutive or frames in field a video or program frames A(1) B(1) A(N) Field or Frame X Field or Frame X+1 B(N)

A Sample Video Fingerprint Generation Algorithm Sample Sample N points Points each are not of necessarily the consecutive uniformly field distributed or as long as frames they are distributed the same way in both images A(1) B(1) A(N) Field or Frame X Field or Frame X+1 B(N)

A Sample Video Fingerprint Generation Algorithm Samples Very Different? no = 0 Yes = 1 If samples Compare are the very same different, samples indicate in a the 1 in two that image sample position, if not indicate a 0 A(1) B(1) 1 0 0 0 1 1 0 0 1 1 Sample 1 N A(N) Field or Frame X Field or Frame X+1 B(N) Repeat for every pair of samples in the images

A Sample Video Fingerprint Generation Algorithm Samples Very Different? no = 0 Yes = 1 1 0 0 0 1 1 0 0 1 1 Sample 1 N Count of all the 1 s (very different samples) Change Index: (0 -N) Field or Frame X Field or Frame X+1 Normalize to 1 Byte Normalized Change Index: 0-255

Video Program A Matching Two Fingerprints Video System Video Program A Delta Based Video Fingerprint Algorithm Delta Based Video Fingerprint Algorithm Streams of Change Index 1 Byte / Field or Frame Fingerprint Comparison Video Matching Factor (%) Program to Program Delay (16.6 msec resolution)

Change Index Streams 1 Byte / Field or Frame Simple Matching Process 255 Simple Convolution engine used to look for matching patterns in the two fingerprint streams Fingerprint Comparison Match Factor (%) Program to Program Delay 0 255 <-DLY-> Time (t) Matching Patterns are identified Patterns don t need to match perfectly. The more they match the higher the Match factor 0 Time (t) If Match Factor exceeds threshold, Delay between the two patterns is established and reported

Characteristics Of This Fingerprint Scheme Delta Based Video Fingerprint Algorithm Based on changes in image (levels and motion) Very simple to generate fingerprints requires very little h/w and s/w resources Implementable on monitoring DA grade card with negligible impact on cost Very simple to compare two fingerprints Can be done in real time using simple h/w or a few microprocessor instructions (no floating point) Hundreds of signals and multiple points per signal can be handled by one average PC Fingerprint stream is very compact 1 Byte per image or 60 Bytes per second for 60Hz video 0.0004% of baseband HD signal data rate 0.02 % of HD signal compressed to 20 Mbps

Characteristics Of This Fingerprint Scheme (2) Change Based Video Fingerprint Algorithm Insensitive to: Video level / color changes Video scaling (up / down conversion) and resolution Video Compression (DCT or Wavelet-J2K) Weakened but not disabled by: Aspect ratio change including adding bars on side or top/bottom Adding of graphics occupying less than 40% of image Disabled by: Frame rate conversion (e.g. 50 to 60 Hz) Prolonged periods of freeze

Basic Fingerprinting Generation Concept Change Based Video Fingerprint Algorithm Next Audio FP Algo V a1 a2 an Baseband Audio and Video Program Signal 156 Mbytes /Sec 1080i60 signal w 6 PCM audio ch Optional Timestamp multiplex Audio and Video Fingerprint Stream ~700 Bytes / sec 13 bytes / fld-frm for Video w 6 Ch Audio

One Channel of Digital Audio Program (48 KHz Sampling) Audio Fingerprint Algorithm Absolute Value Extract Envelope Extract Mean

Audio Fingerprint Algorithm (2) Envelope For every sample point, compare envelope value to mean value Sample Envelope Greater than Mean No = 0 Yes = 1 Result is 48 Kbps If Envelope > Mean Signature based on indicate a 1 Envelope to Mean Variations Mean 1 0 0 0 1 1 0 0 1 1 48K b/s Sub Sample to Reduce Bit Rate Audio Fingerprint 1 0 0 1 1 0 1 923 b/s 115 bytes/sec

Audio Program Matching Two Audio Fingerprints Audio System Audio Program Envelope to Mean Audio Fingerprint Algorithm Envelope to Mean Audio Fingerprint Algorithm 115 Bytes/s audio fingerprint streams Fingerprint Comparison Content Match Factor (%) Program to Program Delay (1 ms resolution)

1 Kbps audio fingerprint streams Program A Program B Audio Fingerprint Comparison Simple Matching Process Match Factor (%) Program to Program Delay Program A Program B 1 0 1 0 <-DLY-> Simple Convolution engine used to look for matching patterns in the two fingerprint streams Time (t) Matching Patterns are identified Fingerprint Streams don t need to match perfectly. The more they match the Time (t) higher the Match factor Delay between the two patterns is established and reported Delay resolution is 1 ms

Characteristics Of Audio Fingerprint Scheme Envelope to Mean Audio Fingerprint Algorithm Based on Variations in Audio Envelope Very simple to generate fingerprints requires very little h/w and s/w resources Implementable on monitoring DA grade card with negligible impact on cost Very simple to compare two fingerprints Can be done in real time using simple h/w or a few microprocessor instructions (no floating point) Hundreds of signals and multiple points per signal can be handled by one average PC Fingerprint stream is very compact (115 bytes/s) and High resolution (1 msec) 0.08 % of baseband audio payload

Characteristics Of Audio Fingerprint Scheme (2) Envelope to Mean Audio Fingerprint Algorithm Insensitive to common processing Bit rate reduction (aka compression) Level shifts including Dynamic Compression Sample rate conversion Disabled by Mixing new content Significantly changing frequency content UpMix and DownMix cases are handled by the use of additional signatures

Points of Discussion What is a media fingerprint? Fingerprint generation schemes Next How do fingerprints compare to Watermarks? Applications in Broadcast? The case for a Fingerprint Standard?

Fundamental Difference Between Finger Prints and Watermarks Fingerprints extract properties from a source then store or transport it separately Watermarks ALTER sources by putting some form of mark or identifier in the media in a hard to remove way Actually the watermark is usually not visible/audible to the human eye or ear.

Fingerprints vs. Watermarks Quality or Property Fingerprints Watermarks Content Remains Unchanged Yes No Survive Process Manipulation Yes Depends on Algo Less Depends on Algo Unique Yes Not Necessarily Can Be Removed No Yes Used Retrospectively Yes No How Transported Typically Stored / Transported Separately Proprietary Yes*** Yes Transported in the Signal

What Can (Must) a Media Fingerprint Survive? Up/Down conversion Encoding/Decoding for video and audio compression Audio Level Conversion, Up and Down Mixing Insertion of logos and other graphics Resizing and spatial conversion Frame rate/temporal conversion Cropping Signal distortions filtering

Fingerprints vs. Watermarks Quality or Property Fingerprints Watermarks Content Remains Unchanged Yes No Survive Process Manipulation Yes Depends on Algo Less Depends on Algo Unique Can differentiate content and versions No Yes Can Be Removed No Yes Used Retrospectively Yes No How Transported Typically Stored / Transported Separately Proprietary Yes*** Yes Transported in the Signal

Can Fingerprints and Watermarks Differentiate? Bambi Different? Godzilla Fingerprints will Be able to identify that they are different Watermark will be able to identify that they are different Joe s Bambi Different? Dave s Bambi Fingerprints will NOT be able to identify that they are different Watermark will be able to identify that they are different

Fingerprints vs. Watermarks Quality or Property Fingerprints Watermarks Content Remains Unchanged Yes No Survive Process Manipulation Yes Depends on Algo Less Depends on Algo Unique Yes Not Necessarily Can Be Removed (hacked) No, not embedded In some cases yes Used Retrospectively (after the fact) Yes No How Transported Typically Stored / Transported Separately Proprietary Yes*** Yes Transported in the Signal

Fingerprints vs. Watermarks Quality or Property Fingerprints Watermarks Content Remains Unchanged Yes No Survive Process Manipulation Yes Depends on Algo Less Depends on Algo Unique Yes Not Necessarily Can Be Removed (hacked) No In some cases yes Used Retrospectively (after the fact) Yes No How Transported Typically Stored / Transported Separately Proprietary Yes*** Yes Transported in the Signal

Retrospective Use - Watermarks I Love Lucy Episode 39 First Copy Ingested 10 Years Ago I Love Lucy Episode 39 Copy 1 Not Watermarked Looking For Copies of I Love Lucy Episode 39 I Love Lucy Episode 39 2nd Copy Ingested 1 yr ago with Watermark I Love Lucy Episode 39 Using Watermark Copy 2 Watermarked Server or Archive

Retrospective Use - Fingerprints I Love Lucy Episode 39 Generate Looking For Copies of I Love Lucy Episode 39 Copy 1 10 Years Ago Copy 2 1 Year Ago I Love Lucy Episode 39 Generate Compare Generate Server or Archive

Fingerprints vs. Watermarks Quality or Property Fingerprints Watermarks Content Remains Unchanged Yes No Survive Process Manipulation Yes Depends on Algo Less Depends on Algo Unique Yes Not Necessarily Can Be Removed No Yes Used Retrospectively (after the fact) Yes No How Transported Typically Stored / Transported Separately Proprietary Yes*** Yes Transported in the Signal

Points of Discussion What is a media fingerprint? Fingerprint generation schemes How do fingerprints compare to Watermarks? Next Applications in Broadcast? The case for a Fingerprint Standard?

Media Fingerprinting High Level Concept Audio and Video Programs Audio+Video Fingerprint Algorithm Audio and Non intrusive TV The Video two signatures are Downstream, device analyses a Production compared Programs to establish if second video+audio and and device Delivery content is the same and generates generates a a low bit finger System what the relative delay is print of rate the audio signature and video Signature is Audio+Video either stored or Fingerprint time-stamped and Algorithm streamed into a standard IT network Fingerprint Database Standard Network compare Content Same? Audio to Video Delay Same?

Media Fingerprinting High Level Concept Audio and Video Programs TV Production and Delivery System Audio and Video Programs Content In Server Audio+Video Off Line Fingerprint Algorithm Fingerprint algorithm Fingerprints can also are be applied stored to on file a server based and anc content can later be sitting on searched a server for content match Fingerprint Database LAN / WAN Audio+Video Fingerprint Algorithm Search What Content is This

Media Fingerprint Applications for Broadcast Television Next 1. Content Verification for Playout 2. Ad Insertion Verification 3. Distributed Lip Sync Monitoring 4. Content Security/Piracy protection in online video sites

Traditional Playout Monitoring Multi Viewer Server Playback Primary and Backup Branded Outputs Primary, Backup & Final Returns Primary Path Primary Server Backup Server R o u t e r Branding Main Branding Backup CC/VBI Inserter Backup Path CC/VBI Inserter Neilsen Encoder Neilsen Encoder Audio Process Audio Process 2x1 Change -over Distribution Encoder Return IRD DTH return Up Link Cable return

Operators typically monitor multiple points in playout chain Content is Valid and Same Across Entire Distribution Path imary erver ackup erver R o u t e r Primary Path Branding CC/ VBI Secondary Path CC/ Branding VBI Neilsen Neilsen Audio Audio 2x1 Change -over Distribution Encoder Return IRD Up Link DTH return Cable return

Visual Content Comparison is Key (case 1) Mismatch Between 2 inputs of 2x1 changeover CC/ VBI CC/ VBI Primary Path Neilsen Audio Secondary Path Neilsen Audio 2x1 Change -over Distribution Encoder Return IRD TX Indicative of a problem with Primary or Secondary Path DTH return Cable return

Visual Content Comparison is Key (case 2) CC/ VBI CC/ VBI Primary Path Neilsen Audio Secondary Path Neilsen Audio 2x1 Change -over Distribution Encoder Return IRD DTH return Cable return TX Mismatch Between Mismatch Playout Between and Satellite Playout and Return Satellite Return Indicative of an Indicative uplink problem of an uplink problem

Visual Content Comparison is Key (case 3) CC/ VBI CC/ VBI Primary Path Neilsen Audio Distribution 2x1 Encoder Change Secondary Path -over Neilsen Audio Return IRD DTH return Cable return TX Mismatch Between Playout and Cable DTH return Indicative of a problem Indicative of DTH a problem Operator Cable Operator

Visual Content Comparison is Key (case 4) Even Subtle Differences in Content Need to Be Checked, Like Missing Graphics and Logos Primary Server Backup Server R o u t e r Branding (primary) Branding (backup) CC/ VBI CC/ VBI Primary Path Neilsen Audio Distribution 2x1 Encoder Change Secondary Path -over Neilsen Audio Return IRD TX

Traditional Playout Monitoring Scenario Channels are monitored by an operator looking at the monitor wall Alarming helps the operator detect problems like freeze, black, silence Operator watches & listens for operational errors such as wrong content, missing graphics, lip sync As channel count grows or channel complexity increases, so does the operator count

Fingerprint Based Monitoring Scenario Compare All Points Finger Print Multi Comparison Viewer Engine Exception Notification Server Playback Primary and Backup Branded Outputs Primary, Backup & Final Returns Primary Path Primary Server Backup Server R o u t e r Branding Main Branding Backup CC/VBI Inserter Backup Path CC/VBI Inserter Neilsen Encoder Neilsen Encoder Audio Process Audio Process 2x1 Change -over Distribution Encoder Return IRD DTH return Up Link Cable return

Fewer Operators Yet Potentially Deeper Monitoring Exception Notification Finger Print Comparison Engine

Fingerprint Based Monitoring Scenario Multi-Point Finger Print Correlation Engine Standard Ethernet Network Primary Server Backup Server Generate FP R o u Generate t e FP r Primary Path Branding Main Branding Backup CC/VBI Inserter Backup Path CC/VBI Inserter Neilsen Encoder Neilsen Encoder Audio Process Audio Process Generate FP 2x1 GenerateChange FP -over Generate FP Generate FP Generate FP Generate FP Distribution Encoder Return IRD DTH return Cable return Up Link

Ideally Fingerprint Generation built into devices Multi-Point Finger Print Correlation Engine Standard Ethernet Network Primary Path Primary Server Backup Server RFP o u t e FP r BrandingFP Main BrandingFP Backup CC/VBI Inserter Backup Path CC/VBI Inserter Neilsen Encoder Neilsen Encoder Audio Process Audio Process FP FP 2x1 Change -over FP Distribution Encoder FP FP Return IRD DTH return Up Link FP Cable return

Key Requirements for Playout Monitorng Fingerprints must be robust to survive multiple conversion, encoding, decoding Fingerprints must be focusable to detect specific problems such as missing graphics Fingerprint generation must be simple so that it can be incorporated inside simple devices (e.g. routers, DA s, IRDs) Multi-Vendor Support so not all the devices have to be from the same vendor

Media Fingerprint Applications for Broadcast Television 1. Content Verification for Playout Next 2. Ad Insertion Verification 3. Distributed Lip Sync Monitoring 4. Content Security/Piracy protection in online video sites

Ad Insertion Verification Cable Head End Ad Scheduling System Ad Files Satellite Receiver Program Advert Ingest, Trans-code, QC SCTE Trigger Ad Files Play: Cue: Ad ID Ad Server Ad Insertion Splicer Fingerprint + Ad ID Splice Cmd Program with Ad Ad Verification System Fingerprint db Finger Print Generator Finger Print Match Search Ad Playout Report To Cable Subs

Ad Insertion Verification Ad Files Cable Head End Satellite Receiver Program Advert Ingest, Trans-code, QC SCTE Trigger Ad Files Ad Scheduling System Play: Ad ID Ad Server Ad Insertion Splicer Fingerprint + Ad ID Splice Cmd Will detect playback faults such as missing audio, or truncated play Program with Ad Ad Verification System Fingerprint db Will independently report which ads played at what time without any operator intervention Finger Print Generator Finger Print Match Search Ad Playout Report To Cable Subs

Key Requirements for Ad Insertion Verification Fingerprints must be compact so that they can be stored in a database Fingerprint comparison must be simple to allow rapid search within a database Multi vendor support so fingerprints generated at ingest can be compared against fingerprint at playout

Media Fingerprint Applications for Broadcast Television 1. Content Verification for Playout 2. Ad Insertion Verification Next 3. Distributed Lip Sync Monitoring 4. Content Security/Piracy protection in online video sites

Lip Sync Can Happen In Many Places in Broadcast Chain Special event venue Network Facility Local Station Mobile studio Encode + Tx IRD Server Facility Infrastructure Branding + Proc Encode + Tx Return IRD Rx + Decode Local Process Encode + Tx Off Air IRD Service Provider (Cable, DTH, IPTV) Rx+ Decode Encode + Tx

Fingerprint Comparison Provides Delay Measurement Program Video + N Ch Audio System Video Audio Fingerprint Ch Generation 1 Audio Ch N Video Audio Fingerprint Ch Generation 1 Audio Ch N Fingerprint Audio Comparison Audio Video Ch Audio to Video Delay Ch Calculation 1 N Overall Program Delay Video to Audio Lip Sync Error (per channel)

Fingerprint Comparison Provides Delay Measurement If Video Delay Audio Delay we have lip sync errors Video Fingerprint Comparison Ch N Audio Fingerprint Comparison Ch 2 Audio Fingerprint Comparison Ch 1 Audio Fingerprint Comparison Can establish Lip Sync error as small as 1 ms Can establish Lip Sync error across any number of audio channels 15 PGM A 0 15 PGM B 0 <- Vid -> DLY PGM A Compare delays PGM B 1 0 1 0 Ch1 <-Aud-> DLY

2 Point Monitoring: Output + Return With only two points to compare a simple h/w module based solution is well suited Lip Sync Probe Primary Path CC/VBI Branding encoder Secondary Path A/V Proc 2x1 Distrib Enc Up Link FP Gen FP Gen FP Comp Branding CC/VBI encoder A/V Proc Return IRD Overall Pgm Delay Video to Audio Delay for Each Audio Ch. Alarm for Excess

2 Point Lip Sync Monitoring: Out vs Cable Return What if the problem is at the Cable or Satellite Distributor Lip Sync Probe FP Gen FP Gen FP Comp ary Path A/V Proc ary Path A/V Proc 2x1 Distrib Enc Up Link Return IRD Service Provider (Cable, DTH) Rx+ Decode Encode + Tx DTH return Cable return Overall Pgm Delay Video to Audio Delay for Each Audio Ch. Alarm for Excess

What if I want to monitor more than 2 points in my system and establish where Lip Sync error is introduced

End to End Monitoring Including Multiple Points Multi-Point Fingerprint Correlation and Lip-sync Detection Standard Ethernet Network IRD Generate FP Server Facility Infrastructure Broadcast Facility Generate FP Branding + Proc Generate FP Encode + Tx Generate FP Return IRD Service Provider Decode Re-encode DTH return Cable return Generate FP Generate FP

Ideal Scenario Is to Have Fingerprint Generation built into key devices in the chain Multi-Point Fingerprint Correlation and Lip-sync Detection Standard Ethernet Network IRD FP FP Server Facility Infrastructure Branding + Proc FP Encode + Tx Service Provider Decode Re-encode DTH return Cable return FP FP Broadcast Facility FP Return IRD

What about cases where signals travel through are distributed across multiple sites?

Remote Fingerprints Streamed via Standard IP Network Central NOC or Monitoring Point Multi-Point Fingerprint Correlation and Lip-sync Detection Low Bit-rate WAN Link Low Bit-rate WAN Link Low Bit-rate WAN Link Special event venue FP Mobile studio Encode + Tx Network Facility IRD FP FP Server Facility Infrastructure Branding + Proc FP Encode + Tx FP Return IRD Local Station Rx + Decode Local Process Encode + Tx Off Air IRD

Miranda End-to-End Lip Sync Monitoring icontrol FP Fingerprint Generated and Transmitted by various Interface Module Frame Sync Collect & Compare Fingerprints + FP LAN/WAN FP Up Converters Detect and Measure Lip Sync Errors FP FP 2X1 IRD

icontrol End-to-End Lip Sync Monitoring Server A Out MC Out TX Return

Key Requirements for Lip Sync Application Fingerprints must be robust to survive multiple conversion, encoding, decoding Fingerprint must be compact to be streamed over network Fingerprint must include timestamp to allow transport over non deterministic networks Fingerprint generation must be simple so that it can be incorporated on simple devices (e.g. routers, DA s, IRDs) Multi-Vendor Support

Media Fingerprint Applications for Broadcast Television 1. Content Verification for Playout 2. Ad Insertion Verification 3. Distributed Lip Sync Monitoring Next 4. Content Security/Piracy protection in online video sites

Content Piracy Protection Server Encode + Tx Trans Code Mod Internet A broadcaster airs a certain show A Friendly Pirate hacks his PVR and extracts a copy of that show and posts it on YouTube People at home can watch it or download it to a portable device

Before airing show, broadcasters prepares fingerprints for all their shows and sends them to YouTube Content Piracy Protection Server Encode + Tx Trans Code Mod Internet Generate Fingerprint A Friendly Pirate hacks his PVR and extracts a copy of that show and posts it on YouTube Search Based on Fingerprint

Classic Fingerprint Example Block Track Monetize Index all content for search?

Points of Discussion What is a media fingerprint? Fingerprint generation schemes How do fingerprints compare to Watermarks? Applications in Broadcast? Next The case for a Fingerprint Standard?

In order for media fingerprinting to reach its full potential And provide end users with a practical way of leveraging the applications described earlier Needs to be interoperable among multiple vendors

The Need for an Interoperable Fingerprint Standard is similar to the need for an interoperable compression standard

Examples In Use Already Rights Management Lip Sync, Automated Playout Monitoring Lip Sync Content Verification, Ad Insertion Verification Content Verification, Rights Management Enable Content Verification, Rights Management Content Verification, Rights Management

What would need to be standardized? Audio Fingerprint Algorithm Video Fingerprint Algorithm Audio/Video Combination Files and Streams

Audio and Video Programs Standardize What? Audio and Video Programs Content In Server Audio+Video Fingerprint Algorithm Audio+Video Off Line Fingerprint Algorithm Fingerprint Database

Why Standardize? Fingerprints taken at multiple independent locations Locations vary widely: Production Truck, Broadcast Facility, Cable Head End, Sample Home Because of this variety, fingerprints need to be inter-operable Vendor A device generates the initial finger print Vendor B device generates fingerprint at a downstream location and is able to establish audio to video content match

Fingerprinting Nirvana Vendor A Lip Sync Application Vendor B Content Compare Application Special event venue Network Facility Local Station Mobile studio Vendor A Equipment Encode + Tx IRD Vendor B Equipment Server Facility structure Infra- Branding + Proc Vendor A Equipment Encode + Tx Return IRD Rx + Vendor Local C Equipment Process Rx + Decode Encode + Tx Off Air IRD Service Provider (Cable, DTH, IPTV) Vendor D Equipment Rx+ Decode Encode + Tx

Fingerprinting Nirvana Vendor A Lip Sync Application Vendor B Content Compare Application Special event venue Network Facility Local Station Mobile studio Vendor A Equipment Encode + Tx IRD Vendor B Equipment Server Facility Infrastructure Branding + Proc Vendor A Equipment Encode + Tx Return IRD Rx + Vendor Local C Equipment Process Rx + Decode Encode + Tx Off Air IRD Service Provider (Cable, DTH, IPTV) Vendor D Equipment Rx+ Decode Encode + Tx

What is being done to promote a standard? SMPTE S22-Lip Sync Evaluation Committee ATSC Specialist Group on Video and Audio Coding (TSG/S6) Miranda has offered its fingerprint algorithm license free, for use in the standard.

Conclusion Fingerprinting differs from watermarking. Fingerprinting has many applications. Standardization is essential for fingerprinting to reach its full potential

Media Fingerprinting: Applications in Broadcast Television Questions, Comments Observations?

Who Wants to Help? Contact Us: Michel Proulx Sara Kudrle Miranda Technologies Miranda Technologies mproulx@miranda.com skudrle@miranda.com +1.514.241.4293 +1.530.265.1000 Would like a copy of the paper? Download it From www.miranda.com/pdf/white-papers/media_fingerprinting.pdf