Ensuring quality experience for consumers Testing & Compliance Mike Tometzki VP Global Sales, Digital TV Labs
Introduction to Digital TV Labs Media & Device Testing Specialists Certification Interoperability Conformance - Quality UKAS accredited Testing Laboratory No.8159 to ISO/IEC 17025:2005 HE-AAC Testing and Certification Services for over 40 Platforms Globally 2
Services & Products Consultancy Operators & Broadcasters Specification Analysis Device Specification Authoring Test Regime Design Deployment Implementation Knowledge Transfer and Training Device Manufacturers Digital TV Market Requirements Receiver Technical Reference Requirements Services Test Regime Operation Test Suite Design Test Regime Auditing Device Test Services Play-out Validation Application Test RF Front End Testing HbbTV Testing CI+ Testing Conformance and Pre-Testing DLNA Testing Dolby and HE-AAC Testing Products Custom Test Suites Automation Test Tools & Harnesses HbbTV App Validator Evora DVB Test Suite Ligada isuite for DASH Test Suite CI+ Test Tool Ligada isuite for HbbTV Test Suite HEVC for DVB 3
Digital TV Labs & HbbTV Official Test lab for Freeview Aust + NZ DTT Supplier of TNT 2.0 Test Suite and DRM services Supply of TDT Hibrida Test Suite and Test Tools HbbTV Consultancy services OIPF Test Suite and Test Harness HbbTV Services Supplied Official Test Harness Custom HbbTV/OIPF Test Suite Test Suite and flexible Test Automation System Supply of HbbTV 2.0 and DRM Test Suite Early Member of HbbTV Association and part of the Steering Board & Marketing Group First Registered HbbTV Test Centre 4
Testing What test materials and procedures are available to manufacturers in that country? Any special tests for that country? Certification Testing and Certification Is there a logo and associated trade mark licensing scheme? What benefits does it bring? Right to use platform logo/brand Access to marketing campaign and brand awareness Receiver listed on website Access to client certificates How is receiver testing carried out? Self-certification versus certification Use of audits
Deploying HbbTV what is needed? Define and create the HbbTV applications Web application server infrastructure Application Testing Define and create the platform IP-video delivery: DASH origin server, Content Delivery Network (CDN), DRM solution (optional), DVB broadcast signalling Migration plan (MHP/MHEG/HbbTV co-existence) Platform Testing Ensure availability of HbbTV receivers in the market Platform specific receiver specification Conformance regime Certification and Interoperability Testing
HbbTV Broadcaster Heaven HbbTV App works on multiple Vendor STB/TV s Single version of the App is deployed and without need for updates New TV/STB platform software releases do not break App Testing the App requires limited effort Users have a reliable, good consumer experience App is released on time & within budget 7
HbbTV Broadcaster Hell Thailand NBTC found that ~70% of STBs in the market don't support HE-AAC properly and the audio goes mute on those services. Spec is basically too vague and no test regime in place. Recall of boxes had to happen Freeview Plus : various issues with Broadcaster Applications found - none conformance to certain TV s, HbbTV API s in conformance on client device, Applications written in PC environment and using extensions outside of HbbTV spec. Re-authoring of Apps was performed over the X-mas period. HbbTV App works differently on some platforms HbbTV App does not work on some platforms at all App need to be deployed as multiple versions New releases of App break on some platforms New TV/STB platform software updates break the App Testing the App is a massive TV platform debug session Users have an unreliable, poor consumer experience App is late to deployment and over Budget 8
HbbTV is Horizontal Market Pay-TV Operator Old Model HbbTV Operator New Model Receivers not controlled by operator/content owner Cannot guarantee app operation on every device MHEG-5 and MHP were first horizontal interactive deployments HbbTV will be first mass-deployment Complex TV standard Testing and conformance vital to interoperability Differences between TV/STB platforms may mean different app version for each device or at best conditional code branches 9
Why Test and Certify? Some markets use optional HbbTV behaviour. Extra tests can be required to get full coverage These tests shorten the roll-out as app-authors, receiver manufacturers, etc have something to work to. Testing platform with applications ok But leaves unknown areas untested New applications can fail and large support costs for broadcaster, platform manufacturer and app developer Proven solution: Use Official HbbTV Test Suite + Digital TV Labs Test Harness HbbTV Test Suite HbbTV App 1 API 1 API 3 API 4 Mandatory HbbTV App 2 HbbTV Platform Extra Test Suite API Untested API 5 API 6 API 7 Optional
Application Testing But - real apps may do things not (yet) covered by a Test Suite! Manufacturers will Test against applications in recorded streams and also those live on air Testing catch-up services can be problematic if they are geo-blocked Manufactures can only test against applications that are available (e.g. HbbTV Live Apps) The HbbTV Test Material Approval Procedure (TMAP) includes some suggestions: Interoperability Workshops Digital TV Labs have tools & services: Validator Syntax Check HbbTV Device Zoo Live Apps Hosting Testing by CE manufacturer PC HbbTV Simulation Selenium on Firefox HbbTV Selenium API Automation Device app automation testing
HbbTV Official Testing In-House Test Manufacturer Registered Test Centre Purchase Official Test Suite Lab Testing Purchase Test Harness (optional) Combined Effect of v1.3 Test Report Test Report HbbTV Logo TML Audit Programme Market Complaints Monitoring System 12
Conclusions Interoperability is challenging: testing much more important than in a simple digital broadcast world Testing with just applications is important, but not sufficient Implementations won t be inter-operable without a test suite Variety of different approaches to test and certification being adopted in the market The HbbTV Association s official test suite is a great resource and is available today Extended coverage for operator specific tests still needed Testing and certification helps migration to HbbTV and adoption of better services, and helps avoid confusing consumers Additional tests can be submitted back to HbbTV
THANK YOU! Any Questions? www.digitaltv-labs.com mike.tometzki@digitaltv-labs.com http://www.linkedin.com/company/digital-tv-labs http://twitter.com/digitaltvlabs