Home AV Networks. T Multimedia Programming. Outline. Architectures Media Centers Distributed Systems. Jari Kleimola November 2, 2006

Transcription

1 Home AV Networks T Multimedia Programming Jari Kleimola November 2, 2006 Outline Systems

2 Integration i standalone combos all-in-one STB STB + HDD DVD HDD VHS TV CD Radio Tape DVD + HDD DVD + VHS TV + STB TV + VHS HiFi Set Media Center A - preconfigured hw - preconfigured sw Media Center B - standard pc - software Intel VIIV AMD Live Hardware Tuner + capture hw HDD DVD/CD drives Graphic card Audio card Remote Control Software Media Player Recorder EPG Media DB Plugins Vinyl Amp

3 Network as AV System architecture Room 1 TV Speakers Room 2 Monitor Speakers Room 3 TV Stereos DMS DMA STB PC NAS (W)LAN AV data DVB Internet Camera Smart Phone Portable Gadget Digital Media Classes: DMS (Server) source acquire record store eg. STB, NAS, DVD DMP (Player) find content play eg. TV, stereos DMA (Adapter) interface legacy devices and network i DVD Laptop

4 Comparison architecture Room 1 TV Speakers DMS DMA (W)LAN AV data Media Center architecture NAS Extender Room 1 TV Speakers Room 2 Monitor Speakers STB PC NAS DVB Internet Camera Tuner DVD HDD PC Room 2 Monitor Speakers Room 3 Smart Phone Room 3 TV Stereos DVD Portable Gadget Laptop Extender /w DVD TV Stereos

5 What s the Difference? approach allows peer-to-peer connections in MediaCenters, everything goes through PC PC has to process all streams locally MediaCenters see only Extenders, not end devices approach scales better adhoc scenarios Platform and independence MediaCenters have unified UIs

6 i Outline Systems

7 i Windows Media Center (2005)

8 Windows Media Center (2005) HTML applications XHTML + CSS, JavaScript, VBScript DOM most of MSHTML DOM limited media center object model (eg. play, metadata) can only run on-demand Add-ins (managed code,.net framework 1) any.net language = C#, C++, VB.NET, JScript.NET,... namespaces: System, MediaCenter.* eg. playback, record, networking, extenders, HTML host control can also run in background dialog box UI

9 i Windows Media Center (Vista)

10 Windows Media Center (Vista) HTML applications (for backward compabitility) WMC Presentation Layer applications (.NET framework 2) any.net language MCML = Media Center Markup Language access to same UI rendering engine that WMC itself uses access managed code from markup XAML browser applications (XBAP) (.NET framework 3) WPF applications run in browser same concept as Java applets

11 i MediaPortal

12 MediaPortal Open source C#.NET framework 2 DirectX 9 Plugins any.net language types process (in background, eg. remote control input) tag reader (metadata parser for media files) external player proxy (eg. itunes, Flash) GUI with skin support (eg. Music Library)

13 Apple Front Row itv QuickTime, DVD Player, itunes and iphoto Apple remote control TV and Radio missing no public SDK STB with networking, USB, HDMI, AV still no TV (?) enhanced Front Row Q ($299) open source CenterStage itheater

14 i Some Others MythTV Linux open source (C++) plugins (C++) distributed architecture XBox Media Center modded XBox required open source (C/C++, asm) MS XBox SDK + MFC 7.1 plugins (Python)

15 i Outline Systems

16 Middleware architecture Room 1 TV Speakers Room 2 Monitor Speakers Room 3 DMS DMA STB PC NAS TV Stereos DVD Interoperability Firewire Jini ilink, HAVi, HANA Java RMI Bonjour addressing + discovery Apple, also for Windows UPnP Universal Plug and Play > 200 certified devices DLNA

17 UPnP Architecture architecture Room 1 TV Speakers DMS DMA Control Point UPnP discover devices send control commands to devices receive event notifications from devices can be any number of CPs can be inside a device just software code i Room 2 Monitor Speakers Room 3 TV STB PC NAS device description (XML) physical part logical part servicelist service description (XML) actions state variables embedded devicelist ui part Stereos DVD

18 i Description (physical) <?xml version="1.0"?> <root xmlns="urn:schemas-upnp-org:device-1-0"> <URLBase>base URL for all relative URLs</URLBase> <device> <friendlyname>short user-friendly title</friendlyname> <manufacturer>manufacturer name</manufacturer> <manufacturerurl>url to manufacturer site</manufacturerurl> <modeldescription>long user-friendly title</modeldescription> <modelname>model name</modelname> <modelnumber>model number</modelnumber> <modelurl>url to model site</modelurl> <serialnumber>manufacturer's serial number</serialnumber> <UDN>uuid:UUID</UDN> <UPC>Universal Product Code</UPC> <devicetype>urn:schemas-upnp-org:device:devicetype</devicetype> <servicelist> <service> <servicetype>urn:schemas-upnp-org:service:servicetype:v</servicetype> <serviceid>urn:upnp-org:serviceid:serviceid</serviceid> <SCPDURL>URL to service description</scpdurl> <controlurl>url for control</controlurl> <eventsuburl>url for eventing</eventsuburl> </service> Declarations for other services (if any) go here </servicelist> <devicelist>description of embedded devices (if any) go here</devicelist> <iconlist> <icon> <mimetype>image/format</mimetype> <width>horizontal pixels</width> <height>vertical pixels</height> <depth>color depth</depth> <url>url to icon</url> </icon> XML to declare other icons, if any, go here </iconlist> <presentationurl>url for presentation</presentationurl> </device> <specversion> <major>1</major> <minor>0</minor> </specversion> </root>

19 i Description (logical) <?xml version="1.0"?> <root xmlns="urn:schemas-upnp-org:device-1-0"> <URLBase>base URL for all relative URLs</URLBase> <device> <friendlyname>short user-friendly title</friendlyname> <manufacturer>manufacturer name</manufacturer> <manufacturerurl>url to manufacturer site</manufacturerurl> <modeldescription>long user-friendly title</modeldescription> <modelname>model name</modelname> <modelnumber>model number</modelnumber> <modelurl>url to model site</modelurl> <serialnumber>manufacturer's serial number</serialnumber> <UDN>uuid:UUID</UDN> <UPC>Universal Product Code</UPC> <devicetype>urn:schemas-upnp-org:device:devicetype</devicetype> <servicelist> <service> <servicetype>urn:schemas-upnp-org:service:servicetype:v</servicetype> <serviceid>urn:upnp-org:serviceid:serviceid</serviceid> <SCPDURL>URL to service description</scpdurl> <controlurl>url for control</controlurl> <eventsuburl>url for eventing</eventsuburl> </service> Declarations for other services (if any) go here </servicelist> <devicelist>description of embedded devices (if any) go here</devicelist> <iconlist> <icon> <mimetype>image/format</mimetype> <width>horizontal pixels</width> <height>vertical pixels</height> <depth>color depth</depth> <url>url to icon</url> </icon> XML to declare other icons, if any, go here </iconlist> <presentationurl>url for presentation</presentationurl> </device> <specversion> <major>1</major> <minor>0</minor> </specversion> </root>

20 i Description (ui) <?xml version="1.0"?> <root xmlns="urn:schemas-upnp-org:device-1-0"> <URLBase>base URL for all relative URLs</URLBase> <device> <friendlyname>short user-friendly title</friendlyname> <manufacturer>manufacturer name</manufacturer> <manufacturerurl>url to manufacturer site</manufacturerurl> <modeldescription>long user-friendly title</modeldescription> <modelname>model name</modelname> <modelnumber>model number</modelnumber> <modelurl>url to model site</modelurl> <serialnumber>manufacturer's serial number</serialnumber> <UDN>uuid:UUID</UDN> <UPC>Universal Product Code</UPC> <devicetype>urn:schemas-upnp-org:device:devicetype</devicetype> <servicelist> <service> <servicetype>urn:schemas-upnp-org:service:servicetype:v</servicetype> <serviceid>urn:upnp-org:serviceid:serviceid</serviceid> <SCPDURL>URL to service description</scpdurl> <controlurl>url for control</controlurl> <eventsuburl>url for eventing</eventsuburl> </service> Declarations for other services (if any) go here </servicelist> <devicelist>description of embedded devices (if any) go here</devicelist> <iconlist> <icon> <mimetype>image/format</mimetype> <width>horizontal pixels</width> <height>vertical pixels</height> <depth>color depth</depth> <url>url to icon</url> </icon> XML to declare other icons, if any, go here </iconlist> <presentationurl>url for presentation</presentationurl> </device> <specversion> <major>1</major> <minor>0</minor> </specversion> </root>

21 UPnP Architecture architecture Room 1 TV Speakers DMS DMA Control Point UPnP discover devices send control commands to devices receive event notifications from devices can be any number of CPs can be inside device just software code i Room 2 Monitor Speakers Room 3 TV STB PC NAS device description (XML) physical part logical part servicelist service description (XML) actions state variables embedded devicelist ui part Stereos DVD

22 i Service Description (actions) <?xml version="1.0"?> <scpd xmlns="urn:schemas-upnp-org:service-1-0"> <actionlist> <action> <name>actionname</name> <argumentlist> <argument> <name>formalparametername</name> <direction>in xor out</direction> <retval /> <relatedstatevariable>statevariablename</relatedstatevariable> </argument> Declarations for other arguments (if any) go here </argumentlist> </action> Declarations for other actions (if any) go here </actionlist> <servicestatetable> <statevariable sendevents= yes xor no > <name>variablename</name> <datatype>variable datatype</datatype> <defaultvalue>default value</defaultvalue> <allowedvaluerange> <minimum>minimum value</minimum> <maximum>maximum value</maximum> <step>increment value</step> </allowedvaluerange> </statevariable>state variables (if any) go here </servicestatetable> <specversion> <major>1</major> <minor>0</minor> </specversion> </scpd>

23 i Service Description (state) <?xml version="1.0"?> <scpd xmlns="urn:schemas-upnp-org:service-1-0"> <actionlist> <action> <name>actionname</name> <argumentlist> <argument> <name>formalparametername</name> <direction>in xor out</direction> <retval /> <relatedstatevariable>statevariablename</relatedstatevariable> </argument> Declarations for other arguments (if any) go here </argumentlist> </action> Declarations for other actions (if any) go here </actionlist> <servicestatetable> <statevariable sendevents= yes xor no > <name>variablename</name> <datatype>variable datatype</datatype> <defaultvalue>default value</defaultvalue> <allowedvaluerange> <minimum>minimum value</minimum> <maximum>maximum value</maximum> <step>increment value</step> </allowedvaluerange> </statevariable> Declarations for other state variables (if any) go here </servicestatetable> <specversion> <major>1</major> <minor>0</minor> </specversion> </scpd>

24 UPnP Architecture architecture Room 1 TV Speakers DMS DMA Control Point UPnP discover devices send control commands to devices receive event notifications from devices can be any number of CPs can be inside device just software code i Room 2 Monitor Speakers Room 3 TV STB PC NAS device description (XML) physical part logical part servicelist service description (XML) actions state variables embedded devicelist ui part Stereos DVD 1 XML master file for device + XML file for each service

25 i Standardized s Audio/Video MediaServer and MediaRenderer Home Automation Digital Security Camera HVAC Lighting Controls Networking Internet Gateway WLAN Access Point Printer and Scanner Printer Basic and Enhanced Scanner Others Remote UI Client and Remote UI Server Basic

26 AV s and Services MediaServer ContentDirectory ConnectionManager AVTransport (optional) MediaRenderer RenderingControl ConnectionManager AVTransport (optional)

27 UPnP AV version 2 Backwards compatible with version 1 MediaRenderer ScheduledRecording service added ContentDirectory mutation actions added ContentDirectory more metadata (eg. EPG) AVTransport recording and DRM related actions and state variables More detailed specifications Better documentation

28 i Play Video Clip using UPnP TV acts as a media renderer (sink) DVR acts as a media server (source) Control Point is implemented here CyberGarage UPnP stack (Java) It is easy to implement devices as well Steps needed: Addressing Discovery Description Control (Eventing)

29 i Play Video Clip using UPnP CyberGarage defines abstract ControlPoint /* * SimpleControlPoint.java * (c) Jari Kleimola 2006 */ package simplecontrolpoint; import org.cybergarage.upnp.controlpoint; // ========================================================= // public class SimpleControlPoint extends ControlPoint { // -- ctor public SimpleControlPoint() { start(); } }

30 0. Addressing Control Point, TV and DVR get IP addresses use DHCP server if available otherwise, use AutoIP IETF draft: pick address from /16 range check if it is already in use (ARP) periodically check for DHCP server CyberGarage UPnP stack manages this for us automatically Addressing Discovery Description Control Eventing

31 1. Discovery s advertise themselves when joining network repeated periodically cancel advertisements when leaving Control Points search for devices based on device or service type, or unique ID devices respond with their URL Protocol SSDP = Simple Service Discovery Protocol HTTPMU/UDP for advertising and search requests HTTPU/UDP for search responses NOTIFY and M-SEARCH verbs all data is in HTTP header fields Addressing Discovery Description Control Eventing

32 i 1. Discovery // ========================================================= // public class SimpleControlPoint extends ControlPoint implements SearchResponseListener { private tv = null; private dvr = null; // -- ctor public SimpleControlPoint() { addsearchresponselistener(this); start(); search("upnp:rootdevice"); } } // -- virtual public void devicesearchresponsereceived(ssdppacket packet) { String uuid = packet.getusn(); String udn = USN.getUDN(uuid); } dev = get(udn); String type = dev.gettype(); if (type.equals("urn:schemas-upnp-org:device:mediarenderer:1")) tv = dev; else if (type.equals("urn:schemas-upnp-org:device:mediaserver:1")) { } dvr = dev; browsemediadb("0"); // superclass has all devices in a list // shown later Addressing Discovery Description Control Eventing

33 2. Description Control Point requests the device description XML URL was returned in search response responds with XML data control point parses the XML, and gets service description URLs Control Point requests service description XMLs responds with XML data control point parses the XMLs Protocol HTTP GET URL was in LOCATION header field of the NOTIFY response XML data in response content Addressing Discovery Description Control Eventing

34 i 2. Description CyberGarage handles parsing automatically: For ControlPoint, we can directly invoke getlist() list of devices in the UPnP network get( id ) device with specific id For getservicelist() list of services offered by the device getservice(xxx) service with specific id, class or url getaction( ) shortcut getstatevariable( ) shortcut getxxx() access any field of device description XML For Service getaction(...) list of actions or specific action getstatevariable(...) list of state variables or specific variable Addressing Discovery Description Control Eventing

35 3. Control Control Point requests content metadata from DVR service is ContentDirectory action is Browse() responds with DIDL-Lite XML content subset of MPEG-7 metadata control point parses the tree structure and displays it user browses the directory, and selects one item to play Protocol HTTP POST content is SOAP SOAP body contains action method and parameters: <urn:actionname> this is actionnameresult in response <urn:argumentname>value</urn:argumentname> more arguments <urn:actionname> Addressing Discovery Description Control Eventing

36 i 3. Control public void browsemediadb(string parent) { Service cds = dvr.getservice("urn:schemas-upnp-org:service:contentdirectory:1"); Action act = cds.getaction("browse"); act.setargumentvalue("objectid", parent); // unique id within MediaServer act.setargumentvalue("browseflag", "BrowseDirectChildren"); act.setargumentvalue("filter", "*"); // what fields are requested act.setargumentvalue("sortcriteria", "+dc:title"); // sort items according to this field act.setargumentvalue("startingindex", 0); // for page-by-page browsing act.setargumentvalue("requestedcount", 0); // for page-by-page browsing if (act.postcontrolaction()) { ArgumentList args = act.getoutputargumentlist(); Argument arg = args.getargument("result"); String strdidl = arg.getvalue(); // this is XML Properties mediaitems = parsedidl(strdidl); // get title id pairs parentframe.showitems(mediaitems); // show items in a listbox } } another ContentDirectory action is Search() all albums by a specific artist all photos taken in june by myself Addressing Discovery Description Control Eventing

37 3. Control Control Point sets streaming protocol contract ConnectionManager::GetProtocolInfo() a matching pair of protocol and format is seeked ConnectionManager::PrepareForConnection() Control Point passes clip s URL to TV action is AVTransport::SetAVTransportURI() uri was given by DVR in response to Browse() Control Point instructs TV to play action is AVTransport::Play() TV and DVR handle streaming OOB according to the protocol set in PrepareForConnection() Protocol is the same as in Browse() Addressing Discovery Description Control Eventing

38 i 3. Control // item is URL public boolean play(string item) { // -- stop if previously playing, just in case Action stop = tv.getaction("stop"); stop.setargumentvalue("instanceid", 0); stop.postcontrolaction(); // -- eliminate spaces etc. String uri = decode(item); } // -- set URL and start streaming Action seturi = tv.getaction("setavtransporturi"); seturi.setargumentvalue("instanceid", 0); seturi.setargumentvalue("currenturi", uri); seturi.setargumentvalue("currenturimetadata", ""); if (seturi.postcontrolaction()) { Action play = tv.getaction("play"); play.setargumentvalue("instanceid", 0); play.setargumentvalue("speed", "1"); return play.postcontrolaction(); } else return false; Addressing Discovery Description Control Eventing

39 4. Eventing Control Point subscribes to TV s transport state SUBSCRIBE to url that was sent in service description subscription is for the entire service TV notifies when state changes NOTIFY sent to control point s server socket Control Point unsubscribes UNSUBSCRIBE Protocol GENA = General Event Notification Architecture HTTP verbs SUBSCRIBE and UNSUBSCRIBE for control point HTTP verb NOTIFY when there s new event simple push model Addressing Discovery Description Control Eventing

40 UPnP Summary SDKs CyberGarage (Java,C,C++,Perl) Microsoft, Intel, Siemens, etc. Links