Variable and State Handling in NCL

Transcription

1 Variable and State Handling in NCL ABSTRACT Luiz Fernando G. Soares 1, Rogério Ferreira Rodrigues 1,2, Renato Cerqueira 1, Simone Diniz Junqueira Barbosa 1 lfgs@inf.puc-rio.br, roger@microsoft.com, rcerq@inf.puc-rio.br, simone@inf.puc-rio.br 1 Pontifícia Universidade Católica do Rio de Janeiro Rua Marquês de São Vicente Rio de Janeiro, RJ Brazil ext: Microsoft Corporation / FAST, a Microsoft Subsidiary Av. Rio Branco 1, Rio de Janeiro, RJ Brazil Most time-based declarative languages have limited support for variable definition and manipulation, which causes developers to resort to imperative languages. However, a declarative language should provide a variable handling model sufficiently rich to describe a wide range of interactive applications, avoiding, as much as possible, the help of an imperative scripting language. On the other hand, the declarative simplicity should not be lost, leaving for the imperative objects more complex manipulations, with the necessary care to avoid any impact in the application s temporal graph. Based on this principle, variables and the presentation state are handled by NCL and Ginga-NCL, as discussed in this paper. NCL is the declarative language of the Brazilian Terrestrial Digital TV System (SBTVD) supported by its middleware called Ginga. NCL and Ginga-NCL are part of ISDB standards and also of ITU-T Recommendations for IPTV services. Categories and Subject Descriptors I.7.2 [Document Preparation]: Languages and systems, markup languages, multimedia, hypermedia, standards. D.3.2 [Language Classifications]: Specialized application languages General Terms Standardization, Design. Keywords Variable handling, declarative languages, NCL, digital TV, middleware. 1. INTRODUCTION Declarative languages emphasize the description of an application, rather than its decomposition into an algorithmic implementation as in imperative languages. Declarative languages can be defined based on a specific model to design applications in a target domain, giving rise to Domain Specific Languages (DSL). Such declarative descriptions are closer to a high-level specification, and thus they are easier to be written and understood than imperative ones, which usually require programming expertise. This paper is particularly interested in time-based declarative application languages (time-based DSL), in the context of multimedia document engineering. Time-based declarative languages define abstractions to specify the spatial and temporal synchronization among components (media objects) of an application, which

2 makes them especially suitable for multimedia applications. The synchronization targets include not only traditional media objects such as video, audio (including streaming video and audio), image, text and others, but also media objects that contain imperative and other declarative code. There are many examples of such languages applied to the multimedia application domain, for instance: NCL (Nested Context Language) [13], the ITU-T Recommendation H.761 for IPTV services [10] and the standard declarative language of the Brazilian terrestrial digital television (DTV) System [1]; XHTML-based DTV middleware languages [2] [4]; SMIL (Synchronized Multimedia Integration Language) [20]; SVG (Scalable Vector Graphics) [22], BIFS-XMT-O (Binary Format for Scenes of extensible MPEG-4 Textual) [8], and MPEG-4 LASeR (Lightweight Application Scene Representation, part 20) [9], based on SVG and other extensions. In multimedia document processing, variable handling is a key feature to provide better presentation control. Variables can be local to a media object (e.g., determining the object s position) or global to the exhibition context (e.g. defining the available network bandwidth). In addition, the access to the multimedia presentation state (e.g. defining for a given moment in time which media objects are occurring, paused, etc.) is also an important aspect for guiding the execution control. In the context of multimedia document engineering, especially those targeting digital TV applications, timebased declarative languages should support the following features related to variable and state handling: defining local and global variables (the latter with several different scopes, that is, different types of persistency and access rights); adapting content and content presentation based on the values of global variables; conditioning navigation to the values of both local and global variables, as evaluated in simple or compound expressions; setting values to both local and global variables, depending on the values of other variables, on the navigational flow and on the spatial behavior of an application; continuous setting of values (animation) to both local and global variables; storing and sharing the current context (state) of an application execution, for its future retrieval and, therefore, for resuming the application at a later time. In order to exemplify the importance of variable and state handling in the DTV domain, and the requirements presented in the previous bullets, consider the following DTV interactive application for the animation called The First John : 1. The film, an animation video about a famous soccer player, is received by broadcasting and exhibited in a TV receiver. By datacasting, several other media objects are received and played synchronously according to an application specification (a multimedia document specification), also received by datacasting. 2. After 12 seconds from the animation beginning, a real photo of the soccer player is presented on the screen. Depending on TV receiver s screen size (specified by a system global variable) the photo will be presented with or without transition effects, illustrating a presentation adaptation of the same data content. 3. Afterwards, still during the animation video, a soccer shoes advertisement may be played, if interactive advertisements are allowed (determined by a service global variable). a) If the advertisement is allowed and when it is ready to be presented, an icon will appear on the screen (upper-right corner in Figure 1), during a certain period of time, in order to notify the presence of interactive content. b) If a viewer wants to buy soccer shoes, he must select the icon: the icon presentation will stop, the animation video will be resized (by manipulating its placement local variables), an advertisement and an HTML form will appear on the screen, all of them over a background picture, as shown in Figure 2. c) The advertisement will be presented either as a video or as an image, depending on the receiver s processing power (determined by another system global variable). If the video is chosen to be presented, it will be presented either with a Portuguese subtitle or without subtitles, depending on the viewer s language (specified by a user global variable), in this case illustrating a content adaptation. d) The form placement starts at the bottom of the screen but its top local variable is then changed continuously in order to centralize the vertical position of the form presentation, illustrating an animation effect.

3 e) When the form is started, a media object containing imperative code is also started. This imperative object is responsible for getting the form initial position and for calculating the final centralized position. The computed value will then be used to set form s top local variable. f) A viewer can fill the form while it has the input key control, which was obtained by setting another service global variable when the form was started. g) The end of the advertisement video will end the exhibition of the form and of the background picture, besides resizing the animation video to its original size. Moreover, consider that a viewer can tune the TV channel at any moment and must be able to see not only the animation video but also the objects that compose the whole application in perfect synchronization among themselves (that is, to be able to see the whole multimedia document presentation). Figure 1 Interactivity allowed Figure 2 Advertisement presentation In general, the variable handling in time-based declarative languages is usually limited. A wide-ranging manipulation, when possible, is done by using imperative objects (media objects that contain imperative code), usually coded using a scripting language. A declarative language should offer a good balance between expressiveness and simplicity. It should provide a variable handling model sufficiently rich to describe a wide range of interactive applications, avoiding, as much as possible, the help of an imperative scripting language. At the same time, the declarative simplicity should not be lost, leaving for the imperative objects more complex manipulations, with the necessary care to avoid any impact in the application s temporal graph. Based on this principle, variables and the presentation state are handled by NCL and Ginga-NCL. This paper presents the NCL and Ginga-NCL approach. NCL and Ginga-NCL provide wide expressiveness in the declarative handling of local and global variables1 and in managing the program presentation state. GingaNCL [1] [10] is the NCL presentation environment composed by the NCL Formatter (or Player) and the NCL Private Base Manager. The NCL Formatter is in charge of receiving an NCL document (application) and controlling its presentation, trying to guarantee that specified relationships among media objects are respected. The formatter deals with NCL documents that are collected inside a data structure known as private base. NCL documents in a private base may be started, paused, resumed, stopped and may refer to each other. The Private Base Manager is in charge of receiving NCL document editing commands and maintaining the active NCL documents (applications being presented), as discussed in Section 5. The approach presented in this paper is not limited to NCL, however. NCL modules can be used to add NCL facilities to other XML-based languages. Thus, the ideas presented can also be inherited by other languages through using NCL modules. Alternatively, the ideas can be taken as a design principle to be incorporated into other time-based declarative language models. 1 In NCL, variables are called properties. In the remainder of this document, the terms variable and property will be used interchangeably when referring to NCL variables.

4 The paper is structured as follows. Section 2 discusses related work regarding XML application languages. Section 3 presents how local and global variables are declared in NCL. Section 4 describes how these variables can be handled by declarative language elements. Section 5 introduces an overview of the multimedia-document state management performed by Ginga. Section 6 discusses how the scripting language of NCL can manipulate variables in a controlled way, without impairing the application s temporal graph. Finally, Section 7 presents the paper final remarks. 2. RELATED WORK XHTML-based declarative languages, such as BML [2] and DVB-HTML [4]], allow declarative manipulation of formatting variables through style sheets, usually written in CSS [14]. Document elements and attributes can also be manipulated through the document tree, using DOM events [18]. This last case, however, usually needs imperative coding, commonly written in ECMAScript [3]. DOM events and external events 2 can trigger a scripting object to provide temporal and spatial synchronization, content and presentation adaptability, and other facilities that are not otherwise possible in XHTML. The approach used to integrate XHTML and ECMAScript for variable manipulation is very intrusive. The ECMAScript code can be written embedded in XHTML documents, or even more intrusively, inside attributes of XHTML elements. The limit between the two programming paradigms is too flexible, making difficult to decouple code and to distribute development work among different programming teams. Moreover, ECMAScript can access and modify the XHTML DOM tree under the total control of the programmer, which may cause unpredictable effects. In contrast, languages like BIFS XMT-O [8], SMIL [20], HTML+Time [19], SVG [22] and LASeR [9] provide a better support for declarative manipulation of variables. They integrate SMIL timing concepts within their framework and provide support for a subset of SMIL system global variables, as it will be described. SVG and HTML+Time provide support to query and set their document tree nodes and attributes using DOM. MPEG-4 LASeR is based on SVG Tiny profile with some other extensions. SMIL allows local positioning-variable definitions and their manipulation. Early SMIL versions allow testing global system-defined variable values, within the language element named <switch>. The first child element of a <switch> (a media object or a composite object) which has its selection predicate evaluated as true, is selected for activation. From version 2.0 on, it has been allowed to attach selection predicates directly to SMIL elements, no longer needing a <switch> element to condition their activation. Until version 2.1, the activation control of a SMIL element was limited to a simple predicate logic evaluation, comparing the predicate value to an absolute value and returning a Boolean value as a result. Moreover, global test variables have been implemented placing most of their initialization and control within the SMIL engine (user agent), rather than within the language. A new proposal for state variable 3 maintenance has been introduced in SMIL 3.0 [11]. In the new version, variables can be tested using a richer arithmetic and logical operator set (=,!=, <=, <, >=, >, and, or, not, +, -, * and div, when XPath [23]syntax is used), besides the previously used = operator. Variables are tested in expressions defined within a new attribute named expr. The new proposal also obliges the expression evaluation each time an element is activated, solving a previous ambiguity of SMIL that also allowed an evaluation during a document parsing. In the new version 3.0 [21], variables can also be defined, referenced and changed through a new <setvalue> SMIL element. The value to be set can be a simple absolute value, or evaluated by an expression defined in the element s expr attribute. The <setvalue> element also defines a begin attribute, specifying when the assignment must happen. Continuous attributions (animations), however, are not allowed in <setvalue> elements. 2 In DTV systems, it is usual to have DSM-CC stream events triggering the execution of imperative codes. 3 SMIL uses the term state variable instead of global variable.

5 Although the new SMIL version recognizes the importance of defining a standard mechanism for saving, restoring and sharing the multimedia presentation state as a whole, it does not define such mechanism. The whole presentation state can only be partially inferred (coarse tuning) by using <setvalue> elements appropriately placed inside a document (similar to what is done when one tries to define debugging points in a program). A fine multimedia presentation state control should allow maintaining the execution history of an application. This is very important when an application needs to migrate from an exhibition context to another one, as for example, from a display with partial screen exhibition to a full screen, from one device to another (e.g. from a TV set to a mobile phone), or even when an application must be stopped and later resumed, from the same interrupted point. These are common situations found in a DTV environment where, in addition, it is very usual to stop a presentation and then to resume it in a future moment in time, preserving all actions previously executed. This situation often happens, for example, when a viewer changes the TV channel and then returns to it, trying to get away from advertisements or simply looking for the beginning of an event in another channel. To manage the state control of a multimedia presentation is to manage the application s temporal graph. Besides allowing for dynamic navigation (i.e., changes in the graph traversal), the presentation state management should allow saving the graph context, at least the current position in the graph. Based on the imperative language syntax, King et al. [12] propose a set of extensions to XML application languages. They focus on the possibility of specifying attribute values using expressions that can be dynamically evaluated, and on allowing a document author to define events based on predicate expressions. Although the proposal has been discussed in the context of SVG and SMIL, it was not included in these languages recommendations. NCL provides wider expressiveness in the declarative handling of local and global variables and in managing the presentation state than the aforementioned related work, as discussed in the next sections. The NCL facilities implemented in Ginga-NCL address all features mentioned in the bullets of Section 1 [15]. Furthermore, NCL recognizes that the richer expressiveness of imperative languages cannot be ignored, mainly for applications that produce a lot of dynamic contents, as a result of complex operations. Therefore, NCL also includes support for imperative objects; in particular imperative objects specified using the NCL s scripting language called NCLua [6]. 3. NCL VARIABLES NCL allows defining local and global properties both implicitly, by the NCL user agent (the NCL formatter/engine/player), and explicitly, by the application author. To explicitly define a variable, we use an NCL <property> element, containing a name attribute, to identify the property, and optionally a value attribute, to assign an initial value to it. Properties defined inside the same parent element must have different values for the name attribute (i.e., the variable name is unique in its scope). As an example, to define the local property named viewed to a certain media object, with the initial value of false, we would write 4 : <media id="myphoto" src="someone.png" descriptor= mydescriptor > <property name="viewed" value="false"/> </media> 4 The src attribute of a <media> element specifies the location for retrieving the media object s content. The attribute descriptor specifies how the media object must be presented.

6 Listing 1. Definition of a local variable. The <property> element can also be used to define a group of properties. In The First John example of Section 1, the dimensions and location of the media object that represents the animation video may be defined either as individual left, top, width and height properties, or as a grouped bounds property, whose value has the format left,top,width,height, as follows 5. When a group of variables is changed, its consistency is checked only at the end of the process. <media id="reusedanimation" refer="animation" instance="instsame"> <property name="bounds"/> </media> Listing 2. Definition of a group of variables. Global variables can be defined by the document author or can be reserved environment variables 6. The global variable type attribute defines the variable scope, its persistency and its access rights. It is important to mention that this attribute has no relation with the type of the attribute value, which is implicitly obtained from the variable semantics. The Ginga middleware defines the following global variable types for the NCL EDTV (enhanced DTV) profile [1] [10]: system and user (e.g. system.language, user.location): variables that are managed by the receiver system. They may be read, but they may not have their values changed by an NCL application, a Lua procedure or an Xlet procedure 7. Their values can be changed only by the receiver s native applications. They persist during the receiver s life cycle. default (e.g. default.selborder, the default color applied to the border of an element in focus when activated): variables that are also managed by the receiver system. However, they may be read and have their values changed by an NCL application, by the receiver s native applications, by a Lua procedure or by an Xlet procedure. They persist during the receiver s life cycle. They are reset to their initial default values each time a new channel is tuned. service (e.g. service.currentfocus, which indicates the <media> element currently on focus): variables managed by the NCL formatter. They may be read and have their values changed by an NCL application of the same service. They may also be read but they may not have their values changed by a Lua procedure or an Xlet procedure of the same service. They persist at least during the service s life cycle. si (eg. si.channelnumber, the number of the current tuning channel) variables that are managed by the receiver middleware. They may be read, but they cannot have their values changed by an NCL application, by a Lua procedure or by an Xlet procedure. They persist at least until the next channel tuning. channel (in the format channel.xxx): variables managed by the NCL engine. They may be read and have their values changed by an NCL application of the same channel. They may also be read but they may not have their values changed by a Lua procedure or an Xlet procedure of the same channel. They persist at least until the next channel tuning. shared (in the format shared.xxx): variables managed by the NCL formatter. They may be read and have their values changed by an NCL application. They may also be read but they may not have their values changed by a Lua or an Xlet procedure. They persist at least during the life cycle of the service that has defined them. 5 The <media> element reusedanimation represents the same animation video (attribute instance= instsame ) that is specified by the <media id="animation" src="../media/animgar.mp4" descriptor="screendesc"> element (attribute refer= animation ). 6 Reserved variables are the ones whose name and semantics are defined in the NCL profile specification [1] [10]. 7 Lua is the Ginga scripting language, while Xlet is the Java code that runs in Ginga.

7 To declare a global variable, a <property> element is used as a child element of the NCL settings object, represented by a <media> element of "application/x-ncl-settings" type. A settings object must be unique in an NCL document and groups all global variables, either explicitly or implicitly declared. However, variables must always be explicitly declared if they are to be manipulated by an NCL application. As an example of global variable declaration, in The First John application described in Section 1 it is necessary to check whether interactive advertisements are to be presented or not. If an author wants to configure this per channel, and to assume the value true as the default, he might declare a global variable of the channel type called interactivity, as follows: <media id="globalvar" type="application/x-ginga-settings"> <property name="channel.interactivity" value="true"/> <property name="service.currentkeymaster"/> </media> Listing 3. Definition of global variables. Since the variable type is channel, it will persist for every event (service or program) in exhibition until the next channel tuning. In the same code chunk, a global variable service.currentkeymaster is also defined This is a reserved variable, managed by Ginga-NCL, and establishes which media player has the key navigation (remote) control. Local variables can also be either defined by the document author or reserved variables. Local variables may be read and have their values changed only by an NCL <link> element, as discussed in Section 4. They shall persist at least during the life cycle of the NCL object that has defined them. There are several reserved names for local variables, such those for an object spatial positioning, and those for defining additional presentation characteristics (e.g. duration and sound level). As an example, the First John application declares the reserved height and top local properties and initializes the height one, as follows: <media id="form" src="../media/form.htm" descriptor="formdesc"> <property name="height" value="18,33%"/> <property name="top"/> </media> Listing 4. Example of reserved local variables. 4. DECLARATIVE HANDLING OF VARIABLES IN NCL In NCL, media objects, including imperative and declarative objects (objects whose content are imperative and declarative code spans, respectively), are related (synchronized) using <link> elements. In order to specify relationships among objects, NCL links are bound to media object interfaces: <area> elements (defining part of the media object content) and <property> elements, as previously defined. Section 4.2 deals with variable manipulations by <link> elements. Variables can also be used to select an object to be presented from a set of objects, or to select how an object must be presented, from a set of alternatives in adaptive NCL applications, as discussed in the next section. 4.1 Adapting Content and Presentation NCL allows the definition of selection rules based on global variables. These rules can be used to select applications content and content presentation, from a set of alternatives. Rules can be simple, defined by the <rule> element, or compound, defined by the <compositerule> element. Simple rules test global variables

8 against specified values, using comparison operators ( eq, ne, gt, lt, gte, lte ). Compound rules test simple or compound rules joined through logical operators ( and, or ). NCL rules are grouped into a base (<rulebase> element) specified in the document header. As an example, The First John application declares a simple rule to test if the receiver s screen size corresponds to that of a portable receiver or not. <rulebase> <rule id="portabledev" var="system.screengraphicsize" value="288,352" comparator="lt"/>... </rulebase> Listing 5. Simple rule definition. Compound rules can also be defined, for example, to test the language to be used in subtitles in The First John application when the receiver s CPU capability is high, as follows: <rulebase>... <compositerule id="advcpuen" operator="and"> <rule id="advcpu01" var="system.cpu" value="333" comparator="ge"/> <rule id="npt" var="system.language" value="pt" comparator="ne"/> </compositerule> <compositerule id="advcpupt" operator="and"> <rule id="advcpu02" var="system.cpu" value="333" comparator="ge"/> <rule id="pt" var="system.language" value="pt" comparator="eq"/> </compositerule> </rulebase> Listing 6. Composite rule definition. Rules may be associated to child elements of <switch> elements. The first child element that has its associated rule evaluated as true is then selected for activation. Rules are dynamically evaluated every time the <switch> element becomes active. For instance, to determine the most suitable media object to be presented as an advertisement in The First John application, we might specify the following <switch> element: <switch id="shoes"> <bindrule constituent="shoesen" rule="advcpuen"/> <bindrule constituent="shoespt" rule="advcpupt"/> <defaultcomponent component="shoesphoto"/> <media id="shoesen" src="../media/shoesen.mp4" descriptor="shoesdesc"/> <media id="shoespt" src="../media/shoespt.mp4" descriptor="shoesdesc"/> <media id="shoesphoto" src="../media/shoesphoto.png" descriptor="shoesdesc"/> </switch> Listing 7. Switch example. The <switch id="shoes"> specifies that if the receiver s CPU capability is low, an advertisement photo must be exhibited (default component presentation when all rules are not satisfied). If the receiver s CPU capability is sufficient to support an advertisement video, depending on the viewer language, a video (spoken in English) will be exhibited without or with Portuguese subtitles. In NCL, <descriptor> elements are used to separate the document content and structure specification from the definition of its presentation behavior. As it can be observed in previous code spans, each media object

9 is associated with a descriptor. The <descriptor> element specifies temporal and spatial information needed to present each media object. A <descriptor> element may have <descriptorparam> child elements, which are used to initialize local variables (<property> elements) defined in the associated media object. Different from other declarative languages, NCL allows rules to be also used to select the presentation characteristics of a media object, through the <descriptorswitch> element. A <descriptorswitch> element groups a set of alternative <descriptor> elements. Similar to the <switch> element, rules can be associated with <descriptorswitch> s child <descriptor> elements. The first rule evaluated as true makes the associated <descriptor> element the chosen one. As usual, rules are dynamically evaluated each time a <descriptorswitch> element becomes active. As an example, to apply a transition effect 8 only if a minimum receiver s screen size is available, we could specify a <descriptorswitch> as follows: <descriptorswitch id="photodesc"> <bindrule constituent="withouttrans" rule="portabledev"/> <defaultdescriptor descriptor="withtrans"/> <descriptor id="withouttrans" region="framereg" explicitdur="5s"/> <descriptor id="withtrans" region="framereg" explicitdur="5s" transin="trans1" transout="trans2"/> </descriptorswitch> Listing 8. Descriptor switch example. 4.2 Correlation Between Variable Handling and the Presentation s Flow Control The major difference between NCL and related declarative languages is its wealth of declarative possibilities in the use of variables to define the temporal and spatial behavior of a document presentation. NCL allows the declarative manipulation of its data model to transform its flow control model, and vice-versa, without compromising the temporal consistency of the presentation. Even manipulations using imperative objects are controlled, as discussed in Section 6, different from what happens with common uses of scripting languages. The NCL spatial and temporal relationships among media objects are based on the causality/constraint paradigm. Relationships are specified using links (represented by <link> elements). Causal relationships specify conditions (link sources) on event states or media object property (variable) values. When conditions are satisfied, actions (link targets) are performed. Actions may change the state of other events or set values to properties. An NCL event is an occurrence in time that may be instantaneous or have a measurable duration. NCL defines the following basic types of events (which can be extended): presentation event, corresponding to playing a content anchor (whole media-object content, or part of this content); selection event, corresponding to a viewer interaction (selection of a content anchor); and attribution event, corresponding to setting a value to a variable (<property>). Content anchors for a <media> element are defined by its <area> child elements. As an example, in The First John application, its is necessary to define two content anchors to synchronize the presentation starting time of the soccer player photo at near the beginning of the animation video, and the soccer icon for viewer interaction in a later moment, as follows: <media id="animation" src="../media/animgar.mp4" descriptor="screendesc"> <area id="segphoto" begin="12s"/> <area id="segicon" begin="45s" end="51s"/> 8 Transition effects are defined in NCL by <transition> elements, and referred by <descriptor> elements through its transin and transout attributes.

10 </media> Listing 9. Content anchor definition. In the previous code span, a content anchor is defined from 12s to the end of the animation (the start of this anchor will trigger the photo presentation) and another anchor is defined during 6 seconds, the maximum time that the soccer icon will be available to start the advertisement. In NCL each event defines a state machine that should be maintained by the NCL formatter. An event can be in the sleeping, occurring or paused state, and can change its state upon receiving one of the following actions: start, stop, pause, resume and abort, as shown in Figure 3Error! Reference source not found.. The current state of each event state machine should be maintained by NCL formatter. Figure 3 - Event state machine. Turning back to The First John application, when the content anchor segphoto begins, the photo of the soccer player must be presented. This specific relationship (defined by a <link> element relating the two media objects) refers to a more generic relation (defined by an <xconnector> element), which in its turn specifies the desired behavior. The following code fragment illustrates such a link: <link id="lphoto" xconnector="onbeginstart"> <bind role="onbegin" component="animation" interface="segphoto"/> <bind role="start" component="photo"/> </link> Listing 10. Example of a simple <link> element. This link can be read as: Following the behavior specified in the connector onbeginstart, when the presentation of animation video s segphoto anchor starts (onbegin role), then starts the presentation of the photo media object (start role). Note that, as previously mentioned, when the photo presentation is started, it can be exhibited with or without transition effects, depending on the global variable tested by the <descriptorswitch> element associated to <media id= photo >. Two more sophisticated <link> elements are used to start the presentation of the soccer advertisement: the first one to condition the presentation of the interactive content, and the other one to present the advertisement when the soccer icon is selected, as shown in Listing 11. The first link can be read as: Following the behavior specified in the connector onbeginteststart, when the presentation of animation video s segicon anchor starts (onbegin role), if the value of the global property channel.interactivity is true (testvar role), then start the presentation of the icon media object (start role). Note the use of bind parameters (<bindparam> elements) for testing the value true (parameterized as var ) against the channel.interactivity variable value. In other words, note the use of variables in condition roles to control the navigational flow of a document presentation.

11 The second link can be read as: Following the behavior specified in the connector onkeyselectionstopsetstart, when the icon is selected (onselection role) with the RED push button, then: (1) start the presentation of the background media object (start role); (2) set the value of the bounds property of the animation media object (set role) to "5%,6.67%,45%,45%" of its current value (i.e., reposition and resize the mainvideo media object); (3) start the presentation of the shoes and form media objects (start roles); (4) start the presentation of the NCLua media object (start role); and (5) set the value of the service.currentkeymaster global variable to the form media object, giving to this object s player the key navigation control. <link id="licon" xconnector="onbeginteststart"> <bind role="onbegin" component="reusedanimation" interface="segicon"/> <bind role="testvar" component="globalvar" interface="channel.interactivity"> <bindparam name="var" value="true"/> </bind> <bind role="start" component="icon"/> </link> <link id="lbegingshoes" xconnector="onkeyselectionstopsetstart"> <bind role="onselection" component="icon"> <bindparam name="keycode" value="red"/> </bind> <bind role="stop" component="icon"/> <bind role="start" component="background"/> <bind role="set" component="reusedanimation" interface="bounds"> <bindparam name="varset" value="5%,6.67%,45%,45%"/> </bind> <bind role="start" component="shoes"/> <bind role="start" component="form"/> <bind role="start" component="lua"/> <bind role="set" component="globalvar" interface="service.currentkeymaster"> <bindparam name="var" value="lua"/> </bind> </link> Listing 11. Examples of <link> elements. Note in the second link the use of bind parameters (<bindparam> elements): first for setting a group of values parameterized as bounds to a group of local variables with the same name; second for setting a value to a global variable. In other words, note the use of variables in action roles to set values to document variables. Links can also be used as a declarative control to set values to a variable based on values of another one, as shown in the next code span. In this example, when the presentation of the form media object starts (onbegin role), then the NCLua media object s calc property is set (set role) to the value got from the form media object s height property (get role). <link id="lbegincalc" xconnector="onbeginset"> <bind role="onbegin" component="form"/> <bind role="get" component="form" interface="height"/> <bind role="set" component="lua" interface="calc"> <bindparam name="var" value="$get"/> </bind> </link> Listing 12. Example of link to get a value from a property and set it to another one

12 Constraint relationships 9, without any involved causality, can also be specified by using NCL links. Consider, for instance, the following constraint defining that two local variables of different media objects, in the case the left variables, must have the same value (thus, the objects must always be left aligned). <link id="linkleftalign" xconnector="maintainvarequal"> </link> <bind role="left1" component="advertglasses" interface="left"/> <bind role="left2" component="advertlegend" interface="left"/> Listing 13. Example of constraint relationship definition. 4.3 Continuous Variable Settings NCL animation primitives allow progressively changing variable values, throughout a specified time lag. Basically, NCL allows two new attributes to be associated with assignment actions in causal links: the duration and by parameters. By default, variable set operations are performed instantaneously. However, if the duration parameter is specified, the assignment operation will take the specified period of time. In this case, the change from the old value to the new one can be linear or discrete. In the latter case, the step modification is specified in the by attribute. Together, duration and by attributes offer support to presentation animations, easily and declaratively specified. The following code span specifies a horizontal movement (changing the right property to 50% of its current value) for the photo media object of the soccer player in the The First John application. The animation begins as soon as the object presentation starts. It is performed step by step (10 pixels at a time, as indicated by the by parameter), and the movement will take 2 seconds (duration parameter). <link id="linkchangetopposition" xconnector="onbeginsetanim"> </link> <bind role="onbegin" component="photo"/> <bind role="set" component="photo" interface="right"> </bind> <bindparam name="var" value="50%"/> <bindparam name="duration" value="2s"/> <bindparam name="by" value="2"/> Listing 14. Progressive variable setting. In the next example, when the newtop property of the Lua component (NCLua object) is set, the newtop value must be retrieved and linearly assigned to the top position property of the form. The change from the old value of the form s height position to the new one will take 2 seconds (duration parameter). Note the use of the bind parameter var specifying that the value must be retrieved from the role named get. <link id="lalign" xconnector="onendattributionset"> <bind role="onendattribution" component="lua" interface="newheight"/> <bind role="get" component="lua" interface="newtop"/> <bind role="set" component="form" interface="top"> <bindparam name="var" value="$get"/> <bindparam name="duration" value="3s"/> </bind> </link> Listing 15. Example of link to get a value from a property and set to another one, progressively. 9 For the Brazilian DTV System, the NCL profile only allows causal relationships.

13 5. STORING AND RETRIEVING THE PRESENTATION STATE Almost all multimedia documents, including DTV applications, are composed of a collection of media objects to be autonomously presented. The document presentation can run as many times as necessary, partially or totally. For each runtime, a new activation is done, without any historical relationship with the previous ones. However, there are many situations where the application s presentation history is important to be maintained, in order to achieve the desired results, for example: When viewers are allowed to explicitly pause a DTV application and then resume it at some later time possibly days or weeks later, and even on a different device; When a viewer changes the TV channel, thus starting another application in the new channel, but then changes his mind and returns to the previous channel, resuming the application and inheriting all information previously given, all answers previously provided, all interactions previously carried out, all environment information (global variables) previously set, etc. In the first example, the application (and all its content) must be resumed from the exact point where it was stopped, or from a preceding point. In contrast, in the second example, the application must be resumed in a future point of the TV program, with respect to the time it had been stopped. Nevertheless, in both cases, all previous information should be preserved. In NCL, the state of a multimedia presentation is maintained using a data structure called hypermedia temporal graph (HTG). This data structure can be initially created from the application specification, and represents all events that can happen during the application execution (or presentation). Besides predictable events, unpredictable events are also represented, like possible viewer interactions, decision points for content and presentation adaptation, etc. As previously defined, an NCL event is an occurrence in time that may be instantaneous or have a measurable duration. Each event defines a state machine, as shown in Figure 3, which should be maintained by the NCL formatter. HTG is composed of vertices, which represent actions (for state changes) performed on event state machines, and directed edges that represent relationships among actions, derived from the document specification in NCL. An edge is labeled by a condition that must be satisfied in order to trigger the action specified in the edge s output vertex. When applications contain only predictable events, graph edges are labeled only by temporal intervals. Figure 4 shows a hypermedia temporal graph for The First John application.

14 Figure 4 Hypermedia Temporal Graph In Figure 2, the application states that when the animation reaches a temporal anchor (a segment in the animation video) in 12s from the beginning of its presentation, a photo must be exhibited for 5 seconds. The application also states that when the animation reaches a temporal anchor (a segment in the animation video) in 45s from the beginning of its presentation, an icon must be exhibited for at most 6 seconds. During this interval, if the icon is selected, the icon presentation must finish, and a shoes advertisement (shoes advert video or image, and the HTML form) must be presented in parallel with the animation video, which must be shrunk (resized) to give place to the advertisement exhibition. The exhibition is done on a background image. At the end of the advertisement, the video must return to its normal size. From the application starting time, the graph traversal identifies every action that must be applied to media players. These actions can have their moment in time computed taking into account the time intervals required to satisfy conditions from the HTG entry point to the corresponding action vertices. This set of actions and corresponding moments in time compose a temporal chain (sequence of actions on object presentations). During an application presentation, all information gathered both from viewers and from the system, all viewer answers, all viewer interactions, and all local and global variable settings are collected, updating the temporal graph. The application s temporal chain is composed step by step the moment each unpredictable event occurs. Therefore, note that the temporal chain summarizes all the past navigation made by a viewer and, thus, represents the current multimedia presentation state, which can be stored and later retrieved, allowing resuming the application from its saved state. An NCL formatter shall be capable of resuming an application from a previously saved state point. If the application has some object with live content, the temporal graph should jump to the current time point of this object. Therefore, the two examples presented in the beginning of this section are supported. In the Ginga middleware, the temporal graph of an application is maintained in a conceptual entity called private base. As mentioned in Section 1, Ginga-NCL is composed of the NCL Formatter and the NCL Private Base Manager. The Private Base Manager is in charge of receiving NCL document editing commands and maintaining the active NCL documents. As an example, in the Brazilian DTV system, there is a private base associated with each TV channel. Ginga-NCL provides support to several private base commands, as follows: openbase (baseid, location): Opens an existing private base to be found with the location parameter. If the private base does not exist in location or the parameter attribute is not informed, a new base is created with baseid. activatebase (baseid): Turns on an opened private base. deactivatebase (baseid): Turns off an opened private base. savebase (baseid, location): Saves all private base content (the temporal graph) into a persistent storage device (if available). The location attribute shall specify the device and the path for saving the base. closebase (baseid): Closes the private base and disposes all private base content. adddocument (baseid, {uri, ior}+): Adds an NCL application to a private base. The NCL document s file can be: a) received from a datacast network as a set of pushed files; for these pushed files, each {uri,id} pair is used to relate a set of file paths in the NCL document specification with their respective locations in a transport system. The set of reference pairs shall be sufficient for mapping any file reference present in the NCL document specification to its concrete location in the receiver memory; b) received from an interactivity channel as a set of pulled files, or may be files already stored in the receiver; for these pulled files, no {uri, id} pairs have to be sent, except the {uri, null } pair

15 associated with the NCL document specification that the editing command requests to be added in baseid, if this NCL document is not received as a pushed file. removedocument (baseid, documentid): Removes an NCL document from a private base. startdocument (baseid, documentid, interfaceid, eventnpt, offset, refdocumentid, refnodeid): Starts playing an NCL document in a private base, starting the presentation from a specific document interface. The time reference provided in the eventnpt field defines the initial time positioning of the document with regards to the NPT time base (Normal Play Time) value of the refnodeid content of the refdocumentid document being received. Three cases may happen: a) If eventnpt is greater than or equal to the NPT time base value of the refnodeid content being received, the document presentation shall wait until NPT has the value specified in eventnpt to be started from its beginning time+offset. b) If eventnpt is less than the NPT time base value of the refnodeid content being received, the document shall be started immediately from its beginning time+offset+(npt eventnpt) seconds. The offset+(npt eventnpt) seconds shall be a positive time value. c) If eventnpt is equal to 0, the document shall start its presentation immediatelly from its beginning time+offset. stopdocument (baseid, documentid): Stops the presentation of an NCL document in a private base. All document events that are occurring shall be stopped. pausedocument (baseid, documentid): Pauses the presentation of an NCL document in a private base. All document events that are occurring shall be paused. resumedocument (baseid, documentid): Resumes the presentation of an NCL document in a private base. All document events that have been paused by a previous pausedocument command shall be resumed. savedocument (baseid, documented, location): Save an NCL document into a persistent storage device (the location attribute specifies the device and the path for saving). From the above NCL editing commands, note that a document in an active private base can be paused and the private base saved. At a later time, the private base can be opened and the document can be resumed from the point it has been paused, therefore meeting the previously stated requirements. NCL editing commands also allow controlling any NCL element present in an application. For each NCL entity, add and remove commands are defined. If an entity already exists, the add command has the update (altering) semantics. Among this set of NCL editing commands there are those that work on variables, that is, <property> elements: addinterface (baseid, documentid, nodeid, xmlinterface): adds an interface (in particular a <property> element) to an object of an NCL document in a private base. The xmlinterface specifies the interface to be added using the XML syntax defined by NCL. removeinterface (baseid, documentid, nodeid, interfaceid): removes an interface (in particular a <property> element) from an object of an NCL document in a private base. The interfaceid shall identify a <property> element s name attribute, in the case of removing a <property> element. setpropertyvalue (baseid, documentid, nodeid, propertyid, value): Sets the value to a property. The propertyid shall identify a <property> element s name attribute. The <property> shall belong to an object of an NCL document in a private base identified by the parameters. 6. STATE AND VARIABLE MANIPULATION BY IMPERATIVE OBJECTS Despite its declarative nature, NCL recognizes that the richer expressiveness of imperative languages cannot be ignored, mainly for applications that produce many dynamic contents, as a result of complex operations.

16 Imperative languages, in particular scripting languages, provide an expressive support for variable handling. However, they have a high cost: they usually require a programming expertise for application development; they occasionally put at risk the application portability; and the temporal graph management and the presentation control are much more difficult to handle as a rule. In addition to these problems, the use of imperative programming languages is more prone to errors made by the application s programmer. Since the loss of the execution control can break the presentation s temporal consistency, and because this is a very difficult problem to be treated in dynamic presentations, scripting languages should be limited in their expressiveness scope. They should not be allowed to change the temporal graph, except by using declarative primitives. The drawbacks of the imperative paradigm are the advantages of the declarative one that, in addition, allows for easier application transformations 10. As aforementioned in Section 1, declarative language should offer a good balance between flexibility and simplicity. It should provide a variable handling model sufficiently rich to describe a wide range of interactive applications, avoiding, as much as possible, the help of an imperative scripting language. At the same time, the declarative simplicity should not be lost, leaving for imperative objects the more complex manipulations, with the necessary care to avoid any impact in the application s temporal graph. Variables and the presentation state of an NCL application are handled according to this principle. Thus, recognizing this situation, NCL supports the use of media objects that represent chunks of code in an imperative language, which are called imperative media objects. Specifically to terrestrial DTV applications, Ginga-NCL supports imperative media objects implemented in Java (NCLet object) and Lua (NCLua object) [1], and for IPTV services at least media objects implemented in Lua [10]. In fact, Lua is the main scripting language used by NCL. Lua [6] is a powerful, fast, lightweight, embeddable scripting language. Lua combines simple procedural syntax with powerful data description constructs based on associative arrays and extensible semantics. Lua is dynamically typed, runs by interpreting bytecode for a register-based virtual machine, and has automatic memory management with incremental garbage collection, making it ideal for configuration, scripting, and rapid prototyping. Lua has been used in many industrial applications, with an emphasis on embedded systems and games. Lua is currently the leading scripting language in games [7]. The NCL/Lua integration provides a set of controlled mechanisms to support the interaction between these two languages, especially for triggering flow control events and for handling NCL variables and the document structure through imperative commands. Although the rest of this section presents details about the use of Lua in NCL documents, most of this information can also be applied to imperative media objects implemented in Java. 6.1 Properties in Imperative Media Objects Like other types of media objects, a <media> element must be used to specify an imperative object in NCL. In this case, the object s content, located through the src attribute, must be an imperative code to be executed. As aforementioned, the terrestrial DTV profiles of NCL allow the application/x-ncl-nclua type, for Lua procedural codes (.lua file extension); and the application/x-ncl-nclet type, for Java (Xlet) codes (.class or.jar file extensions). A <media> element containing imperative code can define content anchors (through <area> elements) and properties (through <property> elements), as all NCL media objects can do. Imperative code chunks may be associated with <area> elements using the label attribute. In this case, the label value must identify a chunk of code (a function, a method, etc.). A <property> element can be mapped to a code function or to a code variable. The name attribute of the <property> element must be used to identify the function or the variable in the imperative code. As usual, <area> and <property> elements can be used as interface points of <link> elements, establishing a two-way bridge between the declarative and imperative environment. 10 In the case of XML languages, XSL transformations can be applied to document trees.

17 In order to declare an NCLua media object that defines two properties (calc and newtop) associated with homonym entities in the Lua code (in this case, a function and a variable, respectively), we could specify the following: <media id="lua" type="application/x-ginga-nclua" src="../media/newheight.lua"> <property name="calc"/> <property name="newtop"/> </media> Listing 16. Imperative Lua media object specification. 6.2 Lua Code Interaction with other NCL Objects The NCL formatter controls the lifecycle of an imperative NCLua object. The formatter is responsible for triggering the execution of an imperative object and for mediating the communication between this object and other NCL objects of an application. As with all media object players (e.g., video player, audio player, image player, etc.), once instantiated, the NCLua player (the language engine) shall execute an initialization procedure. However, different from other media players, this initialization code is specified by the author of the Lua code. This initialization procedure is executed only once, for each instance, and creates all Lua code chunks and data that may be used during the execution of the imperative object and, in particular, registers at least one event handler for communication with the NCL user agent. After the initialization, the execution of the NCLua object becomes event oriented in both directions. That is, any action commanded by the NCL user agent reaches the registered event handlers, and any NCL event state change notification is sent as an event to the NCL user agent (as, for example, the natural end of a Lua code chunk execution). As an example, note in Listing 11 that the <link id="lbegingshoes" > starts the initialization procedure of the NCLua media object when the icon media object is selected. From that moment on, the NCLua object s calc function is ready to be called, as defined by Listing 16. Application authors may define NCL links to start, stop, pause, resume or abort the execution of a Lua code. A Lua player shall interface the imperative execution environment with the NCL user agent. Analogous to conventional media content players, Lua players shall control event state machines associated with the NCLua object. As an example, if the code finishes its execution, the player shall put the event presentation state machine corresponding to the code execution in the sleeping state. However, different from media content players, a Lua player does not have sufficient information to control by itself all event state machines, and shall rely on the Lua application to command these transitions. NCL links can be bound to NCLua object interfaces (content anchors and, in particular, properties). If an NCL <link> element s action starts, stops, pauses, resumes, or aborts the presentation of an anchor, callbacks in the Lua code shall be triggered. In particular, conditions defined by this <link> element can refer to variables (<property> elements) defined in other media objects. Therefore, attribution events defined by changes in property values can trigger callbacks in Lua code. On the other hand, a Lua code can also command the start, stop, pause or resume of its associated NCL content anchors through an event-driven API offered by the language. The corresponding changes in the event state machine associated with the anchor may be used as conditions of NCL links to trigger actions on other NCL objects of the same application, in particular, actions on attribution events causing changes in property values. Thus, a two-way synchronization can be established between the imperative code and the remainder of the NCL document.

18 A Lua code may also be synchronized with other NCL objects through <property> elements (NCL variables). When the <property> element is mapped to a Lua code chunk through its name attribute, a link action start applied to the property shall cause the code execution, with the set values interpreted as parameters passed to the code chunk. When a <property> element is mapped to a Lua variable, the action start shall assign the value to that variable. As usual, the event state machine associated with the property shall be controlled by the Lua player but, in some situations, commanded by the Lua application. Similar to content anchors, conditions defined by the <link> element can refer to variables (<property> elements) defined in other media objects. Therefore, attribution events defined by changes in property values can trigger callbacks in Lua code or changes in Lua object s properties. As an example, in The First John application, a NCLua object is responsible for receiving the current height position of an object and returning a new top value that centralizes the object vertically. The <link id="lbegincalc"...> element gets (role="get") the height value from the form media object and passes it (role="set") to the Lua media object calc function. <link id="lbegincalc" xconnector="onbeginset"> <bind role="onbegin" component="form"/> <bind role="get" component="form" interface="height"/> <bind role="set" component="lua" interface="calc"> <bindparam name="var" value="$get"/> </bind> </link> Listing 17. Passing parameters to a Lua function. On the other hand, a Lua code can also command the start, stop, pause or resume of its associated properties through an event driven API offered by the language. The corresponding changes in the attribution event state machine associated with the property may be used as conditions or assessments of NCL links to trigger actions on other NCL objects of the same application, in particular, actions on attribution events causing changes in property values. Thus, again, a two-way synchronization can be established between the imperative code and the remainder of the NCL document. As an example, in The First John application, the <link id="lalign"...> element specifies that when the attribution of the newtop property made by the Lua media object ends (onend role), then the form media object s top property is set (set role) to the value got from the Lua media object s newtop property. <link id="lalign" xconnector="onendattributionset"> <bind role="onendattribution" component="lua" interface="newtop"/> <bind role="get" component="lua" interface="newtop"/> <bind role="set" component="form" interface="top"> <bindparam name="var" value="$get"/> <bindparam name="duration" value="3s"/> </bind> </link> Listing 18. NCLua media object setting values to another object s variable. In addition to interactions through <area> and <property> elements, the imperative language used with NCL shall offer an API that allows an imperative code to query any pre-defined or dynamic properties of the NCL settings media object (see Section 3). However, it must be stressed that it is not allowed to directly set values to these properties. Properties of the settings media object may only be changed by NCL links, in order to allow an easier consistency control of the application flow by the NCL user agent.

19 6.3 NCL Editing Command API An imperative-code player should also offer an API that provides a set of methods to manipulate the NCL document structure (editing commands) and the NCL s private base (see Section 5). In Lua, this API is provided by the event Lua module [1] [10] in its edit class. The editing commands allow any NCL element to be added to or removed from an NCL document presentation. The commands to manipulate the private base are equivalent to those presented in Section 5. It is important to note that the facility provided by editing commands is not equivalent to the direct manipulation of the DOM representation of an NCL document. These commands represent more abstract operations, which are defined based on the abstractions provided by the NCL document model. The NCL formatter is responsible for performing all these commands and for guaranteeing the overall consistency of the document presentation after each command execution, avoiding undesired side effects. 7. FINAL REMARKS The goal of NCL profiles has been to define a set of functionalities rich enough to allow the development of a wealth of time-dependent applications without resorting to imperative procedures. However, NCL recognizes that an imperative programming support may be required, especially for those applications that demand intensive generation of dynamic content, as result of a complex computation. Thus NCL also provides support for objects containing imperative code, but under control, in order to avoid inconsistencies in the application temporal flow. With the exception of the presentation state control, all ideas presented in the paper are employed in the reference implementation of the Ginga middleware of the Brazilian digital TV system an open source implementation that can be obtained from They are also implemented in all commercial implementations of Ginga. A prototype of an NCL user agent including the presentation state control is now in testing, and will also be available as an open source implementation. However, further work has yet to be done involving multiple devices in resuming a stored application [14]. The complete The First John application example used in this paper can be obtained from Several applications have been developed in order to test the usefulness and the expressiveness of the proposal. The result has been quite satisfactory. Indeed, most DTV applications have been making use only of NCL s declarative primitives and, for the more complex applications, the support offered by NCLua imperative objects has been more than sufficient. Finally, authoring support needs to be defined for hiding the syntax of variable definitions and manipulations for non-expert developers. A graphical support is in its initial stage of development aiming at being incorporated to the NCL Composer [5] tool, also offered as open source software. 8. ACKNOWLEDGMENTS The authors would like to thank Carlos Salles, Romualdo Costa, Marcio Moreno, Marcelo Moreno and Francisco Sant anna who provided a thoughtful discussion of this work, tracked down and fixed problems in the initial reference implementation of Ginga. The authors also thank CNPq, FINEP and FAPERJ for their support. 9. REFERENCES [1] ABNT NBR Associação Brasileira de Normas Técnicas (2007) Digital Terrestrial Television Standard 06: Data Codification and Transmission Specifications for Digital Broadcasting, Part 2 GINGA-NCL: XML Application Language for Application Coding (São Paulo, SP, Brazil, November, 2007). DOI: [2] ARIB Association of Radio Industries and Business (2004). ARIB Standard B-24 Data Coding and Transmission Specifications for Digital Broadcasting, version 4.0, 2004.

20 [3] ECMA International - European Association for Standardizing Information and Communication Systems (1999). ECMA 262 ECMAScript Language Specification. 3rd Edition. DOI: standards/ecma-262.htm. [4] ETSI European Telecommunication Standards Institute (2006). ETSI TS V1.2.2 Digital Video Broadcasting Digital Video Broadcasting (DVB); Multimedia Home Platform (MHP) Specification [5] Guimarães, R.L; Costa, R.R.; Soares, L.F.G. (2008) Composer: Authoring Tool for itv Programs. In Proceedings of European Interactive TV Conference (Salzburg, Austria, July 2008). EuroiTV DOI: [6] Ierusalimschy R., Figueiredo L.H., Celes W. (2006) Lua 5.1 Reference Manual, ISBN [7] Ierusalimschy, R., de Figueiredo, L. H., and Celes, W. (2007) The evolution of Lua. In Proceedings of the Third ACM SIGPLAN Conference on History of Programming Languages (San Diego, California, June 09-10, 2007). HOPL III. ACM, New York, NY, DOI: [8] ISO/IEC International Organization for Standardization (2001) Coding of Audio-Visual Objects Part 1: Systems. 2nd Edition. [9] ISO/IEC International Organization for Standardization (2006) Lightweight Application Scene Representation (LASeR) and Simple Aggregation Format (SAF). [10] ITU-T Recommendation H.761 (2009) Nested Context Language (NCL) and Ginga-NCL for IPTV Services. Geneva, January, [11] Jansen, J.; Bulterman, D. (2009) SMIL State: An Architecture and Implementation for Adaptive Time- Based Web. In Multimedia Tools and Applications. Vol. 43, N. 3 (Netherland, April 2009). ISSN: [12] King P., Schmitz P., Thompson S. (2004) Behavioral reactivity and real time programming in XML: functional programming meets SMIL animation. In Proceedings of ACM Document Engineering (Milwaukee, Wisconsin, USA, October, 2004). DOI: [13] Soares L.F.G., Rodrigues R.F. (2006) Nested Context Language 3.0 Part 8 NCL Digital TV Profiles. Technical Report. Departamento de Informática da PUC-Rio, MCC 35/06. DOI: [14] Soares L.F.G. (2009) Nested Context Language 3.0 Part 12 Support to Multiple Exhibition Devices. Technical Report. Departamento de Informática da PUC-Rio, MCC 03/09. DOI: [15] Soares L.F.G., Rodrigues R.F, Cerqueira R., Barbosa S.D.J (2009) Variable Handling in Time-Based XML Declarative Languages. In Proceedings of the 24th Annual ACM Symposium on Applied Computing. (Honolulu, Hawaii. Março de 2009). [16] Thompson S., King, P., Cameron, H. (2005) Modelling Reactive Multimedia Design and Authoring. In Multimedia Tools and Applications, Vol. 27, N. 1 (Netherland, Septembe 2009). ISSN: [17] W3C World-Wide Web Consortium (1998) Cascading Style Sheet, level 2 CSS. W3C Recommendation. [18] W3C World-Wide Web Consortium (2004) Document Object Model DOM Level 3 Specification. W3C Recommendation. [19] W3C World-Wide Web Consortium (1998). Timed Interactive Multimedia Extensions for HTML. W3C Submission. DOI: [20] W3C World-Wide Web Consortium (2005) Synchronized Multimedia Integration Language SMIL 2.1 Specification, W3C Recommendation. DOI: