icore Internet Connected Objects for Reconfigurable Ecosystem

Transcription

1 icore Internet Connected Objects for Reconfigurable Ecosystem Grant Agreement Nº D6.2 Final proof of concept prototype for Smart Home: Living Assistant Version: Due Date: Delivery Date: Nature: Dissemination Level: Lead partner: Authors: /03/ /05/2014 Report Public UPRC See contributors table Internal reviewers: VTT, TNO The research leading to these results has received funding from the European Community's Seventh Framework Programme [FP7/ ] under grant agreement n

2 Deliverable Title Final proof of concept prototype for Smart Home Deliverable Number 6.2 Keywords: Smart Home Proof of Concept, Cognitive Mechanisms Specification, Communication Interfaces, Demonstration Scenarios. Executive Summary: This deliverable describes the final prototype of the proof of concept for the Smart Home use case. The document starts from an overview of the various components and hardware used describing their functionalities, their implementation details as well as their added value to icore. Then, the use case scenarios are described and the interactions between various components during each scenario is analysed. Finally, performance results for the icore system are presented and the implemented interfaces between components are specified. Date: 09/05/2014 Grant Agreement number: Page 2 of 63

3 Contributors First Name Last Name Affiliation Vera Stavroulaki UPRC Panagiotis Vlacheas UPRC Dimitris Kelaidonis UPRC Vassilis Foteinos UPRC Antonis Moustakos UPRC Panagiotis Demestichas UPRC Andrey Somov CNET Erik Mademann ZIGPOS Danny Schumann ZIGPOS Date: 09/05/2014 Grant Agreement number: Page 3 of 63

4 Table of Contents 1 List of Acronyms and Abbreviations Introduction Proof-of -concept prototype environment Overview Components Service Requestor GUI Natural Language Processing (NLP) User Preferences and Profiling Service Request Analysis Service Template Repository Real World Knowledge (RWK) System Knowledge CVO Management Unit: Approximation and Reuse Opportunity Detection (AROD) CVO Registry CVO Management Unit: CVO Composition Engine (CCE) CVO Management Unit: CVO Dynamic Workflow (Planner) CVO Management Unit: CVO Coordination VO Registry VO Template Repository VO Container VO Factory (VO Creation Tool) VO Front-End Software and Back-End REST Modules VO Back-End Software REST Module for wireless sensor networks Correlation Matrix Security Agent Smart Home CVO Smart Car CVO Smart Traffic Light CVO Smart Lighting CVO (icore BUTLER Integration) Hardware Arduino Sensors Actuators WaspMote Flyport Raspberry Pi Date: 09/05/2014 Grant Agreement number: Page 4 of 63

5 Android Tablet Android Phone Demonstration scenarios Dynamic CVO Creation Scenario description Interactions between components Knowledge-based Instantiation Scenario description Interactions between components CVO Self-Healing Scenario description Interactions between components CVO Coordination Scenario description Interactions between components icore BUTLER Integration Scenario Description Interaction between components Performance Results Prototype Implementation Service level mechanisms performance CVO and VO level mechanisms performance Specification of interfaces Interface between Service Requestor GUI and Natural Language Processing Interface between Natural Language Processing (NLP) component and Service Request Analysis (SRA)component Interface between Service Request Analysis (SRA) component and User Profiling component Interface between Service Request Analysis (SRA) component and CVO Management Unit component Interface between CVO and Service Request Analysis (SRA) component Interface between AROD and CVO Registry Interface between AROD and VO Registry Interface between VO Registry and Security Agent (SA) Interface between Approximation and Reuse Opportunity Detection (AROD ) component and CVO Composition Engine (CCE) component Interface between CVO Composition Engine (CCE) component and Correlation Matrix (CM) component Interface between CVO Software Factory and CVO Template Repository Date: 09/05/2014 Grant Agreement number: Page 5 of 63

6 5.12 Interface between CVO Composition Engine and Planner Conclusion References Date: 09/05/2014 Grant Agreement number: Page 6 of 63

7 List of Figures Figure 1: Step 1 of the request process Figure 2: Step 2 of the request process Figure 3: NLP - SRA Process diagram Figure 4: Generic RWK Model Figure 5: Smart Home RWK Model Figure 6: Generic SK Model Figure 7: Smart Home SK Model Figure 8: CVO Registry GUI - an indicative instance Figure 9: CVO Composition Engine Figure 10: CVO Dynamic Workflow Figure 11: CVO Coordination GUI Figure 12: VO Registry GUI Figure 13: VO Creation Tool and associated processes Figure 14: VO FE & VO BE Figure 15: CVO Desktop Interface Figure 16: Smart Car CVO Figure 17: Smart Traffic Light CVO Figure 18: Smart Lighting CVO Figure 19: Arduino MEGA with Ethernet Shield Figure 20: Motion Sensor, Light Sensor, LED Light Figure 21: Power Socket Figure 22: Accerelometer, eertls device Figure 23: Temperature/Humidity sensor Figure 24: Wireless Lamp Figure 25: Light Sensor Figure 26: WaspMote platform with sensors Figure 27: FlyportIoT platform Figure 28: Raspberry Pi without case Figure 29: Software architecture combining WSN to icore system Figure 30: Android Tablet Figure 31: Android Phone Figure 32: MSC for Dynamic CVO creation Date: 09/05/2014 Grant Agreement number: Page 7 of 63

8 Figure 33: CVO Coordination Process Figure 34: Integration architecture Figure 35: Service Level execution time vs. Number of Available Service Templates Figure 36: Approximation & Reuse Opportunity Detection execution time vs. Number of Available VOs Figure 37: CVO Composition Engine execution time vs. Number of Available VOs Figure 38: VO Discovery Process execution time vs. Number of Available VOs Figure 39: Overall CVO Creation/Instantiation execution time vs. Number of Available VOs Figure 40: Received Bytes vs. Number of Available VOs Date: 09/05/2014 Grant Agreement number: Page 8 of 63

9 List of Tables Table 1: Generic RWK Model - description of concepts Table 2: Generic SK Model - description of concepts Date: 09/05/2014 Grant Agreement number: Page 9 of 63

10 1 List of Acronyms and Abbreviations Acronym Definition AJAX Asynchronous JavaScript and XML API Application Programming Interface AROD Approximation and Reuse Opportunity Detector BN Bayesian Network CCE CVO Composition Engine CM Correlation Matrix CPT Conditional Probability Table curl Client URL Request Library CVO Composite Virtual Object E2E End to End HTML5 HyperText Markup Language 5 HTTP Hypertext Transfer Protocol ICT Information and Communications Technology JSON JavaScript Object Notation MAP Maximum a Posteriori NLP Natural Language Processing PHP Hypertext Preprocessor POS Part-of-Speech RDF Resource Description Framework RWK Real World Knowledge SA Security Agent SK System Knowledge SoC System on Chip SR GUI Service Requestor Graphical User Interface SRA Service Request Analysis URI Uniform resource identifier VO Virtual Object VO BE Virtual Object Back End VO FE Virtual object Front End XML Extensible Markup Language Date: 09/05/2014 Grant Agreement number: Page 10 of 63

11 2 Introduction This document describes the Final Proof of Concept for the icore Smart Home: Living Assistant Use Case. The Final Proof of Concept unifies the functionalities already implemented in the first Smart Home prototype while further extending the application of the icore architecture and concepts. Listed below are the icore architecture aspects that have been extended: Service Requestor GUI: the users of the Smart Home Use Case have access to the service through this simple, web-based, interface. User Preferences and Profiling: this component provides smart, personalized services to the users by means of saving certain information about the users (e.g. the services the user has requested and feedback provided for them) and using them to parameterise the service. Real World Knowledge and System Knowledge: In the context of the Smart Home PoC these components support the Dynamic Composite Virtual Object (CVO) Creation and Knowledge based Instantiation by providing past as well as current information (e.g. past CVOs). VO Registry: this entity is used to hold and manage information regarding the Virtual Objects of the icore System. Various components in the Smart Home Use Case use this registry to retrieve as well as change information about Virtual Objects (VOs). VO Factory: this easy to use tool has been implemented to facilitate the creation of Virtual Objects. An end user as well as a domain expert can use this web-based tool to easily create and register VOs. CVO Management Unit: CVO Coordination: this component is responsible for resolving conflicts that arise when two or more icore entities both require access to a CVO and it can t be granted at the same time. The Prototype Architecture, Validation and Demonstration are described in detail in the following chapters. Date: 09/05/2014 Grant Agreement number: Page 11 of 63

12 3 Proof-of -concept prototype environment 3.1 Overview The Smart Home environment aims to improve the quality of life for the disabled and the elderly as well as provide monitoring tools for both family members and healthcare professionals. Additional objectives of this proof of concept include: (a) easy integration of care taking, (b) remote health care, (c) home automation and (d) online medicine purchasing services. Additionally it creates possibilities of a new business eco-system among the elderly, the impaired, patients, family members, care takers, doctors, nurses and pharmacy stakeholders. 3.2 Components Service Requestor GUI This component is intended for the end-user/stakeholder. It has been created with ease of use in mind and its purpose is to receive a request in natural language and then, through interactions with other icore components, create the Composite Virtual Object (CVO). It has been implemented using a number of technologies; namely: (a) HTML 5, (b) PHP, (c) AJAX, (d) Javascript, (e) MYSQL, (f) Servlets (Java Web Services) as well as a (g) RESTful interface to facilitate the connection between the Servlets and the icore infrastructure. Firstly, the service requestor selects the area that the CVO will cover and clicks the Fetch Functions button (this is done in order to have the available Virtual Objects in the area). Then he selects the type of device (Desktop, Mobile or Tablet) he wants the CVO instantiated on and inserts a Service Request written in natural language as illustrated in Figure 1. He can also select any policies he wants (e.g. High performance, Low expenditure). Afterwards, the Service Execution Request is created for the service requestor and the selected Virtual Objects (VOs) are displayed for confirmation (Figure 2). The Service Requestor GUI provides an icore user with a simple to use interface with the icore System thus allowing him to easily request a service. Date: 09/05/2014 Grant Agreement number: Page 12 of 63

13 Figure 1: Step 1 of the request process Natural Language Processing (NLP) Figure 2: Step 2 of the request process The NLP component is used to perform lexicological analysis on requests expressed in natural language. For this purpose, the WordNet relational dictionary ( is being used. It executes Sentence Detection, Sentence Tokenization, Part-of-speech (POS) tagging tasks on service requests. This component outputs the words of the user request, the POS tags of Date: 09/05/2014 Grant Agreement number: Page 13 of 63

14 the selected words and the syntactical dependencies between the selected words. This output serves as input to the Service Request Analysis component. It therefore contributes to the automated analysis of the user requests by clarifying the useful information in them adding to the overall ease of use of the icore System User Preferences and Profiling This component is used to obtain user preference data as well as manage and represent this information (Indicative information includes: (a) the set of available service templates, (b) the set of services that have been requested/are being used and (c) an indication of the user preferences associated with a particular service and service template). In particular, it holds the Conditional Probability Tables (CPTs) of the icore users. These CPTs are part of the Bayesian Networks (BNs) and they keep the probabilities for the variables of the BNs. As mentioned in the icore deliverable document 5.2 [2], a Bayesian Network (BN) is a compact expressive representation of uncertain relationships among variables of interest in a domain. A BN is a directed acyclic graph where nodes represent random variables. A lot of works on e.g. user modelling includes the use of BNs, as a method for evaluating, in a qualitative and quantitative manner, elements of the user behaviour and consequently updating the user profile. Additionally, this component is responsible for updating the CPTs after the service execution by exploiting the user feedback. This entity contributes to automated analysis and matching of user requirements for the dynamic creation and configuration of CVOs with the aim of providing smart, personalized services Service Request Analysis This component is responsible for the selection of the optimal Service Template. This process is fulfilled either by exploiting the Maximum a Posteriori (MAP) inference algorithm and the user preferences of the Bayesian Networks or by selecting the Service Template with the greatest semantic similarity estimation. The first case is used when there is previous knowledge about the user request into the Bayesian Networks. Otherwise, the second case is used. The input to SRA has the format given by NLP and it consists of: (a) service request, (b) location, (c) user role and (d) user preferences. The user preferences are extracted via the User Preferences and Profiling component. The outcome of this process is a so called Service Execution Request that is used by Approximation and Reuse Opportunity Detector (AROD). This entity contributes to automated analysis and matching of user requirements for the dynamic creation and configuration of CVOs with the aim of providing smart, personalized services. Date: 09/05/2014 Grant Agreement number: Page 14 of 63

15 Service Template Repository Figure 3: NLP - SRA Process diagram In this repository a collection of Service Templates, created by domain experts, is stored. This allows for the Dynamic CVO creation of a CVO. A Service Template defines at a high-level the operations, dependencies, and intelligence needed for the automated deployment of a particular service/application. This includes the (generic) types of CVOs and VOs that will be required for realizing these operations and intelligence ([3]) Real World Knowledge (RWK) The RWK includes particular instantiations of a generic RWK model, which is used as the basis for the development of the RWK in a specific domain. The generic RWK model is depicted in Figure 4, while the Table 1 presents the description of the particular nodes that are included in the model. In general the model is comprised by different nodes that are considered as meta-data containers that include particular meta-data properties. Through the use of the meta-data properties, as well as the corresponding concepts, it is possible to structure a specific RWK model, which will be oriented to the Smart Home PoC. Consequently, by following the above approach, the RWK Model that is associated with the Smart Home PoC is depicted in Figure 5. The current model includes particular instantiations of the concepts that are included in the Smart Home PoC prototype, presenting them as instantiations of the Generic RWK Model concepts. Date: 09/05/2014 Grant Agreement number: Page 15 of 63

16 RWK Root Node hasuri :URI indicates Domain hasuri :URI hasname :String hastype :URI hasdescription :String hasdomainservice hasdomainparameter hasrwconcept Domain Parameter hasuri :URI hasname :String hasfeaturetype :URI hasfeaturevalue :Literal RW Concept hasuri :URI hasname :String hastype :URI Domain Service hasuri :URI hasname :String hasdesciption :String instantiates :URI hasrwfeature RW Feature hasuri :URI hasname :String hasfeaturetype :URI hasfeaturevalue :Literal Figure 4: Generic RWK Model RWK Model Node Description RWK Root Node Domain Domain Parameter RW Concept The RWK Root Node presents the root node of the icore RWK Integrated Model that points to a domain specific RWK Model, such the Smart Home or the Smart Meeting RWK Model. The Domain node presents the domain, on which the RWK model is referred and describes its concepts. Each domain can be described in terms of its type (e.g.: Smart Home, Smart meeting, etc), a name and a textual description in natural language. Each domain may include one or more parameters, the Domain Parameters, which may describe domain specific concepts required for specific situations, such Date & Time, Indoor Location, Status of a process and/or situation, etc. The RW Concept node can be used so as to present the potential Real World concepts that can be involved into a specific domain. In a general approach, it could be considered as the node that can describe the non-icts and Date: 09/05/2014 Grant Agreement number: Page 16 of 63

17 (potentially) the ICT objects, which are included in a specific domain (e.g. Smart Home:[Sensors & Actuators, Persons, Places, etc], Smart Meeting:[Places, Mobile Devices, QR Codes, etc]). RW Feature Domain Service The RW Feature node works as assistant node to the RW Concept node and it is considered as part of the RWK Model, in order to support the capability for further description of a RW concept, where it is required. The Domain Service node presents a service as it is considered in icore. Actually, it can describe either a domain specific or a generic (multiapplied/re-usable) service, which is developed under a specific concept so as to serve specific purposes. For instance, in the Smart Home PoC, a domain specific service refers to the Smart Health Monitoring, which actually can be re-used outside of the initial concept. The domain service it is described in terms of a URI as its unique identifier, a name and textual description in natural language, while it includes a link, as a URI, to the Service Template that has been used for its instantiation. Table 1: Generic RWK Model - description of concepts Date: 09/05/2014 Grant Agreement number: Page 17 of 63

18 <<enumeration>> Outdoor Place <<RW Concept>>+Porch <<enumeration>> Smart Home Domain Parameter <<DomainParameter>>+AreaOfInterest <<DomainParameter>>+Home GeoLocation consistsof 1..* <<RW Concept>> Home consistsof 1...* <<enumeration>> Room hasrwconcept <<RW Concept>>+LivingRoom <<RW Concept>>+BedRoom <<RW Concept>>+BathRoom <<RW Concept>>+Kitchen hasdomainparameter hasrwconcept <<enumeration>> Actuator <Smart Home> +hasuri: URI +hasname: String +hastype: URI +hasdescription: String <<RWConcept>>+Heating System <<RWConcept>>+Airconditioning System <<RWConcept>>+Light <<RWConcept>>+Buzzer - Fire Alarm <<RWConcept>>+DoorLock hasrwfeature <<RW Feature>> Control Feature +hasuri: URI +hasname: String +hasfeaturetype: URI +hasfeaturevalue: Literal hasrwconcept <<enumeration>> Sensor hasdomainservice <<RWConcept>>+Temperature Sensor <<RWConcept>>+Humidity Sensor <<RWConcept>>+Luminosity Sensor <<RWConcept>>+Accelerometer Sensor <<RWConcept>>+Move Detection Sensor <<RWConcept>>+Camera <<RWConcept>>+Body Pulse <<RWConcept>>+Body Temperature hasrwfeature <<RW Feature>> Sensing Feature +hasuri: URI +hasname: String +hasfeaturetype: URI +hasfeaturevalue: Literal hasdomainservice hasrwconcept <<RW Concept>> Person +personid: URI +personname: String hasrwfeature <<enumeration>> Person Feature <<Domain Service>> Smart Environmental Monitoring +hasuri: URI +hasname: Stirng +hasdescription: String +instantiates: URI +MonitorEnvironmentalConditions() +AdjustEnvironmentalConditions() <<Domain Service>> Smart Health Monitoring +hasuri: URI +hasname: String monitors hasdescription: String +instantiates: URI +age :Integer +male :String +role :String[ Elderly, Medical proffesional, Family Member ] +userpreferences :Literal uses 1...* +MonitorHealthStatus() +NotificationsOnHealthStatus() +RaiseAlarmInCaseOfEmergency() uses 1...* uses 1...* monitors 1...* uses 1...* Figure 5: Smart Home RWK Model Date: 09/05/2014 Grant Agreement number: Page 18 of 63

19 System Knowledge In the Smart Home PoC, the System Knowledge (SK) is used for the dynamic selection of the most appropriate CVO Instance(s) that can fulfil a service request, by the corresponding CVO Management Unit functions. The SK is comprised by an information set that is associated with the CVO instance and presents various features. Further to that, a generic System Knowledge model has been developed and depicted in Figure 6, while the description of its particular nodes is provided in the Table 2. Similarly with RWK Model, the SK model is used so as to define a specific instantiation of the System Knowledge in the context of the Smart Home PoC prototype. SK Root Node hasuri :URI indicates System Process hasuri :URI hasname :String hasdescription :String includes System Concept hasuri :URI hasname :String hasdescription :String consumes hasconceptfeature System Resource hasuri :URI hasname :String hasfeaturetype :URI hasfeaturevalue :Literal System Concept Feature hasuri :URI hasname :String hasfeaturetype :URI hasfeaturevalue :Literal Figure 6: Generic SK Model SK Model Node Description SK Root Node The SK Root Node presents the root node of the icore SK Integrated Model that points to a domain specific SK Model, such the Smart Home or the Smart Meeting SK Model. System Process The Service Node node presents a service, which is executing in the system by consuming system capabilities. In particular, this service includes and is realized by a set of different system component that can be classified either as VOs or CVOs. System Concept The System Concept node can represent a wide variety of different system components that actually in our system, namely in the icore platform could be summarized by their classification into VOs and CVOs. Since there will be an association with specific VOs and CVOs it will be possible to include system Date: 09/05/2014 Grant Agreement number: Page 19 of 63

20 parameters that are associated with the (C)VOs. System Concept Feature The System Concept Feature node is capable to present any specific feature / characteristic that is associated with a particular System concept. For instance, in case we have as a System Concept a VO, then the System Concept Feature node can be used so as to represent a set of different VO Parameters, etc. System Resource The node System Resource presents any system resource in terms of CPU, RAM, etc that is consumed during the execution of a System Concept. For instance, whether a CVO/VO is executed, it uses a set of different resources on the system that hosts it. Consequently, through this node it is possible to present such concepts. Table 2: Generic SK Model - description of concepts Moreover, Figure 7 presents a specific instance of the SK model for the Smart Home PoC that is implemented based on the generic SK model. As it can be observed, the particular SK model includes concepts such the CVO Parameters, the System Resources that are used, the System Concepts that are involved in the CVO, etc. Date: 09/05/2014 Grant Agreement number: Page 20 of 63

21 <<System Process>> Smart Home Process +includes 1..* System Concept 1..* +consumes <<System Concept>> CVO Smart Home System Resource +hasconcepttfeature CompositeVO 1..* +hasuri: URI +hasstatus: String <<System Resource>> CPU <<System Resource>> RAM System Concept Feature +CVOOperation_1() +CVOOperation_2() +CVOOperation_n() +iscomprisedby +hassituationparameter +hasrequestparameter 1..* <<enumeration>> Situation Parameter <<CVOSituationParameter>>+Area <<CVOSituationParameter>>+Time <<CVOSituationParameter>>+Available VOs implements <<enumeration>> Request Parameter <<CVORequestParameter>>+Policies <<CVORequestParameter>>+VO Functions implements 1..* CVOSituationParameter +hasuri: URI +hasname: String +hasfeaturetype: URI +hasfeaturevalue: Literal 1..* CVORequestParameter +hasuri: URI +hasname: String +hasfeaturetype: URI +hasfeaturevalue: Literal VitrualObject +hasuri: URI +hasstatus: String +hastype: URI implements <<System Concept>> VO Heating System <<System Concept>> VO Luminosity Sensor implements <<System Concept>> VO Camera implements implements <<System Concept>> VO Airconditioning System <<System Concept>> VO Temperature - Humidity DHT11 Sensor implements <<System Concept>> VO Accelerometer Sensor implements implements <<System Concept>> VO Light <<System Concept>> VO Body Temperature Sensor implements <<System Concept>> VO Move Detection Sensor implements <<System Concept>> VO Body Pulse Sensor implements Figure 7: Smart Home SK Model Date: 09/05/2014 Grant Agreement number: Page 21 of 63

22 CVO Management Unit: Approximation and Reuse Opportunity Detection (AROD) This component comprises the following tasks: (a) exploiting the information stored in the CVO Registry, (b) identifying past service requests that match closely enough the present incoming ones and the situations under which they were issued and (c) matching requested VO functions to approximate ones (in case the requested ones are not available) whenever possible. In this manner, the CVOs can be re-used and the task of CVO composition from scratch can be avoided under certain circumstances ([3]). As per the identification and comparison of past and present situations and requests a set of parameters has been defined: The situation parameters which consist of the time of day the request occurred, the area of interest (location), and the available VOs at that time The request parameters that consist of the set of functions requested and corresponding related features to be maximized or minimized Additionally, AROD will attempt, in the case the requested VO function(s) does not exist, find an approximate match that can, even partially, satisfy the request. To achieve this, a correlation matrix is used. The use of the AROD component results in faster response times, better overall performance, higher reusability, smaller resource usage as well as higher stability CVO Registry The CVO Registry component has been implemented by using openrdf Sesame API and Apache Jena API. It is implemented as an RDF Memory based Graph database, while it is complemented by a customized Java API (CVO Registry API), which supports the communication capabilities and the performance of actions against the CVO Registry. The Sesame API has been used for the development of the Graph Database as a Semantic Repository, whole the Apache Jena API for the development of the CVO Registry Endpoint that allows the performance of SPARQL Requests on RDF data. In the Smart Home PoC, the CVO Registry is used in order to store the created CVOs as well as to enable the direct instantiations of existing CVOs. The description of CVOs is supported by the CVO Model that includes the CVO Semantics that is stored in the CVO Registry as RDF data. Figure 8: CVO Registry GUI - an indicative instance CVO Management Unit: CVO Composition Engine (CCE) This component is responsible for evaluating the virtual counterparts of real world objects, i.e. VOs and finding the most suitable composition of VOs that fulfils the service requirements (requested VO types, policies). Exploiting a heuristic algorithm, it achieves optimal compositions of VOs in minimum time. In addition, it allows the approximation of functions of VOs, in order to guarantee service provisioning. The CCE mechanism exploits a custom heuristic algorithm and is developed as a RESTful web service. Data is XML-formatted/encoded. Date: 09/05/2014 Grant Agreement number: Page 22 of 63

23 Figure 9: CVO Composition Engine CVO Management Unit: CVO Dynamic Workflow (Planner) This component is responsible for deciding the logical connection of the selected VOs. In particular, it provides a graph that describes how input/outputs of VOs are linked, in order to finally achieve a specific goal. Exploiting a well-known algorithm, namely Graphplan (open-source JAVA implementation), it achieves maximum performance in minimum time, enabling the dynamic composition of multiple, diverse VOs into one CVO. The Planner is developed as a RESTful web service. Data is XML-formatted/encoded. Figure 10: CVO Dynamic Workflow Date: 09/05/2014 Grant Agreement number: Page 23 of 63

24 CVO Management Unit: CVO Coordination The coordination component aims to support the coordination and E2E optimization process of the available CVOs in the CVO Level. In particular, the CVO Coordination process enables the dynamic management of the simultaneous access to and use of available CVOs from other entities, such as other CVOs. Specifically, the CVO Coordination component observes the use of available running CVOs by other entities and based on specific semantic priority indicators combined with the current situation, decides on the sequence for the use of the CVO from each extern entity. Consequently, when more than one entity is simultaneously using and/or accessing the same CVO, this is monitored by the CVO Coordination entity, which collects the available information with respect to the current situation, as well as the semantic priority indicators for each entity, aiming to derive a priority list for the CVO use. In this way the best possible management of the concurrently running, inter-related CVOs is achieved, in terms of the avoidance of potential conflicts arising from their simultaneous access and use by other entities. In general the specifications of the CVO Coordination are described below. Breaking the process in different phases, in the first phase we have the detection of a conflict caused by more than one incoming requests for access and use to a particular CVO. In the second phase the Coordination component, collects information about the CVOs that try to use the monitored CVO and notifies the CVO that is under coordination. The information includes the identifiers of the CVOs (CVO IDs) that performed the conflicted requests. Having the CVO IDs, in the third phase, the Coordination interacts with the CVO Registry so as to acquire the semantic priority indicators. Since the indicators have been acquired, the Coordination component performs data computation so as to calculate a priority rank for each CVO. Finally, a specific priority list is constructed and sent back to the CVO, phases 4 and 5 respectively, through a Conflict Resolution Response, so as to manipulate fairly the access of the external CVOs. An example in the Smart Home PoC, constitutes the demonstration presented regarding the smart city domain where there are two different CVOs, the Smart Home CVO and the Smart Car CVO that try to access and use simultaneously the Smart Traffic Light CVO. The CVO Coordination component that is running in the background, detects the access requests conflict and it takes over to resolve the conflict, by giving priority to the Smart Home CVO. An indicative example regarding the use of the CVO Coordination in the Smart Home PoC for example (Figure 11) when movement is detected a command is sent to the Smart Traffic Light CVO that a pedestrian needs to cross the road. The Traffic Light CVO however is also getting a command that a car wants to cross. Since it cannot resolve this conflict the data is sent to the CVO Coordination which is responsible for making the decision. This component aims to create a priority list for the CVOs to use whenever there is a conflict between them. To achieve this, semantic priority indicators are used in combination with the current situation. It aims the best possible management of concurrently running, inter-related CVOs, in terms of the avoidance of potential conflicts arising from their simultaneous access and use by other entities. The latter constitutes the real added value of this component, since it comes as a solution for resource management in the IoT, and consequently leading to the improvement of IoT systems performance. Date: 09/05/2014 Grant Agreement number: Page 24 of 63

25 VO Registry Figure 11: CVO Coordination GUI The VO registry includes information about VOs that are available in the system. Various entities that are available in the framework can interact with the VO registry so as to perform semantic-based discovery and selection of VOs, as well as to retrieve and/or modify the corresponding information. The innovation of this component corresponds to the ability of the integration of existing ontologies in an abstract way, giving the ability to develop generic descriptions for the VOs, without the need to link with specific predefined ontologies. The VO Registry is implemented as a Semantic Repository that stores structured data with its metadata in a form of Graph Data Models. For the development of the Graph Models, the RDF specifications are used, while the instantiated models are stored into RDF Databases (DBs), which constitute one of the sorts of the Semantic Repositories. For the implementation of the RDF DBs the openrdf Sesame Java API is used that provides both a Web Server to host the RDF DBs, as well as a well defined API to support the management of the repositories and their stored data. Further to that, the VO Registry is accessible over HTTP and it is hosted on the Sesame Web Server. Each RDF DB has its own SPARQL endpoint through which the end users (either human or software agents) can access VO data. A SPARQL endpoint is accessible over the internet by using a specific URI. Additionally, a curl API that supports the interfacing between VO Registry and third party entities, has been developed, allowing requests and responses in XML and JSON formats. Alternatively, a VO registry can be implemented using the commercial Xively service ( This option is explained in details in icore d6.1 in Sections and Xively, however, is not as powerful and multifunctional a registry as the icore one. It is worth noting that sensors used on hardware platforms generate raw data, i.e. measurement values without any context information. Data transformation chain raw data RDF document is described in details in icore D6.1 in Section The innovation of this component corresponds to the ability of the integration of existing ontologies in an abstract way, giving the ability to develop generic descriptions for the VOs, without the need to link with specific predefined ontologies. Date: 09/05/2014 Grant Agreement number: Page 25 of 63

26 VO Template Repository Figure 12: VO Registry GUI This entity holds a collection of VO Templates. A VO Template can be used to create a particular VO and provides a generic structure for the semi-automatic implementation of various VO types. In the Smart Home PoC it has been used the GitHub API to support the development of the Software repository that will store the VO Template Software packages. In this case, the end-user, having the URI of a specific VO Template can send a request to access the repository and download the corresponding software packages. Since VO Software module is developed, by using the VO Template, it can be deployed to the VO container so as to start its execution. The required methods for the interaction between external entities and the repository will be included in the VO API VO Container This component is the middle-man for connections to the Arduino/WaspMote/Flyport platform. It is the execution environment in which the VOs run and it s designed for ease of use and it can allow a CVO to receive data from multiple Arduino platforms using a single IP. The VOs can be deployed in the VO container through a user friendly GUI that allows the user to import and upload the VO in the cloud, so as to be accessible remotely, under a specific URI. The VO Container component supports the dynamic deployment of java-based software directly in the cloud and its wrap-up under a Web Service Endpoint so as to be accessible by external users VO Factory (VO Creation Tool) The VO Creation Tool constitutes a powerful tool that includes creation and registration mechanisms so as to allow the end-use to perform the creation and the registration of the VOs in an easy way. In addition the tool allows the user to manipulate the VO related data stored in the VO Template Repositories and the VO Registries. Figure 13, presents the overview of the VO Creation Tool functionality in a high-level approach. In particular, the tool offers a user friendly Graphical User Interface (GUI) that helps the end-user to add the description of VOs in a traditional way by completing data forms. The background functionality of the VO Creation Tool takes over the dynamic and automatic transformation of the information into meta-data properties, based on the VO Model, constructing thus the VO Semantics that in turn is stored in the VO Registry. In order Date: 09/05/2014 Grant Agreement number: Page 26 of 63

27 somebody to use the VO Creation Tool, should have specific credentials that are associated with a specific user role and a specific unique API-Key. Depending on the user role, the end user has different and diverse rights in the system. Moreover, the system uses the API-Key so as to enable various algorithmic functionalities for the performance of different action in the system such the VO Registration. Figure 13: VO Creation Tool and associated processes The added value of the VO Creation tool corresponds to the advantage given by the Web-based GUI that allows the users to create in an easy way the description of their VOs, without the need to have IT knowledge or any skill for programming VO Front-End Software and Back-End REST Modules The exploitation of the REST architecture principles constitutes the key concept for the implementation of the development of the VO Front-End (FE) and the VO Back-End (BE) in the Smart Home PoC. The VO FE corresponds to the part that is responsible for the communication between VO CVO and the VO BE is the responsible part for the realization of the communication between VO - ICT. An indicative diagram of what is described above, is depicted in Figure 14. The figure is a simplification as it omits the VO FE interfaces to the VO management unit (and thus the cognitive management functionality). The VOs as SW agents is developed as Web Resources implemented as Web Services accessible over the Internet using REST protocol. By the development of an abstract communication interface for the VO FE it is allowed the accessibility of VOs by various and diverse entities, overcoming the technological heterogeneity. In particular, the VO FE can be developed as one or a set of RESTful Web Services that implements specific functionalities of the VO, such as to provide the measurement data of a sensing process, to retrieve information about VO features, etc. Using a top-level URI that will constitute both the Date: 09/05/2014 Grant Agreement number: Page 27 of 63

28 identifier for the VO, as well as the pointer of the VO as Web Resource that is available in the Internet, it is possible to provide access to external entities which wish to use the VO and its functionalities. CVO VO Registry VO URI VO Front-end Back-end ICT Object Figure 14: VO FE & VO BE Through the exploitation of the information that is available in the VO registry, an external entity can be aware about the top-level VO URI, as well as about additional information regarding its provided functionalities features, etc. Specifically, the data that is associated with the VO are able to describe the VO as a SW entity, giving specific information about how the VO functions can be accessed (lower-level URIs VO function URI) and consumed, which parameters and data-types are required by a function, which are function s output results, etc. Consequently, an external entity can communicate with the VO FE, using the URIs that are associated with the VO and its functions. Under each URI different methods of the VO RESTful WS are implemented and executed in order to provide specific functionality to their consumers, while the instantiation of the VO Information Model, which correspond to this VO provides data for the description of RESTful WS. On the other hand the VO BE exists in Arduino gateway that is used in the Smart Home PoC. Its purpose is to provide access to the functionalities provided by the ICT Objects connected on the Arduino platform VO Back-End Software REST Module for wireless sensor networks The VO Back-End Software REST Module for wireless sensor networks handles functionalities and access of ICT objects connected through standardized and proprietary wireless sensor actor networks (WSAN) like ZigBee, LightweightMesh (LWMesh) and ZIGPOS eertls systems Correlation Matrix This component has been implemented as a RESTful web service. It provides correlations between various VO functions as well as the ability to manage them. More specifically, each correlation is a decimal number from 0.0 to 1.0 representing the satisfaction of one function towards another (e.g. video can fully satisfy image so the correlation between them is 1.0). The correlations are stored within a mysql database Security Agent The Security Agent component is a RESTful web service and it has been implemented in order to ensure security in the VO level. Specifically, when a user is registered in the icore System he is given Date: 09/05/2014 Grant Agreement number: Page 28 of 63

29 certain roles (indicatively: VO_OWNER, SIMPLE_USER, PREMIUM_USER) that represent his access rights. The user roles are stored using a MYSQL database. These roles are then passed on to the VO Registry component whenever he issues a request. The VO Registry then checks, by means of the Security Agent, whether or not the user has access to each VO he requests. This component therefore contributes to the overall security of icore Smart Home CVO This CVO implements the Smart Home Functionality. That includes automated temperature, lighting adjustment and automated alerting of medical centre as well as a self-healing feature to replace broken sensors Smart Car CVO Figure 15: CVO Desktop Interface In this CVO basic automated driving functionality is implemented. This is mainly used in conjunction with the Smart Traffic Light CVO to showcase the coordination mechanism. Date: 09/05/2014 Grant Agreement number: Page 29 of 63

30 Smart Traffic Light CVO Figure 16: Smart Car CVO This CVO is designed to be the controller of an automated traffic light in a Smart City. In the Smart Home context this CVO is used long with the Smart Car to showcase the coordination mechanism. Date: 09/05/2014 Grant Agreement number: Page 30 of 63

31 Figure 17: Smart Traffic Light CVO Smart Lighting CVO (icore BUTLER Integration) The Smart Lighting CVO is a web-based application that is responsible for the sensing of ambient luminosity values, by using the corresponding VO, as well as the real time location of a human user who is moving around in a place, such as a room of a smart home. The CVO gets the above three measurement values and by using a specific formula, which constitutes its service logic, force decisions regarding the control of light that is available in the place where the user is moving. In particular, the end-user can provide its preferences regarding the light intensity in the room, as well as regarding the distance in which the light should be turned on, since the user is close to the light. The light is turned on if the user is going closer to it than the preferred distance and if the light intensity is lower than the current one that is measured. The CVO has been developed by using responsive website technology and therefore it is available either by PC or by Smart Mobile devices (Smart Phones Tablets, etc), just by using a web browser. Date: 09/05/2014 Grant Agreement number: Page 31 of 63

32 Figure 18: Smart Lighting CVO 3.3 Hardware Arduino For the interconnection of sensors-actuators in the Smart Home PoC the Arduino platform is used (Figure 19). Figure 19: Arduino MEGA with Ethernet Shield Date: 09/05/2014 Grant Agreement number: Page 32 of 63

33 Sensors Actuators For the demonstration of the Smart Home PoC features a number of sensors and actuators are used through multiple available hardware platforms (Arduino, WaspMote, RaspberryPi, Flyport etc ): Motion Sensor Light Sensors Temperature-Humidity Sensor Accelerometer Sensor LED Lights Fan Heater (an LED lamp inside a casing) Buzzer PowerPlug Figure 20: Motion Sensor, Light Sensor, LED Light Figure 21: Power Socket Date: 09/05/2014 Grant Agreement number: Page 33 of 63

34 Figure 22: Accerelometer, eertls device Figure 23: Temperature/Humidity sensor Figure 24: Wireless Lamp Figure 25: Light Sensor Various available sensors/actuators are distributed on several hardware platforms to demonstrate hardware independency of icore architecture through use of VO representing ICT objects WaspMote In contrast to Arduino platform with an Ethernet shield, we use WaspMote platform which enables wireless communication. WaspMote, similarly to Arduino, is capable of sensing temperature, humidity and light. However, its wireless connectivity allows Sarah to perform local monitoring of house conditions due to easy platform deployment which is free from connectivity and cabling that might cause some difficulties for her. Another reason to use different hardware platform with somewhat alike functionality is to address heterogeneity problem. Date: 09/05/2014 Grant Agreement number: Page 34 of 63

35 Humidity Light Power supply Wireless connectivity Temperature Figure 26: WaspMote platform with sensors WaspMote platform can be easily adapted to a number of smart home applications by substituting an expansion board located on top of mother board. This is the main difference between Arduino where a user can add/remove single sensors/actuators and WaspMote where a user can connect the entire application specific board with all associated components at once Flyport In contrast to Arduino and WaspMote platforms which focus on sensing and actuation of the house environmental conditions, FlyportIoT platform puts main emphasis on the monitoring of elderly people, i.e. Sarah, health status and house security. The core element of the platform is a PIC microcontroller (MCU) by Microchip. Communication capability is realized by WiFi module by Microchip. Wireless communication makes Flyport easy for deployment platform, so that elderly persons can easily change their location without cabling. In our Smart Home use case we adapt Flyport for two scenarios: Medical: monitoring of body temperature, pressure, and heart rate; in the case when of monitored parameters exceeds a threshold the platform has actuation capabilities for local (visual and sound alarm) and remote (medical center notification via the Internet and WiFi) alarming. Security: monitoring of gas leakage in the Sarah s kitchen and door locking using fingerprint sensor this solution is especially convenient for elderly people and helps them to avoid undesirable situations when they lose an entrance key; similar to medical scenario Flyport performs local and remote notifications in the case of abnormality. Date: 09/05/2014 Grant Agreement number: Page 35 of 63

36 MCU Wi-Fi module Figure 27: FlyportIoT platform Raspberry Pi The RaspberryPi is a credit-card-sized single board computer that gains a lot of interest in communities that use it mainly for development and teaching of basic computer science. Next to many GPIO s that can be used to connect sensors and actuators it comes with Broadcom system on chip (SoC), which allow the use of various Linux distributions. ZIGPOS developed an IEEE (ZigBee) bridge that can be plugged on the Raspberry Pi. In our Smart home use case the Raspberry Pi has been adapted with integrated IEEE device to act as a gateway for wireless sensor networks. ICT object associated in standardized networks (e.g. ZigBee, LWMesh) are automatically associated to the Gateway. Software components implement the VO Back-End Software REST Module for wireless sensor networks connecting automatically to the VO Registry. Date: 09/05/2014 Grant Agreement number: Page 36 of 63

37 Figure 28: Raspberry Pi without case Android Tablet Figure 29: Software architecture combining WSN to icore system An android tablet is used to showcase the mobile app for the Smart Home CVO (Figure 30). The technical specifications of this tablet are: Ram: 1 GB Processor: 1.6 GHz dual core Date: 09/05/2014 Grant Agreement number: Page 37 of 63

38 Screen: 10.1", 1024 x 600 pixels Storage: 8 GB Android version: 4.1 Jelly Bean Figure 30: Android Tablet Android Phone For the demonstration of the mobile app for the Smart Home CVO an android phone is used (Figure 31). The technical specifications of this phone are: Ram: 512 MB Processor: 1 GHz single core Screen: 3.7", 480 x 800 pixels Storage: 4 GB Android version: Jelly Bean Date: 09/05/2014 Grant Agreement number: Page 38 of 63

39 Figure 31: Android Phone Date: 09/05/2014 Grant Agreement number: Page 39 of 63

40 4 Demonstration scenarios 4.1 Dynamic CVO Creation Scenario description In this scenario the user has asked for a CVO that will check on the health of Sarah, the elderly inhabitant of the house in the Smart Home PoC, and that will be able to control the conditions of the room. The user is also able to pick between three devices to instantiate the CVO on (tablet, PC, phone). Furthermore, the user is given a choice of policies (i.e. Low Expenditure, High Quality, etc) and an area (the house in this case) to select where to search for available VOs. The process is better shown on the figure below. Figure 32: MSC for Dynamic CVO creation Interactions between components Initially the user selects the area of interest to him and clicks Fetch Functions. The Service Requestor GUI then sends a VO discovery request to the VO Registry so as to display the available VO functions in the area to the user. Afterwards, the user selects the device the CVO will be instantiated in, any policies he wishes (i.e. Low Expenditure, High Quality) and types a request in natural language to be sent to the NLP component (Natural Language Processing) which will, in turn, send an acknowledgement to the SR GUI and then perform a lexicological analysis on the request. The NLP component will then forward the analysed data to SRA (Service Request Analysis) which will select the optimal service template and forward it to the AROD component. At this point AROD will search for and return to SR GUI the VO functions that will be used in the CVO. The user can now inspect the VO functions that will be used and, if he chooses to continue, forward the completed request to AROD. The AROD component will then check for existing CVO instances that can fulfil the Date: 09/05/2014 Grant Agreement number: Page 40 of 63

41 requested parameters. For this process the CVO Registry will be queried and the results (if any) will be checked one by one for compatibility with the current request and situation parameters. If a compatible enough result is found then that CVO will be re-instantiated [3.2], otherwise a request will be made to the CVO Composition Engine (CCE). The CCE component is responsible for finding the optimal configuration of VOs for the service that has been requested. When that has happened the CCE will send the configuration to the CVO Dynamic Workflow component (Planner) so that a scheme can be made concerning the communication and connection of the VOs. The Planner at this point will save the CVO composition to the CVO Registry and then initiate it. 4.2 Knowledge-based Instantiation Scenario description In this scenario a user requests, once more, a service as described in the Dynamic CVO creation scenario. In this case a CVO that can fulfil the requested service already exists in the CVO Registry Interactions between components The process described in the Dynamic CVO creation is repeated as is until the AROD component is reached. That is, the user needs to select an area of interest, request a service from NLP and receive a set of VO functions. Afterwards the data are once more sent to the AROD component. AROD in this case will detect the previous CVO and initiate it once again without the need of going through the CCE and Planner components that are responsible for a large portion of the CVO creation time. 4.3 CVO Self-Healing Scenario description A light sensor in the Smart Home CVO is damaged and sending false readings Interactions between components When the light sensor is detected as damaged (or when it is removed for demonstration purposes) the CVO will initially display a message with the information. It will then move on to send a replacement request to the AROD component which will in turn make the specific VO appear as Unavailable (since it is damaged) in the VO Registry and forward the information to the CCE component. The CVO Composition Engine will then look for an available VO (in the area that had been specified for the CVO creation) and assuming it finds one it will change the composition of the CVO that is saved in the CVO Registry and forward the information to AROD. AROD will then send the new sensor information to the CVO and, finally, the CVO will display a message that the process has been completed and start taking readings from the new sensor. 4.4 CVO Coordination Scenario description The Smart Home CVO, Traffic Light CVO, icore Car CVO and the Coordination mechanism are active. Sarah, the inhabitant, tries to cross the road while traffic is inbound. Date: 09/05/2014 Grant Agreement number: Page 41 of 63

42 Smart Home CVO Smart Car CVO Smart Traffic Light CVO CVO Management Unit Coordination Mechanism Request for Green Light Requests Green Light Conflicting requests Sends request for conflict resolution Conflict Resolution Request Decisions on Traffic Light management Resolves the conflict and sends the corresponding decisions Response on Green Light request Response on Green Light request Interactions between components Figure 33: CVO Coordination Process When Sarah leaves the house a motion sensor will pick up her movement. The Smart Home CVO will then activate the porch light so she can see and also send a request to the Traffic Light CVO to stop any cars trying to pass. Meanwhile the icore Car CVO is constantly sending requests to the Traffic Light to get a green light. That means that when Sarah tries to cross the street the Traffic Light will have received two requests that it cannot resolve at the same time. To solve this conflict the Traffic Light CVO will forward the information to the Coordination mechanism. The coordination mechanism then decides (through weight data that exists for each entity car, human) which of the two requests has priority over the other. In this case Sarah has priority over the car so the Coordination mechanism sends the decision to the Traffic Light which will then turn green for pedestrians and red for cars. 4.5 icore BUTLER Integration Scenario Description This scenario is used for showing abilities to use cross communications between two different IoT based platforms named i-core and BUTLER to fulfil requested services using advantages of both platforms. Here Sarah wants to automate the light control based on her preferred conditions and location inside her environment. Therefore, she requests a service able to check light intensity conditions and setting light via actuators inside her environment. In detail, the lights will be controlled and turned on, if the current light intensity is below her preferred threshold and if she is nearby the light itself. Otherwise the light will stay off for the purpose of energy saving Interaction between components In this Demonstrator a BUTLER- system is running providing location based context from a BUTLER compliant Smart Server. Therefore an RTLS-system is connected to the Smart Gateway providing the BUTLER system location information of a mobile node connected to Sarah. In the BUTLER system a Smart Server is able to retrieve information from any associated ICT object through the Smart Gateway and use/forward it to other modules/components. This Smart Server is able to associate to an icore architecture using the VO-registry method and register itself as a VO able to provide location based context to the system. Date: 09/05/2014 Grant Agreement number: Page 42 of 63

43 The VO discovery request to the VO Registry results in the available VO functions that are in particular light actuators, light intensity sensors and location information VO. When the luminosity sensor recognize a value below the threshold the Smart Home CVO will then request the actual position of Sarah and compare it to the position of the light of interest. If the location matches the location of the lamp actuators the CVO will request the Lamp to turn on. Also, if the lamp is turned on and the Smart Home CVO recognizes that Sarah has left the Position and is out of scope of the lamp, it will request the device to turn off in order to save energy. Figure 34: Integration architecture 4.6 Performance Results This section presents some indicative results on the performance of the implemented cognitive management framework (as has already been written [3]) in terms of the time required and the bytes exchanged for the various entities of the framework to accomplish their task, including the time for the CVO instantiation. It should be noted that the presented results are mainly related to the Dynamic CVO Creation and Knowledge-based CVO instantiation processes as these are the most demanding among the four processes described in the previous section Prototype Implementation A prototype of the proposed cognitive management framework has been implemented. The current corresponding implementation comprises, apart from software components for the various functional components presented in the previous chapter, a number of actual sensors and actuators. In terms of hardware aspects of the equipment that was used for the experiments, the NLP and SRA mechanisms, as well as the Service Template Repository were executed on a PC with: CPU: Intel(R) Core(TM) i7-3632qm at 2.20GHz memory (RAM): 8.00 GB In addition, the AROD and CCE mechanisms, as well as the CVO Registry were hosted on a PC with: CPU: Intel(R) Core(TM) i7-2630qm at 2.00GHz Date: 09/05/2014 Grant Agreement number: Page 43 of 63

44 memory (RAM): 4.00 GB The VO Registry was hosted in a server machine with: CPU: Intel(R) Xeon(R) E5520 at 2.27GHz memory (RAM): 16.0GB The RWOs that were used and represented as VOs consisted of different types of sensors and actuators, such as temperature, humidity, luminosity sensors, lamps, fans, etc. Finally, the communication between the different machines was performed over a LAN and each machine was connected via Wi-Fi on a wireless network router Service level mechanisms performance Figure 35, presents results on the performance of Service level mechanisms with and without prior knowledge about a given user request. In the case that no prior knowledge is available all available Service Templates have to be searched in order to select the most appropriate one and derive corresponding VO and CVO information. Thus, this approach is dominated by the amount of available Service Templates and the amount of nodes comprised in each of the Service Templates. Actually, the complexity of the Service template selection process is: ( ) which means that, as the amount of the Service Templates (i.e. w) and the amount of nodes (i.e. n) of each Service Template increase, the execution time increases too. In the case of available knowledge on a particular user request, Bayesian statistics are exploited for the derivation of the optimal Service Template, thus eliminating the need of looking into each node of each Service Template. Therefore, the overall process of selecting the most appropriate service template in this case depends only on the number of Service Templates. So, the complexity of this selection process is: () As can be observed in Figure 35, when prior knowledge is available less than half the time is required for processing a user request by the Service level mechanisms. Generally, the goal of automated users requirements analysis and matching for the dynamic creation and configuration of CVOs with the aim of providing smart, personalized services through the selection of the most appropriate Service Template is fulfilled in a realistic time-frame considering that the Service level mechanisms execution take place during an initial, design-time phase. At this point it should be highlighted that the two lines on the diagram, will never intersect each other, since they have totally different increasing rate in proportion of the increasing number of available Service Templates. Further optimization of the Service level mechanisms execution, is scheduled for future work. Date: 09/05/2014 Grant Agreement number: Page 44 of 63

45 Figure 35: Service Level execution time vs. Number of Available Service Templates CVO and VO level mechanisms performance This sub-section focuses on results related to the performance of CVO and VO level mechanisms and processes. In particular, the time required for the execution of the AROD and the CCE entities as well as the VO discovery processes is presented. More specifically, results for eight different cases of the number of available VOs that could possibly be part of the created CVO are presented. In addition impact of the diversity of the structure/nature of each VO in terms of its representation data is investigated. The eight different cases correspond to increasing numbers of the available VOs ranging from 10 up to 80 and the diversity of the amount of information that correspond to each VO. The information for each VO is represented as a meta-data graph, which has a specific size, depending on the VO features. The graph includes associated meta-data containers with specific meta-data properties and each container corresponds to a specific VO feature, such as the VO functions, function inputs, outputs, etc. Each graph has its own size that is expressed in terms of the number of statements, where each statement represents a meta-data property. It should be noted that some of the VOs were emulated, i.e. did not correspond to the actual RWOs, while in each case, the average number of statements was between 130 and 150 statements per VO. The diversity of the VO information size and the different number of VOs, affect the overall execution time of the related processes in a linear increase of time. The experiments have been repeated five times for each case. Although the outcomes were quite similar in each case, the provided results correspond to the mean times, in order to be more accurate. In all cases, the requested CVO was instantiated successfully. Figure 36 depicts the time required for the operation of the AROD entity, which searches for appropriate CVOs based on specific criteria, which correspond to situation and request parameters. This execution time is quite constant and very low (around 9 msec). Denoting with x the number of the stored CVOs, the complexity of this process can be computed as: Date: 09/05/2014 Grant Agreement number: Page 45 of 63

46 () It should be noted that only a small number of previously recorded CVOs are considered, i.e. two. Furthermore, it is observed that this operation is not directly affected by the number of available VOs in the area of interest, but by the number of the available CVOs that match the search criteria and include a specific number of VOs in their composition. Figure 37 presents the time required for the CCE entity to select the optimal CVO composition according to the available VOs that were suggested by the AROD entity, while information about these VOs is retrieved from the VO registry. This time corresponds to the optimization process of selecting the appropriate VOs for the instantiation of the optimal CVO. The operation of CCE is based on a heuristic algorithm that aims at identifying a suitable composition in minimum time. The algorithm splits the problem into smaller separate ones (Divide and Conquer approach). Its objective is to find the optimal solution for each requested function separately and afterwards, to compose them into an optimal composition of functions. In particular, each function of the available VOs is evaluated, taking into account the defined policies and the features. For each requested function, one optimal function of a VO is found. At the end of this process, a number of functions of VOs are selected for providing the requested functions. The same VO may be exploited for more than one requested function. The time-efficiency of the CCE can be justified through its low complexity. Denoting as v the number of VOs and f the number of requested functions, its complexity can be calculated from the following formula: ( ) as v available VOs need to be evaluated for each of the f requested functions. As can be observed (and as expected) the time required for the selection of appropriate VOs increases as the number of available VOs increases, as there are more VOs to be considered. However, it remains within realistic boundaries, especially considering that the CCE process for the selection of VOs will take place initially during the (automated) creation of the CVO. Figure 38presents the time required for the VO discovery process, which actually includes the time for the discovery and acquisition of information of relevant available VOs. As can be observed, the time required for retrieving information on the available VOs, linearly increases as the number of relevant VOs in a certain area of interest increases. However it remains realistic especially when considering that this time is related to the initial discovery of VOs for the automated creation of a CVO, and not during run-time of a particular service/application. Further mitigation of the VO discovery time could be achieved by using distributed VO registries and parallelization of the VO search. This is an aspect for future work. As we described above, the description data for each VO consist of a set of statements. In addition, the number of statements per VO may be different. Thus, the complexity of the VO Discovery process can be presented by the statement: () where s corresponds to the total number of the statements that compose the data for the description of the available VOs. Figure 39 presents the overall time, which is needed in each case for the creation of a (new) CVO and the knowledge-based instantiation of an existing (previously recorded) CVO. The time for the creation of a new CVO ( from scratch ) comprises the time for; (i) the AROD, (ii) the CCE and (iii) the VO discovery. The time for the knowledge-based instantiation of an existing CVO on the other hand corresponds to the time required for the AROD. The time for the knowledge-based instantiation of an existing CVO remains much lower than the time required for the more complex process of a new CVO. As we observe a major portion of the needed overall time for the instantiation of the new CVO is due to the VO discovery process. Table 1 provides an overview of the ratio of the VO discovery process execution time to the overall CVO instantiation. As can be observed the VO discovery process execution time ranges from 40% to 46% with respect to the overall CVO instantiation time. Finally, the amount of bytes that were received from each entity in these cases has been measured (Figure 40). The number of the received bytes is similar for the AROD and the CCE mechanisms. This Date: 09/05/2014 Grant Agreement number: Page 46 of 63

47 is anticipated as the major portion of this data corresponds to the information that is retrieved from the VO registry for the requested VOs. As expected this amount of bytes increases as the number of available relevant VOs (in a particular area of interest) increases. On the contrary, the number of bytes that are received by the VO registry is quite stable, low and is due to the VO IDs only sent by the AROD and/or the CCE mechanisms. Again this amount of bytes increases as the number of available relevant VOs (in a particular area of interest) increases. In general, from the results derived, it can be concluded that the proposed cognitive management mechanisms perform well and thus their deployment for commercial use is feasible. Figure 36: Approximation & Reuse Opportunity Detection execution time vs. Number of Available VOs Date: 09/05/2014 Grant Agreement number: Page 47 of 63

48 Figure 37: CVO Composition Engine execution time vs. Number of Available VOs Date: 09/05/2014 Grant Agreement number: Page 48 of 63

49 Figure 38: VO Discovery Process execution time vs. Number of Available VOs Date: 09/05/2014 Grant Agreement number: Page 49 of 63

50 Figure 39: Overall CVO Creation/Instantiation execution time vs. Number of Available VOs Date: 09/05/2014 Grant Agreement number: Page 50 of 63

51 Figure 40: Received Bytes vs. Number of Available VOs Date: 09/05/2014 Grant Agreement number: Page 51 of 63

52 5 Specification of interfaces Listed below are the interfaces between the implemented components of the Smart Home PoC. 5.1 Interface between Service Requestor GUI and Natural Language Processing This interface provides the interaction between the Service Requestor GUI and the Natural Language Processing (NLP) component. Through this interface, the Service Requestor GUI provides a request in natural language. Moreover, it provides the user role, the date that the request was triggered and the corresponding location. On the other hand, the NLP component analyses the request into words. In addition, it finds the Part-of-Speech tags of each word and the syntactical dependencies between the words. Method String executenlp (String request_in_nl, AppSituationParamssitParams,AppRequestParamsreqParams) Request (Input to NLP) String request_in_nl. The service request expressed in natural language. AppSituationParams: Complex structure that defines the situation parameters in terms of; (a) Date, (b) RectangleArea and (c) UserRole. AppRequestParams: Complex structure that defines the request parameters in terms of; (a) HashSet<VOFunction>, (b) ICVOHost. Response (Output from NLP) String notificationmessage: The message takes one of two values; OK or ERROR. Datatype Structure AppSituationParams: [Date,RRectangle, String] Included Parameters (a) Date date: It s a variable that represents date and time of service execution triggering. (b) RRectangle rectanglearea: This object holds four numeric doubles (Double xmin, Double xmax, Double ymin, Double ymax ) which represent the location where the request was triggered. (c) String userrole: It s a String that declares the role of user who triggered the request. Datatype Structure AppRequestParams: [HashSet<VOFunction>,ICVOHost] Included Parameters (a) HashSet<VOFunction> functions: It s the collection of selected VO Template Names. Date: 09/05/2014 Grant Agreement number: Page 52 of 63

53 (b) ICVOHostcvoHost: It s an object that holds the URI of CVO. 5.2 Interface between Natural Language Processing (NLP) component and Service Request Analysis (SRA)component This interface provides the interaction between the Natural Language Processing (NLP) component and the Service Request Analysis (SRA) component. In particular, the NLP component triggers the SRA component by providing its results which are a bunch of words, the related Part-of-Speech tags, the syntactical dependencies between them, the Situation Parameters and the Request Parameters of the service request. The SRA component after getting the above attributes, it proceeds to the selection of the most suitable Service Template. Method String executesra (Map<Integer,String> words, Map<Integer,String> tags, Map<Integer,String> dependencies, AppSituationParamssitParams, AppRequestParamsreqParams) Request (Input to SRA) Map<Integer,String> words: The service request split into sentences. Each sentence tokenized into words. Map<Integer,String> tags: Tag of each word per sentence.for example, the n and v corresponds to nouns and to verbs, respectively. Map<Integer, String> dependencies: Syntactical dependencies between words for each sentence.for example, the dobj[distributes, products] means that the noun products is the direct object of the verb distributes. AppSituationParams: Complex structure that defines the situation parameters in terms of; (a) Date, (b) RectangleArea and (c) UserRole. AppRequestParams: Complex structure that defines the request parameters in terms of; (a) HashSet<VOFunction>, (b) ICVOHost. Response (Output from SRA) String notificationmessage: [OK, ERROR] 5.3 Interface between Service Request Analysis (SRA) component and User Profiling component This interface provides the interaction between the Service Request analysis (SRA) component and the Service User Profiling component. The SRA component requests from the SRA component the Conditional Probability Tables (CPTs) of the user who triggered the service request. The CPTs, among the others, contain the preferences of this user regarding previous selected Service Templates. These CPTs will be used by the SRA component in order to select the optimal Service Template by exploiting these user preferences. Method CPT getuserpreferences (String userrole) Request (Input to User Profiling) String userrole: The user role of thecorresponded user Response (Output from User Profiling) Date: 09/05/2014 Grant Agreement number: Page 53 of 63

54 Conditional Probability Table (CPT): The Conditional Probability Tables contain probabilities about User Preferences, User Roles, Service Templates, User Request. It is retrieved from User Preferences and it is forwarded to Service Request Analysis. 5.4 Interface between Service Request Analysis (SRA) component and CVO Management Unit component This interface provides the interaction between the Service Request Analysis (SRA) component and the CVO Management Unit component. The Service Request Analysis sends the selected VO Template Names in the format of Service Execution Request (SER). The CVO Management Unit proceeds to the creation of the optimal CVO. Method String sendser(serobjectser) Request (Input to CVO Management Unit) SERobjectser: This object holds the Service Execution Request. Among others, it holds the selected VO Template Names in format of HashSet<VOFunctions>. It contains the connected graph of CVO and VO types. It is forwarded to the CVO Management Unit in order to start the CVO creation. Response (Output from CVO Management Unit) String notificationmessage: This is the response/acknowledgement from the CVO Management Unit in order to report a failure/ or notification. May take the values OK or ERROR. Datatype Structure SERobject: [String, AppSituationParams, AppRequestParams] Included Parameters String: Defines a unique identifier for the request AppSituationParams: It's acomplex structure that defines the situation parameters in terms of: Datedate: It s a variable that represents date and time of service execution triggering RRectanglerectangleArea: This object holds four numeric doubles (Double xmin, Double xmax, Double ymin, Double ymax ) which represent the location where the request was triggered. StringuserRole: It s a String that declares the role of user who triggered the request. AppSituationParams: Complex structure complex structure that defines the situation parameters in terms of: HashSet<VOFunction>functions: It s the collection of selected VO Template Names. ICVOHostcvoHost: It s an object that holds the URI of CVO. Date: 09/05/2014 Grant Agreement number: Page 54 of 63

55 5.5 Interface between CVO and Service Request Analysis (SRA) component This interface provides the interaction between the CVO and the Service Request Analysis (SRA) component. After the service execution, the CVO provides to the Service Request Analysis component the user satisfaction in order to update the user preferences related to the selected Service Template. The user satisfaction is in the format of a grade (i.e. 1-5) which the user inserts it when he prompts to. Method String senduserfeedback (URI requestid, String userfeedback) Request (Input to SRA) URI requestid: The unique identifier of the current request. String userfeedback: The user grade which represents the satisfaction of user for the service execution. This user feedback is used from the learning mechanism of User Profiling in order to update the Conditional Probability Tables (CPTs) in the User Preferences entity. Response (Output from SRA) String notificationmessage: This is a notification for the progress of training the Bayesian Networks. May take the values OK or ERROR. 5.6 Interface between AROD and CVO Registry The purpose of this interface is to allow the interaction between the AROD component and the CVO Registry. Specifically, the AROD component is able to search for existing CVO instances by using specific search constraints. Since the search constraints match the existing CVOs, the AROD component receives related information on them. Method CVORegistryResponse CVODiscovery (CVORegistryRequest _request) Request (Input to CVO Registry) CVORegistryRequest: Complex structure that defines the request towards the CVO Registry. Response (Output from CVO Registry) CVORegistryResponse: Complex structure that defines the possible status of the Response and the available CVOs that match the search constraints. Datatype Structure CVORegistryRequest: [Request-id,Request-Type, Content-Type, HTTP-Method, CVO Registry API-Key, User-Role, Access-Right, Request Data] Included Parameters (a) Request-id: URI that constitutes the unique identifier of the request Date: 09/05/2014 Grant Agreement number: Page 55 of 63

56 (b) Request-Type: String that describes the type of the request in terms of Registration, Discovery, Update, Delete. (c) Content-Type: String the presents the type of the included data, e.g. SPARQL, XML, JSON, etc. (d)http-method: String that presents the HTTP-method that is used for the request, (e.g. HTTP/GET) (e)cvo Registry API-Key: String that corresponds to a unique identifier for the entity which uses the API. (f)user-role: Array of Strings that describe the Role(s) of the User in the icore system. (g)access-right: Array of Strings that describe the access rights that have been associated to the specific entity / user. (h)request Data: String that corresponds to the data format of the CVORegistryRequest payload. Datatype Structure CVORegistryResponse: [Status-Code,Status-Message, CompositeVirtualObject] Included Parameters (a) Status-Code: Integer that presents the code of the response status, (e.g. 200). (b) Status-Message: String that presents the message of the response, (e.g. Success ). (c)compositevirtualobject: Array of URIs that corresponds to the CVOs, which match the CVORegistyRequest. 5.7 Interface between AROD and VO Registry The purpose of this interface is similar with the previous one on AROD and CVO Registry interaction, namely the AROD component to be able to search for existing VO instances by using specific search constraints. Since the search constraints match the existing VOs, the AROD component receives related information on them. Method VORegistryResponse VODiscovery (VORegistryRequest _request) Request (Input to VO Registry) VORegistryRequest: Complex structure that defines the request towards the VO Registry. Response (Output from VO Registry) VORegistryResponse: Complex structure that defines the possible status of the Response and the result of the available VOs. Datatype Structure VORegistryRequest: [Request-id, Request-Type, Content-Type, HTTP-Method, VO Registry API-Key, User-Role, Access-Right, Request Data] Included Parameters Date: 09/05/2014 Grant Agreement number: Page 56 of 63

57 (a) Request-id: URI that constitutes the unique identifier of the request (b) Request-Type: String that describes the type of the request in terms of Registration, Discovery, Update, Delete. (c) Content-Type: String the presents the type of the included data, e.g. SPARQL, XML, JSON, etc. (d)http-method: String that presents the HTTP-method that is used for the request, (e.g. HTTP/GET) (e)cvo Registry API-Key: String that corresponds to a unique identifier for the entity which uses the API. (f)user-role: Array of Strings that describe the Role(s) of the User in the icore system. (g)access-right: Array of Strings that describe the access rights that have been associated to the specific entity / user. (h)request Data: String that corresponds to the data format of the VORegistryRequest payload. Datatype Structure VORegistryResponse: [Status-Code,Status-Message, VirtualObject] Included Parameters (a) Status-Code: Integer that presents the code of the response status, (e.g. 200). (b) Status-Message: String that presents the message of the response, (e.g. Success ). (c)virtualobject: Array of URIs that corresponds to the VOs, which match the VORegistyRequest. 5.8 Interface between VO Registry and Security Agent (SA) This interface allows the VO Registry to interact with the Security Agent component in order to check if the required permissions for each request are satisfied by the user role. Method String authenticate (String apikey, HashSet<String> roles) Request (Input from VO Registry) String apikey: A user unique key for the authentication process. HashSet<String> roles: The user roles for the user (that identify his permissions). Response (Output from Security Agent) ok (if the user was successfully authenticated) unauthorized (if user does not have the required permissions) StackTrace (if an unknown error occurred) 5.9 Interface between Approximation and Reuse Opportunity Detection (AROD ) component and CVO Composition Engine (CCE) component This interface provides the interaction between the Approximation and Reuse Opportunity Detection (AROD) component and the CVO Composition Engine (CCE) component. The AROD Date: 09/05/2014 Grant Agreement number: Page 57 of 63

58 component triggers the CCE component by providing data related to the requested VOs. On the other hand, the CCE component responds to AROD component with the optimal constructed CVO for the corresponded Service Execution Request. Method CCEARODResponse VOEvaluation (ARODCCERequest _request) Request (Input to CCE) ARODCCERequest: Complex structure that defines the set of data, which corresponds to the request from AROD to the CCE mechanism. It contains situation and request parameters corresponding to the request. For example, the area of interest, the time the request was issued or the available VOs in this area, the VO type names and policies/requirements. Response (Output from CCE) CCEARODResponse: Complex structure that defines the set of data, which corresponds to the response from CCE to the AROD mechanism. In particular, amongst the others, it containsa set of VOs that are compliant to the request and provide an optimal composition based on the requested policies. Datatype Structure ARODCCERequest: [String, AppSituationParams, AppRequestParams, ARODInterface] Included Parameters String: Defines a unique identifier for the request AppSituationParams: Complex structure complex structure that defines the situation parameters in terms of: HashSet<VirtualObject>: The available objects in the area of interest. Date: This represents the date and time of the request RectangleArea: This represents four doubles x min, x max, y min, y max, defining the area of interest String: It defines the user role AppSituationParams: Complex structure complex structure that defines the situation parameters in terms of: HashSet<VOFunction> : This represents the requested functions HashSet<Policy> : This represents the requested policies ICVOHost : This represents the CVO host interface ARODInterface: This data type represents and holds the AROD interface. Datatype Structure CCEARODResponse: [ARODCCERequest, CompositeVirtualObject, CCEARODResponseStatus] Included Parameters ARODCCERequest: It's acomplex structure that defines the request from AROD Date: 09/05/2014 Grant Agreement number: Page 58 of 63

59 CompositeVirtualObject: Complex structure that defines final result of the composition of the VOs, in terms of: ArrayList<FuncVOUriPair>: Arraylist with functions of VOs and URIs of these functions VOFunction URI CCEARODResponseStatus: Complex structure that defines the possible status of the Response, and it can take three values: OK ERROR UNITIALIZED 5.10 Interface between CVO Composition Engine (CCE) component and Correlation Matrix (CM) component This interface provides the interaction between the Correlation Matrix (CM) component and the CVO Composition Engine (CCE) component. The CCE component requests from the CM component, the correlation value for pairs of VO type names. On the other hand, the CM component responds to the CCE component with the correlation value of the requested pair of VO type names. Method CorrelationMatrix getcorrelationmatrix(correlationrequest _request) Request (Input to CM) CorrelationRequest: It's a complex structure that describes the requested functions of VOs. Stringf1: Corresponds to a function of a VO String f2: Corresponds to a function of a VO double correlation: Default value null Response (Output from CM) CorreletionMatrix: It's acomplex structure that includes correlation values for requested functions of VOs: List<Correlation> correlations 5.11 Interface between CVO Software Factory and CVO Template Repository Through this Interface the CVO Software Factory can request non-instantiated CVO templates from the CVO Template Repository. Method CVOTMPRepositoryResponse CVOTMP_Discovery (String[] _request) Request (Input to CTR) String[]: An array of strings that correspond to the non-instantiated CVO Template names, which are requested from the CVO templaterepository. Date: 09/05/2014 Grant Agreement number: Page 59 of 63

60 Response (Output from CTR) CVOTMPRepositoryResponse: Complex structure that defines the possible status of the Response and the result of the CVO Templates that match the search constraints. Datatype Structure CVOTMPRepositoryResponse: [Response-Id,Status-Code, Status-Message, CVO Template] Included Parameters (a) Response-Id: A unique identifier in terms of URI that represents the id of the response message. (b) Status-Code: Integer that presents the code of the response status, (e.g. 200). (c) Status-Message: String that presents the message of the response, (e.g. Success ). (d)cvo Template: Complex structure that defines a CVO Template in terms of; (1) URI, (2) LogicModule and (3) LinkedHashSet<VOTemplate> Interface between CVO Composition Engine and Planner This interface is used for the exchange of data between the CCE and the Planner components. The CCE forwards the selected instances of VOs and the Planner elaborates this information and provides the plan that corresponds to these instances. This plan describes their relationship (workflow). Method CCEPLResponse PlanCreation (CCEPLRequest request) Request (Input to Planner) CCEPLRequest request: Complex structure that defines the set of data, which corresponds to the request from CCE to the PLANNER mechanism Response (Output from Planner) CCEPLResponse response: Complex structure that defines the set of data, which corresponds to the response from PLANNER to the CCE mechanism Datatype Structure CCCEPLRequest: [PlannerRequest] Included Parameters (a) String id: Request id (b) Vector<State> start: Set of states (c) Vector<State>goal: Set of states (d)vector<action>actions: Set of actions, derived from the available functions State: Action: String : Describing the state Date: 09/05/2014 Grant Agreement number: Page 60 of 63

61 String : Describing the action Preconditions : Set of states Effects : Set of states Datatype Structure CCEPLResponse: [String-id, String-status, PlanResult] Included Parameters (a) String-id: Request id (b)string-status: Either OK or ERROR (c)planresult: Complex structure that describes the graph that corresponds to the plan. It consists of states and actions. Date: 09/05/2014 Grant Agreement number: Page 61 of 63

62 6 Conclusion This document has demonstrated the enhanced description of proof of concept for the icore Smart Home application presented in the deliverable document D6.1. This deliverable described icore concepts and architecture components in more details and reported on improvements and updates on some of them, in particular, showcasing of demonstration scenarios and specification of components interfaces. The deliverable first presented detailed description of icore components related to Service, VO, CVO levels. Section 3 of the document has shown a number of demonstration scenarios on self-x functionalities, components management, integration with BUTLER components followed by performance evaluation at Service, VO, and CVO levels. Section 4 has provided the specification of interfaces between all implemented components within Smart Home PoC. It is worth noting that this deliverable apart from describing theoretical aspects of icore components, interfaces, and functionality for Smart Home PoC has demonstrated the implementation examples and performance evaluation for some of them. This approach on implementation of theoretical principles of icore was well appreciated in a number of exhibitions and resulted in winning of awards at FuNeMS 2012 and Date: 09/05/2014 Grant Agreement number: Page 62 of 63

63 7 References [1] FP7/ICT project icore (Internet Connected Objects for Reconfigurable Eco-systems, ICT ), Oct Sept. 2014, Website: accessed Sep [2] icore Deliverable 5.2 (D5.2), Final version of the Application knowledge inference toolkit, December [3] Dimitris Kelaidonis et.al., Realizing the Internet of Things through a Cognitive Management Framework, IEEE IoT Journal, submitted for publication. Date: 09/05/2014 Grant Agreement number: Page 63 of 63