Knowledge Representation Methods for Smart Devices in Intelligent Buildings Ph.D. candidate: Supervisor: Dr. Michele Ruta 22 marzo 2013
Outline The Semantic Web of Things vision Home and Building Automation (HBA): state of the art Standards and appliances Ambient Intelligence Knowledge-based HBA: framework and approach Semantic enhancement to EIB/KNX standard Mini-ME: the Mini Matchmaking Engine Agent Framework Conclusion and future work 2 of 32
Semantic Web of Things Emerging vision in ICT, integrating the Internet of Things [ITU, Internet Reports, 2005] and Semantic Web [Berners-Lee et al., Scientific American, 2001] approaches: promote the pervasive computing paradigm on a global scale share, reuse and combine information in the World Wide Web Process semantic annotations coming from a large numbers of heterogeneous micro-devices Embed intelligence into everyday objects and locations Take into account pervasive computing constraints unpredictability dependence on context severe resource limitations Adapt procedures and tools for query and reasoning to the pervasive computing requirements 3 of 32
Knowledge Representation Systems Interface TBox ABox Reasoning Services Editing Services Applications Terminological Box (a.k.a. ontology, conceptual knowledge) describes a domain with respect to a reference vocabulary containing classes and properties Assertional Box (factual knowledge) contains individuals describing the modeled excerpt of the observed reality Knowledge Base (KB = TBox + ABox) used to infer further knowledge Classic KBs are intended as fixed and centralized components 4 of 32
SWoT Architecture [Ruta et al., IEEE-ICSC, 2012] Ubiquitous Knowledge Bases (u-kbs), evolution of classic KB paradigm assertional knowledge distributed across multiple micro-devices individuals characterized by: ( address 1. unique ID (e.g. EPC code, MAC 2. OUUID of reference ontology 3. semantic annotation 4. data-oriented attributes discovery protocol to build a local KB subset only when reasoning is needed 5 of 32
Home and Building Automation: state of the art Goal Increase comfort and building efficiency Decrease waste and maintenance costs Integration of different home systems Most important HBA standards: ZigBee (HA Profile) LonWorks X-10 EIB/KNX low Cost widespread ethernet support (KNXnet/IP protocol) 6 of 32
Ambient Intelligence Classic Domotics Static and not flexible architectures Constrained interoperability Reduced functionalities and scenarios User-driven interaction (low autonomicity) Agent-based Domotics Flexible and scalable Services and resources accessible via agent-oriented frameworks Concurrency, cooperation, negotiation enabled Semantic-based Domotics Improved interoperability Rich description of user/service profiles Decentralized architecture supporting autonomous device-driven interactions 7 of 32
Proposed approach Goal A knowledge-based agent framework for HBA: home self-configuration through collaboration of autonomous smart agents, capable to provide services and address complex requests semantic annotation of user profiles, device settings and appliance behaviors w.r.t. an OWL-DL ontology modelling typical home environments Description Logics [Baader et al., DL Handbook, 2003] are a family of logic languages for Knowledge Representation in a decidable fragment of First Order Logic ALN (Attributive Language with unqualified Number restrictions) DL was chosen as reference language Name Syntax Semantics Top / Bottom Concept, All the objects in the domain / The empty set Atomic Concept / Atomic Negation A, A All the objects belonging / not belonging to the set A Intersection C π D The objects belonging to both sets C and D Universal Restriction R.C All the objects participating in the R relation whose range are the objects belonging to set C Unqualified Existential Restriction R At least one object participating in the relation R Unqualified Number Restrictions ( n R), ( n R) (= n R) Respectively the minimum, the maximum and the exact number of objects participating in the relation R 8 of 32
Proposed solution Technological Solution A. Semantic-based enhancement of EIB/KNX protocol standard [Ruta et al., IEEE ICM, 2011]: integration of a semantic micro-layer preserving a full backward compatibility advanced service and resource discovery support B. Logic-based negotiation process to: adapt concept covering [Ragone et al., JWSR, 2007] to select one or more functionalities whose combination fills the user/device request negotiate on available home and energy resources through a usertransparent and device-driven interaction discover the (set of) elementary services that maximize the overall utility support non-expert users in selecting home configurations ranked w.r.t. a global utility 9 of 32
Framework Architecture [Ruta et al., IEEE TII, 7(4), 2011] KNX NETWORK EIB/KNX BUS KNX ROUTER Device Manager LAN IP BACKBONE NETWORK Client Manager Mobile Matchmaker CENTRAL UNIT MOBILE CLIENTS 10 of 32
A. New Interface Objects (IOs): EIB/KNX semantic extension [Ruta et al., IEEE ICM, 2011] Generic Profile of the Device (GPD) object: 1 device, 1 GPD Specific Profile of the Device (SPD) object: 1 device, n SPDs B. New IOs structure and properties: PID_OUUID: 16-bit Ontology Universally Unique Identifier (OUUID) marking the reference ontology PID_OUUIDs: specifies the OUUID group (only for GPDs) PID_SEMANTIC_HEADER: compressed annotation - header PID_SEMANTIC_BODY: compressed annotation - body C. New Application layer Protocol Control Information (APCI) codes for application layer primitives: A_SEMANTIC_SUBMISSION.req A_SEMANTIC_SUBMISSION.res homomorphic compression of ontological languages 11 of 32
Prototypical Testbed (1/2) Camera Weather Station Alarm Testbed Main Panel 12 of 32
Prototypical Testbed (2/2) Central Unit GUI Mobile Client 13 of 32
User Profiler (1/2) Goals: Avoid explicit user interaction Automatically detect user needs by exploiting information available on her smartphone 14 of 32
User Profiler (2/2) Features: Identify the daily stay points Retrieve the nearest Point Of Interest (POI) querying suitable Web services Google Places LinkedGeoData [Stadler et al., SWJ, 2012] Build a daily user profile 15 of 32
Mini-ME: the Mini Matchmaking Engine [Ruta et al., ORE, 2012] Basics A prototypical mobile matchmaker for semantic-based pervasive contexts Most widespread matchmakers cannot run on common mobile platforms without a significant re-write or re-design effort Support to Semantic Web ontology languages and syntaxes through the OWL API Reasoning Services Standard inferences (subsumption, satisfiability, classification) Non-standard inferences for moderately expressive KB concept abduction [Ruta et al., WIAS, 2011]: determines what should be hypothesized in a resource S in order to completely satisfy a request D. The solution H (Hypothesis) can be interpreted as what is requested in D and not specified in S concept contraction [Ruta et al., WIAS, 2011]: determines which part of D is conflicting with S. If one retracts conflicting requirements in D, G (Give up), a concept K (Keep) is obtained, representing a contracted version of the original request concept covering [Ragone et al., JWSR, 2007]. 16 of 32
Concept Covering Problem (CCoP) [Ragone et al., JWSR, 2007] Given a request D and a set of resources S = {S 1, S 2,..., S k } the Concept Covering Problem (CCoP) aims to find a pair (S C, H) where S C includes concepts in S (partially) covering D and H is the (possible) part of D not covered by concepts in S C Resources can be described by means of the IOPE model (Input, Output, Precondition, Effects) [Martin et al., WWW, 2007] Automatic selection, composition, and interoperation of resources to perform a complex task, given a high-level description of an objective Attempt to satisfy a request as much as possible, through the conjunction of elementary resources available in the environment 17 of 32
Experiments: Ontology Classification Correctness of implementation and comparison with state-of-the-art matchmakers Reference dataset: 143 ontologies Mini-ME PC results: 83 correct, 58 unsupported language, 2 timeout (1h) Performance comparison on PCs with: FaCT++, HermiT, Pellet Mini-ME is competitive for small-medium ontologies (<1200 classes) 18 of 32
Experiments: Ontology Classification Correctness of implementation and comparison with state-of-the-art matchmakers Reference dataset: 143 ontologies Mini-ME PC results: 83 correct, 58 unsupported language, 2 timeout (1h) Performance comparison on PCs with: FaCT++, HermiT, Pellet Mini-ME is competitive for small-medium ontologies (<1200 classes) 19 of 32
Agent Framework [Ruta et al., AAMAS-ATES, 2012] exposes user needs and preferences acts as a mediator in a negotiation round supports KNX enhancements in case of legacy appliances sends semantic-based requests to the mediator agent for negotiating a home profile 20 of 32
Semantic-enhanced interaction Request originated by a mobile agent 21 of 32
Logic-based Negotiation Protocol Integration of knowledge representation and reasoning techniques thought for the Semantic Web Ontology Languages (OWL, DIG, RDF) Inference Services Semantic Matchmaking Theoretical framework based on Description Logics (DLs) Negotiation protocol [Ragone et al., EC-Web, 2009], originally devised for e-marketplace scenarios, revised for building energy systems multi-issue incomplete information rational agents The Home Agent (mediator) splits one-to-many negotiation (requester agent VS device agents) in several concurrent one-to-one negotiations 22 of 32
Negotiation Process The negotiation follows an alternating offers pattern with minimum concession the goal is to remove conflicting preferences between D and S i agents take turns in making concessions (requester moves first) in each round, agent drops the conflicting preference with minimal utility Utility function: u β (A) = i { u β (P k ) A P k } Agents utilities are normalized to 1 to eliminate outliers and make them comparable i u β (P k ) = 1 u β (P k ) 0 The process ends with: a conflict deal, the global utility of an agent is lower than its disagreement threshold an agreement, there is nothing more to negotiate on and the global utility of each agent is greater than its disagreement threshold 23 of 32
Case Study: Solve Concept Covering Problem REQUEST + CONTEXT ON OFF Request Covering algorithm 1. Find incompatible active services 2. Is the request already completely covered? (Concept Abduction) [Ruta et. al., WIAS, 2011] 3. Skip inactive services contrasting with currently active ones 4. The negotiation protocol via Concept Covering [Ragone et. al., JWSR, 2007] allows to select one or more inactive functionalities, whose combination covers request features The algorithm returns: the set of services to be activated; the (possibly empty) set of ones to be disabled; a description of the uncovered request, if present. 24 of 32
Case Study: Negotiation Example (1/2) D : semantic annotated user profile S : semantic description of services D: User Request i β i u(β i ) 1 issuggestedforsensation.cold 0.6 2 = 0 available kwh 0.2 3 = 10 outside Temperature 0.2 t β 0.8 u 1 : 0.8 * 1 = 0.8 S 1 : Heat Pump i σ 1,i u(σ 1,i ) 1 issuggestedforsensation.cold 0.5 2 = 0 available kwh 0.1 3 12 outside Temperature 0.2 4 8 outside Temperature 0.2 t S1 0.6 D: User Request S 2 : Heater at half power i β i u(β i ) i σ 2,i u(σ 2,i ) 1 issuggestedforsensation.cold 0.6 1 issuggestedforsensation.cold 0.4 2 = 0 available kwh 0.2 2 3 available kwh 0.3 3 = 10 outside Temperature 0.2 3 8 outside Temperature 0.3 t β 0.8 t S2 0.6 u 2 : 0.8 * 0.7 = 0.56 25 of 32
Case Study: Negotiation Example (2/2) D : semantic annotated user profile S : semantic description of services D: User Request i β i u(β i ) 1 issuggestedforsensation.cold 0.6 2 = 0 available kwh 0.2 3 = 10 outside Temperature 0.2 t β 0.8 u 3 : 0.8 * 0.8 = 0.64 S 3 : Heater at full power i σ 3,i u(σ 1,i ) 1 issuggestedforsensation.cold 0.6 2 6 available kwh 0.2 3 2 outside Temperature 0.2 t S3 0.6 Heat Pump Heater at half power Heater at full power Scenario #1 0.8 0.56 0.64 The Heat Pump is more efficient in case of an outdoor temperature of 10 and no stored energy. 26 of 32
Main contribution Distributed knowledge-based agent framework for HBA Semantic evolution of ISO/IEC 14543-3 EIB/KNX protocol Effective prototypical mobile matchmaker for pervasive computing contexts Semantic-enhanced resource discovery, composition and negotiation in HBA 27 of 32
Future work directions Integrate to the general framework: additional domotic protocols (Lonworks, ZigBee) Semantic Sensor Networks (SSN) based on Constrained Application Protocol (CoAP) protocol [Ruta et al., ISWC-SSN, 2012] Improve the automatic user profiling module Extend Concept Abduction and Contraction algorithms to more expressive logic languages (ԐL/ԐL ++ DLs) Support of OWLlink protocol for interfacing the matchmaker with general purpose Semantic Web applications Extend the framework toward a Smart Grid vision with Home-to-Home negotiation Large-scale simulation campaign to fully evaluate the approach within a Neighborhood Area Network (NAN) Mining of data about electric energy consumption for energy-based home and building profiling (progetto ResNovae Reti, Edifici, Strade - Nuovi Obiettivi Virtuosi per l Ambiente e l Energia PONREC 2007/2013, d.d. 84/Ric.) 28 of 32
Publications List (1/3) Papers in international journals 1. M. Ruta, F. Scioscia, E. Di Sciascio, G. Loseto. Semantic-based Enhancement of ISO/IEC 14543-3 EIB/KNX Standard for Building Automation. In IEEE Transactions on Industrial Informatics, Volume 7, Number 4, page 731-739 2011. 2. M. Ruta, F. Scioscia, E. Di Sciascio, G. Loseto. Semantic-based resource discovery and orchestration in Home and Building Automation: a multi-agent approach. In IEEE Transactions on Industrial Informatics, 2013 to appear. Papers in proceedings of international conferences 3. M. Ruta, F. Scioscia, E. Di Sciascio, F. Gramegna, S. Ieva, G. Loseto. RFID-assisted product delivery in sustainable supply chains: a knowledge-based approach. In the 4th International Workshop on RFID Technology Concepts, Applications, Challenges (IWRT 2010), page 3-15 June 2010. 4. M. Ruta, F. Scioscia, E. Di Sciascio, G. Loseto. A semantic-based evolution of EIB Konnex protocol standard. In IEEE International Conference on Mechatronics (ICM 2011), page 773-778 April 2011. 29 of 32
Publications List (2/3) 5. G. Loseto. A Semantic-based Pervasive Computing Approach for Smart Building Automation. In 8th Extended Semantic Web Conference (ESWC 2011) June 2011. 6. M. Ruta, F. Scioscia, G. Loseto, E. Di Sciascio. An Agent Framework for Knowledgebased Homes. In 3rd International Workshop on Agent Technologies for Energy Systems (ATES 2012), page 27-34 June 2012. 7. M. Ruta, F. Scioscia, E. Di Sciascio, F. Gramegna, G. Loseto. Mini-ME: the Mini Matchmaking Engine. In OWL Reasoner Evaluation Workshop (ORE 2012), Volume 858, page 52-63 July 2012. 8. M. Ruta, F. Scioscia, S. Ieva, G. Loseto, E. Di Sciascio. Semantic Annotation of OpenStreetMap Points of Interest for Mobile Discovery and Navigation. In 1st IEEE International Conference on Mobile Services (MS 2012), page 33-39 July 9. M. Ruta, F. Scioscia, G. Loseto, F. Gramegna, E. Di Sciascio. A Mobile Reasoner for Semantic-based Matchmaking. In the 6th International Conference on Web Reasoning and Rule Systems (RR 2012), Volume 7497, page 254-257 September 2012. 30 of 32
Publications List (3/3) 10. M. Ruta, F. Scioscia, G. Loseto, F. Gramegna, A. Pinto, S. Ieva, E. Di Sciascio. Alogicbased CoAP extension for resource discovery in semantic sensor networks. In the 5th International Workshop on Semantic Sensor Networks (SSN 2012), Volume 904, page 17-32 November 2012. Papers in proceedings of national conferences 11. G. Loseto, M. Ruta. Knowledge Representation Methods for Smart Devices in Intelligent Buildings. In Proceedings of AI*IA 2012 Doctoral Consortium, Volume 926, page 18 22 - June 2012. 12. M. Ruta, F. Scioscia, F. Gramegna, G. Loseto, E. Di Sciascio. Knowledge-based Real- Time Car Monitoring and Driving Assistance. In 20th Italian Symposium on Advanced Databases Systems (SEBD 2012), page 289-294 June 2012. 13. G. Loseto, F. Scioscia, M. Ruta, E. Di Sciascio. Semantic-based Smart Homes: a Multi- Agent Approach. In 13th Workshop on Objects and Agents (WOA 2012), Volume 892, page 49-55 September 2012. 31 of 32
Awards ESWC 2011 Best Poster Award - A Semantic-based Pervasive Computing Approach for Smart Building Automation KNX Award 2012 Nominee Progetto Semantic Web of Things at Home with KNX" realizzato da Michele Ruta, Eugenio Di Sciascio, Floriano Scioscia e 32 of 32