DataBridges: data integration for digital cities

Transcription

1 DataBridges: data integration for digital cities Thematic action line «Digital Cities» Ioana Manolescu Oak team INRIA Saclay and Univ. Paris Sud-XI

2 Plan 1. DataBridges short history and overview 2. RDF for sharing: Linked and/or Open Data 3. Areas of work and main results 4. Scientific zooms 5. Conclusion Ioana Manolescu, Digital Cities,

3 1 DataBridges short history and overview Ioana Manolescu, Digital Cities,

4 Activity overview Started in Partners as of October 2010: Inria, Paris XI, DFKI, Nokia Partners as of March 2012: Inria, Paris XI, DFKI + start-up, Alcatel/Bell Labs, TU Delft, Aalto, KTH, DataPublica Main goal: facilitating the flow of data in digital cities 1. Simplify the production, gathering and sharing of the data 2. Improve data dissemination through personalization 3. Solve challenges raised by large-scale data sharing Ioana Manolescu, Digital Cities,

5 Data flows in digital cities Information providers Individuals Data sharing Companies Administrations Ioana Manolescu, Digital Cities,

6 DataBridges vision for the data sharing marketplace Information providers Individuals 1. Make data easy to share 2. Heterogeneous selfdescribing 3. W3C standard RDF usable through multiple platforms and tools 4. Personalize: build and handle profiles 5. Scale Companies Administrations Ioana Manolescu, Digital Cities,

7 2 RDF for sharing: Linked and/or Open Data Ioana Manolescu, Digital Cities,

8 Linked vs. Open Data 1. Linked Data: «recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF» (Tim Berners-Lee) vision for the Web 2. Open Data: «idea that certain data should be freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control» In principle, orthogonal to the Linked aspect In practice, Linked a means toward Open Ioana Manolescu, Digital Cities,

9 Linked Open data cloud (09/11) Ioana Manolescu, Digital Cities, Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch.

10 Some open data from Etalab (FR) GDP per French region (Le Journal Du Net, 07/09/2011) Ioana Manolescu, Digital Cities,

11 Some open data from Etalab (FR) Drawing it is just the last step! Organic agriculture per French region (Le Journal Du Net, 07/09/2011) Ioana Manolescu, Digital Cities,

12 3 Main work areas and results Ioana Manolescu, Digital Cities,

13 Work distribution (1/2) 1. Make data easy to share 2. Heterogeneous self-describing 3. W3C standard RDF usable through multiple platforms and tools Aalto and KTH: 1. Identifying best practices and tools for open data business workshops. 2. Identifying European datasets with business potential. 3. Outlining possibilities to use format in education and entrepreneurship support. Paris XI: Augmenting public data with semantics for semantic integration. DFKI: Preparing access to Linked Open Data for personalized interactive maps Ioana Manolescu, Digital Cities,

14 Work distribution (2/2) 1. Make data easy to share Paris XI/INRIA: Use semantics to convert tables to LOD 2. Personalize: build and handle profiles Delft and Alcatel: Augmenting social data with semantics for user modeling and personalization. 3. Scale Paris XI/INRIA: Identifying links across RDF data sets (entity resolution) INRIA/Paris XI: Scalable storage for RDF data TU Delft: Efficient querying of distributed RDF data sources Ioana Manolescu, Digital Cities,

15 Main results 1. International publications (ISWC, ESWC, PVLDB, ) 2. Master thesis, PhD student involvement 3. Enriched open data about digital cities: Paris Sud/Inria 4. Green hackathon in Stockholm: develop applications and tools related to sustainability and green economy data 5. Joint generic architecture for data exchange in digital cities 6. Two integrated prototpyes (demos this afternoon): Paris Sud/Inria + DFKI: facetted browsing on digital cities open data Inria/Paris Sud + TU Delft: efficient workflows on open (RDF) data Ioana Manolescu, Digital Cities,

16 4 Selected technical zooms Ioana Manolescu, Digital Cities,

17 Zoom 1: querying large volumes or RDF data RDF: (subject, property, object) triples Simple Self-describing Irregular Semantic Web Challenge: billion triples. Query performance? Heavy indexing (triple table + 12 indexes ) Special column stores Remark: query workload often known Idea: use it to organize the store Ioana Manolescu, Digital Cities,

18 Efficient stores for RDF data workloads F.Goasdoué, K. Karanasos, J. Leblay and I. Manolescu. View Selection for Semantic Web Databases, Proceedings of VLDB, 2011 Ioana Manolescu, Digital Cities,

19 Efficient stores for RDF data workloads Cost reduction (%) achieved by our view search algorithm F.Goasdoué, K. Karanasos, J. Leblay and I. Manolescu. View Selection for Semantic Web Databases, Proceedings of VLDB, 2011 Ioana Manolescu, Digital Cities,

20 Zoom 2: TU Delft activities in 2011 Design and first iteration of the implementation of U-Sem framework for analyzing and modeling user and usage data Main focus on: Components for (1) enrichment, linkage and alignment of user and usage data and (2) analysis and user modeling. Main data sources: Twitter and Flickr. RDF Gears for orchestrating the functionality of the aforementioned components Scientific papers published at major conferences such as UMAP, ICWE, ESWC, ISWC Ioana Manolescu, Digital Cities,

21 user profiles (FOAF, Weighted Interests) U-Sem Framework SPARQL & U-Sem query extensions clients domain knowledge observations user characteristics Creating user modeling pipelines/workflows: Query data Endpoints aggregation (SPARQL) Understanding the U-Sem Application Transforming Logic user entity identification conflict resolution semantic meaning Interpreting (Authentication, of Access Control, data Plug-in Linking into management) user structures user/usage (semantically data accounts that and support (user) enriched) user profile data reasoning applications data profile fragments generation (e.g. user competence profiles) Storage Endpoints entity extraction (RDF) Semantic Enrichment, Linkage, and Alignment data aggregation ontology alignment topic detection Orchestration Logic (= RDF Gears) entity extraction entity identification Analysis and Schema User Modeling alignment and alignment of (user) data coming from different sources profile generation conflict resolution knowledge discovery rule-based reasoning temporal reasoning Endpoints enable U-Sem clients to utilize user modeling functionality Web U-Sem Data Repositories Other Repositories Ioana Manolescu, Digital Cities,

22 U-Sem Example: Twitter-based User Modeling (TweetUM) Key challenge: Inferring user profiles from Twitter activities that allow for personalization. Key building blocks: Semantic Enrichment: understanding the semantic meaning of Twitter activities Filtering: identify relevant tweets Topic modeling: deduce topics a user is concerned with Weighting strategies: infer level of interest and competence for a topic Applications: - recommender systems (tested for news, POIs) - personalized search (faceted search, Twitcident) user s Twitter activities Recommender Profile? user TweetUM U-Sem Topic service Weighting suite for Twitter-based strategies User Modeling Modeling Semantic Enrichment Profile topic weight FOAF-based profile (set of weighted topics) Filtering Ioana Manolescu, Digital Cities,

23 Zoom 3: Some open data comes in tables! Estimated population per French region, January 2011 Ioana Manolescu, Digital Cities,

24 From tables to linked data Population evolution in the area of Nord-Pas-de-Calais Nom de l'arrondissement Population au 1 er janvier 1999 Population au 1 er janvier 2008 Variation de population entre 1999 et 2008 Variation annuelle moyenne entre 1999 et 2008 Avesnes-sur-Helpe ,21 How do we get here? Cambrai ,06 Douai ,03 Dunkerque ,12 Lille ,16 Valenciennes ,01 Département du Nord ,05 Arras ,38 Béthune ,16 Boulogne-sur-Mer ,02 Calais ,01 Lens ,19 Montreuil ,60 Saint-Omer ,43 Département du Pas-de-Calais ,14 Région Nord-Pas-de-Calais ,08 Ioana Manolescu, Digital Cities,

25 Data typing approach DBPedia Geographic ontology data 1 Type named entities (DBPedia) types 2 Type pertinence for the application domain types 3 Type validation Titles (columns, tables) Types Confidence % C. Morel, G. Quercini, C. Reynaud, B. Safar Ioana Manolescu, Digital Cities,

26 5 Conclusion Ioana Manolescu, Digital Cities,

27 Data sharing will only increase As all things become digital: Your banking data Your pictures, photos, social life Your file system Your thoughts The city hall s: Nanny portal service (Paris 13e) Cultural service information Online public service booking (Palaiseau) Companies marketplaces to buy and sell We work on technologies and tools for personalized, extensible, expressive, efficient data sharing, mostly based on RDF and Semantic Web. Ioana Manolescu, Digital Cities,

28 Merci / questions?