Linked Data Generation

Transcription

1 Linked Data Generation Oscar Corcho, Asunción Gómez Pérez ({ocorcho, Universidad Politécnica de Madrid Credits: Raúl García Castro, Guillermo Alvaro, Oscar Muñoz, Jose Angel Ramos Gargantilla, María del Carmen Suárez de Figueroa, Boris Villazón, Alex de León, Víctor Saquicela, Luis Vilches, Miguel Angel García, Manuel Salvadores, Juan Sequeda, Carlos Ruiz Moreno and many others Work distributed under the license Creative Commons Attribution-Noncommercial-Share Alike 3.0 Introduction to Linked Data Linked Data publication - Methodological guidelines for Linked Data publication - RDB2RDF tools - Technical aspects of Linked Data publication [Linked Data consumption] Contents 2 1

2 What is the Web of Linked Data? An extension of the current Web - where information data and services are given well-defined and explicitly represented meaning, - so that t it can be shared and used by humans and machines, better enabling them to work in cooperation How? - Promoting information exchange by tagging web content with machine processable descriptions of its meaning. - And technologies and infrastructure to do this - And clear principles on how to publish data What is Linked Data? Linked Data is a term used to describe a recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF. - Part of the Semantic Web - Exposing, sharing and connecting data - Technologies: URIs and RDF (although others are also important) t) 2

3 The four principles (Tim Berners Lee, 2006) 1. Use URIs as names for things 2. Use HTTP URIs so that people can look up those names. 3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL) 4. Include links to other URIs, so that they can discover more things. esignissues/linked Data.html 5 Linked Open Data evolution

4 Linked Open Data Evolution 7 LOD Cloud May 2007 Facts: Focal points: DBPedia: RDFized vesion of Wikipiedia; many ingoing and outgoing links Music-related datasets Big datasets include FOAF, US Census data Size approx. 1 billion triples, 250k links Figure from [4] 8 4

5 LOD Cloud September 2008 Facts: More than 35 datasets interlinked Commercial players joined the cloud, e.g., BBC Companies began to publish and host dataset, e.g. OpenLink, Talis, or Garlik. Size approx. 2 billion triples, 3 million links Figure from [4] 9 LOD Cloud March 2009 Facts: Big part from Linking Open Drug cloud and the BIO2RDF project (bottom) Notable new datasets: Freebase, OpenCalais, ACM/IEEE Size > 10 billion triples Figure from [4] 10 5

6 LOD clouds How should we publish data? Formats in which data is published nowadays - XML - HTML - DBs - APIs - CSV - XLS - However, main limitations from a Web of Data point of view - Difficult to integrate - Data is not linked to each other, as it happens with Web documents. 6

7 How do we publish Linked Data? 1. Exposing Relational Databases or other similar formats into Linked Data - D2R - Triplify - R2O - NOR2O - Virtuoso - Ultrawrap - 2. Using native RDF triplestores - Sesame - Jena - Owlim - Talis platform - 3. Incorporating it in the form of RDFa in CMSs like Drupal 13 How do we consume Linked Data? Linked Data browsers - To explore things and datasets and to navigate between them. - Tabulator Browser (MIT, USA), Marbles (FU Berlin, DE), OpenLink RDF Browser (OpenLink, UK), Zitgist RDF Browser (Zitgist, USA), Disco Hyperdata Browser (FU Berlin, DE), Fenfire (DERI, Ireland) Linked Data mashups - Sites that mash up (thus combine Linked data) - Revyu.com (KMI, UK), DBtune Slashfacet (Queen Mary, UK), DBPedia Mobile (FU Berlin, DE), Semantic Web Pipes (DERI, Ireland) Search engines - To search for Linked Data. - Falcons (IWS, China), Sindice (DERI, Ireland), MicroSearch (Yahoo, Spain), Watson (Open University, UK), SWSE (DERI, Ireland), Swoogle (UMBC, USA) Listing on this slide by T. Heath, M. Hausenblas, C. Bizer, R. Cyganiak, O. Hartig 14 7

8 Linked Data browsers (Disco) Linked Data Mashup (LinkedGeoData) Mi 8

9 Linked Data Mashup (DBpedia Mobile) Migraci Linked Data Search Engines (Sindice and SIG.MA) Entity lookup service. Find a document that mentions a URI or a keyword. 9

10 Linked Data Search Engines (NYT) The New York Times: Alumni In The News - Linked Data Search Engines (NYT) The New York Times: Source code is available and is based on SPARQL queries 10

11 One additional motivation: Open Government Government and state administration should be opened at all levels to effective public scrutiny and oversight Objectives: - Transparency - Participation - Collaboration - Inclusion Cost reduction - Interoperability - Reusability Leadership - Market & Value Some Links: B. Obama Transparency and Open Government T. Berners-Lee - Raw data now! J. Manuel Alonso - Qué es Open Data? Open Government Data 8 Principles of Open Government Data 21 Open Government. USA and UK 22 11

12 Linked Data Mashup (data.gov) Clean Air Status and Trends (CASTNET) - Linked Data in the UK Education - Parliament - t i /id/ /1227 Maps - E.g., London: Transport - p SameAs service - Challenges

13 Research Funding Explorer - Linked Data Mashup (data.gov.uk) Open Government Spain. Euskadi 26 13

14 Open Government Spain. Abredatos 27 Open Government Spain. Zaragoza 28 14

15 Open Government Spain. Asturias 29 Linked Data Mashup (Water quality) Water quality in Asturias beaches

16 Introduction to Linked Data Linked Data publication - Methodological guidelines for Linked Data publication - RDB2RDF tools - Technical aspects of Linked Data publication [Linked Data consumption] Contents 31 Methodological guidelines for Linked Data publication Motivation Related Work GeoLinkedData Identification of the data sources Vocabulary Development Generation of the RDF data Publication of the RDF data Data cleansing Linking the RDF data Enable effective discovery Future Work 16

17 GeoLinkedData It is an open initiative whose aim is to enrich the Web of Data with Spanish geospatial data. This initiative has started off by publishing diverse information sources, such as National Geographic Institute of Spain (IGN-E) and National Statistics Institute (INE) Motivation The Web of Data is mainly for English speakers Poor presence of Spanish» % English» % Spanish Source:Billion Triples dataset at Thanks to Aidan and Richard 17

18 Related Work Impact of Geo.linkeddata.es Número de tripletas en Español (July): Número de tripletas en Español (End august): Before geo.linkeddata.es en 99, ja 0, fr 0, de 0, pl 0, it 0, es 0, After geo.linkeddata.es en 94, es 5, ja 0, fr 0, de 0, pl 0, it 0, Asunción Gómez Pérez 36 18

19 Process for Publishing Linked Data on the Web Identification of the data sources Vocabulary development Generation of the RDF Data Publication of the RDF data Data cleansing Linking the RDF data Enable effective discovery 1. Identification and selection of the data sources Identification of the data sources Vocabulary development Instituto Geográfico Nacional Generation of the RDF Data Publication of the RDF data Data cleansing Linking the RDF data Instituto Nacional de Estadística Enable effective discovery 19

20 1. Identification and selection of the data sources Instituto Geográfico Nacional (Geographic Spanish Institute) - Multilingual (Spanish, Vasc, Gallician, Catalan) - Conceptualization mistmatches - Granularity (scale concept) - Textual information - Particularaties Longitude latitude Instituto Nacional de Estadística (Statistic Spanish Institute) Monolingual Numerical information Particularaties Geo (textual level) Temporal Asunción Gómez Pérez Identification and selection of the data sources IGN-E 20

21 1. Identification and selection of the data sources Year Industry Production Index Province 2. Vocabulary development Identification of the data sources Vocabulary development Generation of the RDF Data Publication of the RDF data Data cleansing Linking the RDF data Enable effective discovery 21

22 2. Vocabulary development Identification of the data sources Vocabulary development Generation of the RDF Data Publication of the RDF data Data cleansing Linking the RDF data Enable effective discovery Features - Lightweight : Taxonomies and a few properties - Consensuated vocabularies To avoid the mapping problems - Multilingual Linked data are multilingual The NeOn methodology can help to - Re-enginer Non ontological resources into ontologie Pros: use domain terminology already consensuated by domain experts - Withdraw in heavyweight ontologies those features that you don t need - Reuse existing vocabularies Asunción Gómez Pérez 43 Knowledge Resources Non Ontological Resources Glossaries Classification Schemas Dictionaries Taxonomies 2 Lexicons Thesauri Ontological Resources O. Design Patterns O. Repositories and Registries Flogic RDF(S) OWL Ontological Resource Reuse Non Ontological Resource Reuse Ontology Design Pattern Reuse O. Aligning O. Merging 2 Ontological Resource 7 Reengineering Non Ontological Resource Reengineering Alignments 1 O. Specification Scheduling O. Conceptualization O. Formalization O. Implementation 9 O. Localization 8 Ontology Restructuring (Pruning, Extension, Specialization, Modularization) RDF(S) OWL Flogic 44 Ontology Support Activities: Knowledge Acquisition (Elicitation); Documentation; Configuration Management; Evaluation (V&V); Assessment 1,2,3,4,5,6,7,8, 9 22

23 Vocabulary development: Specification Content requirements: Identify the set of questions that the ontology should answer - Which one are the provinces in Spain? - Where are the beaches? - Where are the reservoirs? - Identify the production index in Madrid - Which one is the city with higher production index? - Give me Madrid latitude and altitude -. Non-content requirements - The ontology must be in the four official Spanish languages Asunción Gómez Pérez Lightweight Ontology Development hydrographic al phenomena (rivers, lakes, etc.) WGS84 Geo Positioning: an RDF vocabulary scv:dimensio n scv:item scv:dataset Vocabulary for instants, intervals, durations, etc. Classes 33 Ontology for OGC Geography Markup Language 33 Object Properties Data Properties Names and international code systems for territories and groups reused Following the INSPIRE (INfrastructure for SPatial InfoRmation in Europe) recommendation. hydrontology,scovo, FAO Geopolitcal, WGS84, GML, and Time 23

24 Contexto Directiva INSPIRE Objetivos: - INSPIRE intenta conseguir fuentes armonizadas de Información Geográfica para dar soporte alaformulación, implementación y evaluación de políticas comunitarias (Medio Ambiente, etc). - Fuentes de Información Geográfica: Bases de datos de los Estados Miembros (UE) a nivel local, regional, nacional e internacional. Luis Manuel Vilches Blázquez INSPIRE - Anexos Luis Manuel Vilches Blázquez 24

25 hydrontology Existencia de gran diversidad de problemas (múltiples fuentes, heterogeneidad de contenido y estructuración, ambigüedad del lenguaje natural, etc.) en la información geográfica. Necesidad de un modelo compartido para solventar los problemas de armonización y estructuración de la información hidrográfica. hydrontology es una ontología global de dominio desarrollada conforme a un acercamiento top-down. - Recubrir la mayoría de los fenómenos representables cartográficamente asociados al dominio hidrográfico. - Servir como marco de armonización entre los diferentes productores de información geo-espacial en el entorno nacional e internacional. - Comenzar con los pasos necesarios para obtener una mejor organización y gestión de la información geográfica (hidrográfica). Luis Manuel Vilches Blázquez Fuentes Catálogos de fenómenos Tesauros y Bibliografía BCN25 EGM & ERM BCN200 GEMET CC.AA. Nomenclátor Geográfico Nacional Nomenclátor Conciso FTT ADL Getty WFD Diccionarios y Monografías Luis Manuel Vilches Blázquez 25

26 Criterios de estructuración Directiva Marco del Agua - Propuesta por Parlamento y Consejo de la UE - Lista de definiciones de fenómenos hidrográficos Proyecto SDIGER - Proyecto piloto INSPIRE - Dos cuencas, países e idiomas Criterios semánticos - Diccionarios geográficos Diccionario de la Real Academia de la Lengua WordNet Wikipedia Bibli Bibliografía fí d de varias i á áreas d de conocimiento i i t Herencia: Estructuración actual de catálogos Asesoramiento expertos en toponimia del IGN Luis Manuel Vilches Blázquez Modelización del dominio hidrográfico Luis Manuel Vilches Blázquez 26

27 Implementación & Formalizacón + Pellet conceptos (classes), 47 tipos de relaciones (properties) y 64 tipos de atributos (attribute types) Luis Manuel Vilches Blázquez 2. Vocabulary development: HydrOntology Asunción Gómez Pérez 54 27

28 3. Generation of RDF Identification of the data sources Vocabulary development Generation of the RDF Data Publication of the RDF data Data cleansing Linking the RDF data Enable effective discovery From the Data sources - Geographic information (Databases) - Statistic information (.xsl) - Geospatial information Different technologies for RDF generation - Reengineering patterns - R20 and ODEMapster - Annotation tools - Geometry generation 3. Generation of the RDF Data NOR 2 O INE ODEMapster IGN Geospatial column Geometry2RDF IGN 28

29 3. Generation of the RDF Data / instances Classification schemes Thesauri NOR 2 O Lexicons NOR2O is a software library that implements the transformations proposed by the Patterns for Re-engineering Non-Ontological Resources (PR-NOR). Currently we have 16 PR-NORs. PR-NORs define a procedure that transforms a Non-Ontological Resource (NOR) components into ontology elements. NOR 2 O FAO Water classification Classification scheme Path enumeration data model Implemented in a database Re-engineering Model for NORs Patterns for Re-engineering Non-Ontological Resources (PR-NOR) NOR Reverse Engineering Conceptual Requirements Transformation Specification Conceptualization Ontology Forward Engineering Design Formalization Implementation Implementation RDF(S) Non-Ontological Resource Ontology 29

30 Technological support PR-NOR library at the ODP Portal Industry Production Index 3. Generation of the RDF Data NOR2O Year Province NOR 2 O 30

31 hydrontology & Bases de Datos NGN 1: multilingüe Migración de datos a la Web de los Datos - Enfoques, técnicas y 3. Generation of the RDF Data R2O & ODEMapster Creation of the R2O Mappings 31

32 3. Generation of the RDF Data Geometry2RDF Oracle STO UTIL package SELECT TO_CHAR(SDO_UTIL.TO_GML311GEOMETRY(geometry)) AS Gml311Geometry FROM "BCN200"."BCN200_0301L_RIO" c WHERE c.etiqueta='arroyo' 3. Generation of the RDF Data Geometry2RDF 32

33 3. Generation of the RDF Data Geometry2RDF 3. Generation of the RDF data RDF graphs IGN INE BTN25 BCN200. IPI es/dataset/ign/btn25 es/dataset/ign/bcn200 es/dataset/ine/ipi So far 7 RDF Named Graphs triples 33

34 4. Publication of the RDF Data Identification of the data sources Vocabulary development HTML Linked Data SPARQL Generation of the RDF Data Publication of the RDF data Including Provenance Support Pubby Data cleansing Pubby 0.3 Linking the RDF data Enable effective discovery Virtuoso Publication of the RDF Data 34

35 4. Publication of the RDF Data - License License for GeoLinkedData - Creative Commons Attribution-ShareAlike GNU Free Documentation License Each dataset will have its own specific license, IGN, INE, etc. 5. Data cleansing Identification of the data sources Vocabulary development Generation of the RDF Data Publication of the RDF data Data cleansing Linking the RDF data Enable effective discovery Lack of documentation of the IGN datasets Broken links: Spain, IGN resources Lack of documentation of the ontology Missing english and spanish labels Building a spanish ontology and importing some concepts of other ontology (in English): - Importing the English ontology. Add annotations like a Spanish label to them. - Importing the English ontology, creating new concepts and properties with a Spanish name and map those to the English equivalents. - Re-declaring the terms of the English ontology that we need (using the same URI as in the English ontology), and adding a Spanish label. - Creating your own class and properties that model the same things as the English ontology. 35

36 URIs in Spanish - RDF allows UTF-8 characters for URIs - But, Linked Data URIs has to be URLs as well - So, non ASCII-US characters have to be %code 5. Data cleansing 6. Linking of the RDF Data Identification of the data sources Vocabulary development Silk - A Link Discovery Framework for the Web of Data Generation of the RDF Data Publication of the RDF data DBPedia GeoLinkedData Geonames Data cleansing Linking the RDF data Enable effective discovery First set of links: Provinces of Spain - 86% accuracy 36

37 6. Linking of the RDF Data Asunción Gómez Pérez Enable effective discovery Identification of the data sources Vocabulary development Generation of the RDF Data Publication of the RDF data Data cleansing Linking the RDF data Enable effective discovery 37

38 DEMO Provinces 38

39 Industry Production Index Capital of Province Rivers 39

40 Beaches Generate more datasets from other domains, e.g. universities in Spain. Future Work Identify more links to DBPedia and Geonames. Cover complex geometrical information, i.e. not only Point and LineString-like data; we will also treat information representation through polygons. 40

41 Contents Introduction to Linked Data Linked Data publication - Methodological guidelines for Linked Data publication - RDB2RDF tools - Technical aspects of Linked Data publication [Linked Data consumption] 81 Using an RDF repository It allows storing and accessing RDF data For example, SESAME ( Download it from - openrdf-sesame sdk.zip - Deploy the.war in Tomcat (JDK and Tomcat needed) Create a repository at - Check:

42 Linked Data frontend To expose data as Linked Data - Including content negotiation, etc. For example, Pubby - Installation - Use pubby-0.3.zip - Deploy the webapp folder (and rename) in Tomcat - Modify config.n3 - Restart tomcat - Check: LD access LD Frontend (Pubby) RDF repository (Sesame) SPARQL query REST API For example, RDF2Go - Java abstraction over RDF repositories

43 For example, SNORQL ( Add SPARQL explorer Validating Linked Data URLs 86 43

44 Introduction to Linked Data Linked Data publication - Methodological guidelines for Linked Data publication - RDB2RDF tools - Technical aspects of Linked Data publication [Linked Data consumption] Contents 87 RelFinder: finding relations in Linked Data E.g., relations between films - Pulp Fiction, Kill Bill y Reservoir Dogs 44

45 Exercise on data.gov.uk - Public schools in London that contain the word music Exercise: find information in DBPedia Image by Find ficticious serial killers in DBPedia (etc) 45

46 Designing URI sets for the Public Sector (UK) _sector_uri.pdf 91 Asociación Española de Linked Data 92 46

47 Linked Data Generation Oscar Corcho, Asunción Gómez Pérez ({ocorcho, Universidad Politécnica de Madrid Credits: Raúl García Castro, Guillermo Alvaro, Oscar Muñoz, Jose Angel Ramos Gargantilla, María del Carmen Suárez de Figueroa, Boris Villazón, Alex de León, Víctor Saquicela, Luis Vilches, Miguel Angel García, Manuel Salvadores, Juan Sequeda, Carlos Ruiz Moreno and many others Work distributed under the license Creative Commons Attribution-Noncommercial-Share Alike