PlanetData Showcases: Linked/Open/Big Data in Smart Cities Data Integration Andreas Harth (KIT), Oscar Corcho(UPM), Marko Grobelnik (JSI) 25 November 2013, Luxembourg
Motivation The relevance of many types of data perishes or degrades over time (e.g., weather information, moving objects) Timely decision making requires access to live data and inclusion of new sources in a flexible manner. Our goals (Near) real-time access to a variety of data sources in a range of data formats and access modalities Rapidly integrate sources via modeling and to generate a Linked Data interface to live source Flexibly adapt the system to new areas Slides adapted from presentation: Andreas Harth, Craig Knoblock, Steffen Stadtmüller, Rudi Studer and Pedro Szekely. "On-the-fly Integration of Static and Dynamic Linked Data". Fourth International Workshop on Consuming Linked Data (COLD 2013). Co-located with ISWC 2013, Sydney, Australia. Slide 2
Static vs. Dynamic Sources We look for a minimal model that supports Web Architecture Realtime data access Resource manipulation (read/write) Various sources have different update intervals (from minutes to weeks) We treat the access to all sources in the same way via polling (HTTP GETs) Thus, the only distinction between static and dynamic sources is how fast we refresh the query results for each source Slide 3
Scenario POIs (Crunchbase, OSM, Wikimapia) Venues, Events (Eventful, LastFM) Buses, Stops (LA Metro, Madrid Metro) Vehicles (Campus Cruisers) Marine Vessels (AIS) News (EventRegistry, Twitter) Slide 4
Architecture Slide 5
Modelling Sources with Karma Karma is a data integration tool Map data onto an ontology to generate Linked Data Karma extension to enable the on-the-fly lifting of API I/O data according to a pre-defined mapping model See http://isi.edu/integration/karma/ for more info. Slide 6
Linked Data Access to Event APIs LastFM API Given a lat/lon of a location, return a list of event identifiers http://km.aifb.kit.edu/services/lastfmwrap/geo.geteven ts?lat={?lat}&long={?lon} Given an event identifier, return details about the event http://lastfm.rdfize.com/events/{event-id} Eventful API List events given a keyword search term and a date range http://km.aifb.kit.edu/services/eventfulwrap/search?lo cation={?loc}&date={?date} Slide 7
LastFM Data-Fu Program (I) Program at http://km.aifb.kit.edu/services/datafu/lastfm with input lat/lon (in RDF via HTTP POST) Rule to search for events at given location: {?p geo:long?lon.?p geo:lat?lat. } => { [] http:mthd http:get ; http:requesturi <http://km.aifb.kit.edu/services/lastfm wrap/geo.getevents?lat={?lat}&long={?lo n}>. }. For the input point with lat/long perform an HTTP GET at the KIT LastFMWrapper URI constructed with the lat/long Slide 8
Demo Los Angeles http://people.aifb.kit.edu/aha/2013/d3/index.kml Location of buses and ships are updated Madrid http://people.aifb.kit.edu/aha/2013/d3/madrid.kml Next bus arrival times are updated Luxembourg http://people.aifb.kit.edu/aha/2013/d3/luxembourg.kml No live sources yet Slide 9
Conclusion System interoperation in distributed environments with Linked Data as interface Rapid integration of new sources (via Karma models and Data-Fu scripts) Realtime access to networked data via Data-Fu scripts/programs http://code.google.com/p/data-fu/ Ability to rapidy integrate new sources via Karma models http://www.isi.edu/integration/karma/ Future work SameAs handling Manipulating resource state (Read-Write Linked Data) Optimisations for limited bandwidth environments Slide 10
LastFM Data-Fu Program (II) Rule for retrieving information about the found events, including geolocation of event: {?e rdf:type lode:event. } => { [] http:mthd http:get ; http:requesturi?e. }. For every resource of type event perform an HTTP GET at the resource URI Query to return a table with lat/lon and label to transform to KML/Google Earth: :q1 qrl:select (?event?place?label?lat?lon ) ; qrl:where {?event <http://purl.org/net/c4dm/event.owl#place>?place.?event rdfs:label?label.?place geo:lat?lat.?place geo:long?lon. }. Output is every entity with latitude, longitude and associated label Slide 11