The CHARMe Project: Commentry Metadata for EO Dataset Characterization of metadata to enable high-quality climate applications and services

Transcription

1 The CHARMe Project: Commentry Metadata for EO Dataset Characterization of metadata to enable high-quality climate applications and services Maurizio Nagni (on behalf of the CHARMe consortium) Science and Technology Facility Council

2 Why are climate data important? Reinsurance Are we meeting our greenhouse emissions targets? How will our global exposure to risk change over the next 30 years? Humanitarian aid Which regions of the world will be water-stressed in 2030? Government Will the storm risk to offshore assets increase? Oil and gas

3 Step 1: Defining the problem How can EO data 1 users decide whether a resource 2 is fit for their purpose 3? 1 Not only 2 Dataset, paper, metadata, video, news... 3 We consider that data quality and fitness for purpose are the same thing

4 Step 1: Defining the problem Domain analysis Scientific literature Huge, verbose, inaccessible to some communities, not well linked to source data Technical reports and conference proceedings Hard to find, scattered or inaccessible Data centres providing more and more metadata, but don t usually include community feedback, not all countries and communities have data centres! Websites and blogs From CEOS Handbook to a scientist s blog Increasingly useful, but scattered User knows: intrinsic information (spatial, temporal) data provenance (which process generated the dataset) how public used the dataset?

5 Step 2: Toward a solution The semantic web vision (or why we need the Linked Data) Query: isaboutclimatechange(?x, Tokio) paper Huey contains soil moisture of C dataset Dewey contains NDVI of C dataset Louie contains temperatures of C contains [photo level temperature] of... inductive reasoning (rule learning) DataWow.org X contains temperatures of C X is about climate change IKnow.gov isaboutclimatechange(godzilla, Tokio) deductive reasoning (logical inference) isaboutclimatechange(louie, Duckburg) isaboutclimatechange(nemo, Pacific Ocean) isaboutclimatechange(asterix, Brittany).. ClimateNow.com

6 Step 2: Toward a solution Harmonizing the sources RDF allows subject-predicate-object expressions X isaboutclimatechange Resource (with URI) Papers, metadata, datasets may have DOI or URL Y What about commentary metadata? Post-fact annotation: citations, notes Tsunami at Friday 11 March 2011, Contradict paper Z Result of assessment: validations campaigns, reanalysis Corrects observation from Mike, I like the result! Provenance: dependency on other datasets, processing algorithm chain Property of data distribution: data policy, licensing, reliability External events that may affect the data: volcanic eruption, instrument failure General rule: information which originate from users or external entities, not the original data providers (sometime not even from the data provider!)

7 Step 2: Toward a solution Chasing the commentary metadata with Open Annotation Annotating, the act of creating associations between distinct pieces of information, is a pervasive activity online in many guises (from Open Annotation data model) <anno1> a oa:annotation ; oa:hasbody <body1> ; oa:hastarget <OceanSurfaceDataset_X>. <body1> cnt:contentastext ; cnt:chars Tsumani at Friday 11 March 2011

8 Step 3: Implementing a solution The IT tools CHARMe as Service: information to appear where users are already looking Data provider Data website provider website 3 rd party system OAuth RESTfull interface CHARMe server Apache Jena Lucene CHARM e node CHARMe node CHARM e node CHARMe node CHARMe as OpenSource system to record Commentary metadata CHARM CHARMe e node node

9

10 What CHARMe will not enable Give me the best dataset on sea surface temperature The best dataset depends on the application CHARMe will not provide a new quality stamp for dataset But will be able to link to such things if other people publish them, e.g. Bates maturity index, QA4EO certification CHARMe will not provide access to actual data But will enable discovery of data Not planning to create (another) one-stop shop for information We want the information to appear where users are already looking

11 Step 4: What the users may expect Beware of 5-star ratings (words of wisdom from xkcd.com) YouTube 5-star rating stats in 2009

12 More advanced uses of CHARMe Viewing significant events in timeseries data (instrument failures, volcanic eruptions) (cf. Google Finance) Creating and discovering annotations about dataset subsets (cf. maphub.github.io) Enabling intercomparisons of data and metadata (cf. ES-DOC)

13 Thank you!