Transforming Data with FME for Inspire

Size: px

Start display at page:

Download "Transforming Data with FME for Inspire"

Edwina Holt
8 years ago
Views:

1 Transforming Data with FME for Inspire Ken European Services Manager Safe Software INSPIRE KEN & EuroSDR Schema Transformation Workshop ENSG - Marne-La-Vallée October 9 th 2013

2 Agenda Introduction Format Transformation Schema Mapping with FME Validation of Output Web Services with FME Server Looking Ahead FME & Inspire Q&A

3 Agenda Introduction Safe Software & FME Format Transformation New GML writing in FME Schema Mapping with FME Basic Schema related transformers Schema Mapper Success Stories Inspire Solution Pack Validation of Output XMLValidator transformer Other FME Validation Tools Web Services with FME Server WFS Service WxS Services with FME Looking Ahead FME & Inspire Q&A

Inspire Solution Pack Validation of Output XMLValidator transformer Other FME Validation

4 Safe Software

5 Safe Software Inc. ~100 exuberant employees Partners World-Wide Located in Surrey, British Columbia Canada Thousands of happy customers

6 FME Feature Manipulation Engine FME Desktop FME Server FME Cloud

7 Powering the flow of data Convert data from one format to another Transform data into the precise model you need Share data for use in other applications Integrate data to use in your preferred system Validate data to identify and fix quality issues 300+ supported formats

in other applications Integrate data to use in your preferred

8 FME Workbench Workspace Code No Code!! 400+ transformers give you unlimited flexibility

9 INSPIRE Data Harmonization Supported by FME Evaluation Data assessment INSPIRE Clients INSPIRE Reader Assembly Reading & Join Publication WxS,GML,PDF,KML Transformation schema, geometry Validation QA, XSD, values

10 Format Transformation

11 300+ Supported Formats Check for your format: safe.com/formats

12 New GML Writer in FME 2014 GML writing using application schema (XSDs) Destination data model captured directly from the application schema No Longer a Template Based Approach Import Feature Type in FME

captured directly from the application schema No

13 New GML Writing in FME 2014 Multiple Nested namespace support as defined by the application schema. For example in an INSPIRE watercourse element:

$C:\PS\InspireParis2013\CadastralParcels\1 INSPIREwriter_A1.CadastralParcelsStart.fmw Geometry Property Setters and Aggregates in INSPIREwriter_A1.$

14 New GML Writing We write GML Geometry directly from FME geometry just like any other format Multi-Geometry support with GeometryPropertySetter Transformer C:\PS\InspireParis2013\CadastralParcels\1 INSPIREwriter_A1.CadastralParcelsStart.fmw Geometry Property Setters and Aggregates in INSPIREwriter_A1.CadastralParcels.fmw

$C:\PS\InspireParis2013\CadastralParcels\1 INSPIREwriter_A1.CadastralParcelsStart.$

15 Schema Mapping with FME

16 INSPIRE - Challenge You want to meet INSPIRE data provision requirements, but your data is organized rather differently?

17 Schema Mapping Feature Type and Attribute Mapping Feature Type Mapping in FME Workbench Attribute Mapping in FME Workbench Data Transformation 17

18 Schema Mapping Values Mapping (domains) Value Mapping

19 Schema Mapping FME s SchemaMapper Transformer FME Workspace Name mapping Name & value mapping

20 Other types of transformations Geometric Transformations for example multiple named geometries

21 Other Types of Transformations Coordinate System Transformation Fast and easy projection conversion > 0 projections > 2000 predefined coordinate systems Support for user-defined systems (projection, datum, ellipsoid) Updated INSPIRE related coordinate system names to make the easier to find BritishNatGrid Belge72.Lambert72A EPSG references LL WGS-84 Italy-32 State plane and many more

22 Schema Mapping Demonstration AttributeCopiers, AttributeCreators, SchemaMapper, Reprojector GeometryPropertySetter in INSPIREwriter_A1.CadastralParcels.fmw Also show 3 Filters&Ranges&MappersExample.fmw

23 Some Schema Mapping Success Stories

24 INSPIRE SDIs: Implementations by Metria June 2013

25 Overview INSPIRE Projects in Sweden by Metria Swedish EPA: Protected Sites Harmonization Data integration from HelComm, Natura and EPA

26 Protected Sites Schema Mapping: NVR to INSPIRE FilterAttribute Filter Attribute Value SourceAttribute Field_Inspire DID INSPIRE_LOCALID VALID_FROM_DATE INSPIRE_ LEGALFOUNDATIONDAT E DECISIONDATE INSPIRE_DATE Destination AttributeValue Full SE DestinationAttribute INSPIRE_APPLICATIONSCHE MA INSPIRE_NAMESPACE creation INSPIRE_DATETYPE IUNC_CATEGORY 0 Empty value INSPIRE_DESIGNATION IUNC_CATEGORY Ia strictnaturereserve INSPIRE_DESIGNATION IUNC_CATEGORY Ib wildernessarea INSPIRE_DESIGNATION IUNC_CATEGORY II nationalpark INSPIRE_DESIGNATION IUNC_CATEGORY III naturalmonument INSPIRE_DESIGNATION IUNC_CATEGORY null Empty value INSPIRE_DESIGNATION OBJECTNAME INSPIRE_SITENAME PROTECTIONTYPE 1 natureconservation INSPIRE_PROTECTIONCLASS IFICATION PROTECTIONTYPE 2 natureconservation INSPIRE_PROTECTIONCLASS IFICATION

27 Swedish Protected Sites Update Swedish Environmental Protection Agency Production system for download services following on the successful pilot last year. Metria hosts the protected sites view services. Metria performs schema mapping for five protected sites source datasets to INSPIRE using FME Server.

31 con terra INSPIRE Solution Pack INSPIRE Data Harmonization INSPIRE Data Providing Consumer ETL European SDI Internal Information-/ Data Management System INSPIRE Data Models & Services

destination schema destination schema schema mapping Load

32 con terra - FME INSPIRE Solution Pack: Schema Mapping ETL import create import Workflow description source data destination schema destination schema schema mapping Load Data INSPIRE GDB FME data import process FME INSPIRE Solution Pack automated

33 Components of the FME INSPIRE Solution Pack Additional INSPIRE information Tutorial workspace (complete sample mapping [AdminUnits]) Additional HTML Workbench Help (description of INSPIRE GDB data model) Direct access to the INSPIRE data specification (link to specific themes) Additional functionality Template workspaces (destination schema and predefined workspaces) Destination data schema for all Annex 1 themes (INSPIRE GDB of ArcGIS for INSPIRE ) More than 100 additional INSPIRE transformers INSPIRE specific value and attribute mapping (voidable values) transformer for recurrent tasks (ID management, lifespan setter)

INSPIRE Transformers (Attribute and Value Mapping) Automated filling of obligated attribute legalstatus legalstatus_void Possible values for obligated attributes (if not void) agreed notagreed

34 INSPIRE Transformers (Attribute and Value Mapping) Automated filling of obligated attribute legalstatus legalstatus_void Possible values for obligated attributes (if not void) agreed notagreed Predefined void value reasons (if void) 0 = no reason given 1 = reason: unknown 2 = reason: unpopulated AttributeRenamer functionality usable if attribute already existing Show AdminUnitsISP.fmw

35 Validation of Output

36 Validation INSPIRE schema validation ( with.xsd) Data integrity Unique IDs Geometric integrity Null values (nullable?) Valid values: ranges and domain codes Bounds Custom validity rules specific to domain AttributeClassifer, Tester etc

37 Validation Workspace

38 Validation Report Bad Syntax

39 Validation Report Bad Schema

40 Validation Report Good Metadata Show Validator.fmw

41 Other Validation Options Tester Field names Data types (numeric, date etc) IDs CoordinateSystemExtractor GeometryFilter GeometryOGCvalidator SpatialFilter (check extents)

42 Updating: XMLUpdater Insert, Delete, Replace, Replace Contents

43 Web Services with FME Server

44 Web Service Request Response url request via get, post, put, delete Web Server Response JSON, XML All web services follow the same pattern

46 Power of FME is Transformation Build any web service! Code No Any service is now a workspace away. Code!! FME Data Streaming service delivers response.

47 FME Workspace as Web Service Broker for WFS GetCapabilities Request FME WFS Workspace Response XML FME Server

48 FME Workspace as Web Service Broker for WFS describefeaturetype Request FME WFS Workspace Response XML FME Server

49 FME Workspace as Web Service Broker for WFS getfeature Request FME WFS Workspace Response XML FME Server

50 FME Workspace as Web Service Broker for WFS GetCapabilities Request Response XML FME WFS Workspace GetFeature Request Response GML FME Server

51 WFS Workspace

52 Workspace Parameters from URL Service Request Version Feature Types Bounds

53 Messaging: GetCapabilites, DescribeFeature

54 Data Stream: GetFeature Response

55 WFS with FME Server Show: GeoNamesWFSgml13.fmw 1.amazonaws.com/fmeserver/repositories/Inspire/GeoNamesWFSgml13.fmw

56 Looking Ahead

57 Looking Ahead FME & INSPIRE Annex II and III New FME GML writer makes it easier to import new destination schemas for future themes support reading and writing of associated data types 3D: INSPIRE built in support for 3D, AIXM etc. tracking draft 3d schema development 3D reprojection (CSMapReprojector and gridshift)

58 Looking Ahead FME & INSPIRE Application Domain Extensions (ADE s) FME Supports Application Domain Extensions (ADE s) for CityGML (e.g. 3D IMGeo NL). INSPIRE ADE xsds for Annex III buildings without xml templates

59 Looking Ahead FME & INSPIRE WxS Webservice Framework to support any web service protocol (WFS, WPS, WMTS, SOS, ODATA, ) Real Time Services exploring how FME Server supports real time services for Annex III via our Notification Service Sensor support HTML Web Socket Server Mobile support: JSON, messaging protocols

60 INSPIRE Data Harmonization Supported by FME INSPIRE Reader Evaluation Data assessment INSPIRE SDI Data Sharing GML Writer Assembly Reading & Join Publication WxS,GML,PDF,KML Transformation schema, geometry Validation QA, XSD, values

61 Get Involved! youtube.com/fmechannel blog.safe.com

62 Q&A

63 Thank You! Ken New to FME? More Questions? Please see your local FME partner

Data Integration for ArcGIS Users Data Interoperability. Charmel Menzel, ESRI Don Murray, Safe Software

Data Integration for ArcGIS Users Data Interoperability Charmel Menzel, ESRI Don Murray, Safe Software Product overview Extension to ArcGIS (optional) Jointly developed with Safe Software Based on Feature