An Overview of Geospatial Standards Support within the ArcGIS Platform Satish Sankaran Marten Hogeweg
Agenda Interoperability Challenges Standards Organizations and Standards Standards Data formats and API, Metadata and Web Services ArcGIS Standards Support Pragmatic Interoperability (de-facto vs de-jure standards) Demonstrations
Interoperability Challenges Mobile Web Create Manage Visualize Analyze Collaborate Discover Cloud Enterprise Local - Multiple Platforms ( Operating Systems, Databases, ) Desktop - Multiple Architecture ( Local, Enterprise, Web, Cloud, ) - Multiple Clients ( Desktop, Web, Mobile, ) - Multiple Developer Environments (.NET, Java, ) - Multiple Protocols (SOAP, REST, OGC,.) - Multiple Encodings ( XML, Raster Formats, GML, JSON,.)
The ArcGIS Platform - An Open & Interoperable Platform - Supporting a thriving ecosystem of focused apps and applications. - A Standards Compliant Platform - Supporting many relevant standards including those from ISO TC 211 and OGC. http://www.esri.com/software/open
Developer Procurement personnel System Integrator User Standards ETL Open Interoperability
Many Standards Organizations. ISO ANSI/INCITS IHO WS-I OGC CEN OMA OASIS FGDC ACSM DGIWG ASPRS GSDI EPSG
WFS Many Standards SOS WPS WMS ISO 19139 WCS ISO 19115 GML Simple Features Open LS FGDC - CSDGM GeoXACML Geopackage OWS Context CityGML
GIS Standardization functional areas Basic Geographic Structures (semantic interoperability) Geometry, topology, coordinate reference systems Content Description (semantic interoperability) Standardized Data Models, Feature Catalog Modeling languages, Application schemas Metadata Data Management (semantic and technical) Database management Data Formats (technical interoperability) Structure, Syntax, language, API Visualization (semantic and technical) Symbology, cartography, display GeoWeb Services (technical interoperability) Web Catalog, Web Services, SOA
GIS Standardization functional areas Basic Geographic Structures ISO 19107 Spatial Schema, ISO 19111 Coordinate Reference Systems Content Description FACC, SDSFIE, Standardized Data Models ISO 19110 Feature Catalog Methodology, UML, FGDC Metadata, ISO 19115 Metadata Data Management OGC Simple Features Data Formats VPF, OGC GML, ISO 19139, 80+ industry and national formats Visualization SLD, MIL-STD 2525 Web Services OGC WMS, WFS, WCS, OpenLS Registry and catalog services CAT 1.0, OAI-PMH, CS-W (ebrim)
Standards Data Formats Metadata Web Services
Data Formats Simple Features CAD formats GML IFC KML NIEM Geopackage.. Gml JPEG 2000 OWS Context WaterML, AiXM,
What are simple features? Simple Feature specification defines: - Data access model - Database schema - 2D Geometry model for points, lines, polygons - Well-Known data formats for geometry and spatial reference ISO and OGC specifications for simple feature access Feature Tables contain rows (features) sharing common properties (Feature Attributes). Geometry is a Feature Attribute. 10 11 area1 area2 yellow green Feature Table 12 area3 Blue Feature 13 area4 red Geometry Feature Attribute
Geography Markup Language (GML) GML or Geography Markup Language is an XML based encoding standard for geographic information developed by the Open Geospatial Consortium (OGC). Defines an XML schema for representing geographic features (attributes, geometries, relationships, Topologies, Observations, etc). GML Simple Features - Initially motivated to help WFS use of GML 3 - GML 3 has many optional elements of schema, which complicates GML parsers - GMLSF Profile fixes application schemas to only allow the elements prescribed in the profile - This allows WFS servers and clients to be simpler and more efficient; the types of data which they need to anticipate are finite and reasonably structured for faster parsing
GML Support in ArcGIS GML Support in ArcGIS Desktop - Simple Feature GML support is available to all ArcGIS users - Import/export data - Use directly as a map layer - Use in a geoprocessing model - Consume GML served by any Web feature Service (WFS) GML Support in ArcGIS Desktop (ArcMap) using the Data Interoperability Extension - Support for specific Application schemas available using the Data Interoperability Extension. - Several application schemas supported - Custom schemas defined using transformation tools - Consume GML served by any Web feature Service (WFS) GML Support in ArcGIS Server Web Feature Service (WFS) implementation - GML is streamed using the Web Feature Service Protocol
Kml Support in ArcGIS KML is an XML grammar used to visualize geographic data in an earth browser. KML uses a tag-based structure with nested elements and attributes and is based on the XML standard. Serve Using ArcGIS Server Consume ArcGIS Desktop (ArcMap, PRO) Runtime ArcGIS Earth ArcGIS Online / Portal Geoprocessing Tools Layer To KML Tool The Layer To KML geoprocessing tool allows individual layers to be exported directly from ArcMap, ArcGlobe or ArcScene. Map To KML Tool The Map To KML geoprocessing tool allows multiple layers to be exported into a single KML source.
ArcGIS Data Interoperability Extension Format Support Adds support for 200+ data formats Custom Formats Direct use in ArcGIS Desktop Quick Data Translation Data Interoperability Tools Toolbox Convert between data formats Spatial ETL (Extract, Transform, Load) Semantic Data Translation Data Restructuring
Standards Data Formats Metadata Web Services
Metadata Leading the way to semantic interoperability Data about data - Geographic area covered - Currency - Rules of use - Positional accuracy - Data dictionary - Means of encoding - Datum - Map projection A Key interoperability technology: - Producers explain their product - Users learn about products Non standard products can be understood International Metadata Standards FGDC CSDGM ISO 19115, 19119 Metadata ISO 19139 Metadata XML Schema Implementation
ISO Metadata ( Today Future ) Ted Haberman : hdfcgroup.org
Metadata standards today Content Standard for Digital Geospatial Metadata Aka FGDC Metadata Extensions ISO 19115 Geospatial Information - Metadata Aka ISO Metadata Profiles <xsd:element name="bounding" type="boundingtype"/> <xsd:complextype name="boundingtype"> <xsd:sequence> <xsd:element ref="westbc"/> <xsd:element ref="eastbc"/> <xsd:element ref="northbc"/> <xsd:element ref="southbc"/> </xsd:sequence> </xsd:complextype>
Overview of Metadata Standards
ISO 19115 (-1) Profiles Australia-New Zealand Profile North American Profile of ISO 19115:2003 (NAP) CAN/CGSB-171.100-2009 INCITS 453-2009 INSPIRE Metadata Directive ISO19115/ISO19119 Application Profile for CSW 2.0 C3Grid ISO 19115 Metadata Profile Marine Community Metadata Profile of ISO 19115 Energy Industry Profile of ISO 19115-1 USGIN ISO 19139 Profile DISDI Geospatial Metadata Profile (DGMP) National System for Geospatial Intelligence Metadata Foundation (NMF) A metadata profile for precision agriculture based on ISO 19115 standard. WMO Core Profile of the ISO 19115 Metadata Standard Transformation of HDF-EOS metadata from the ECS model to ISO 19115-based XML Draft African metadata profile of ISO19115 Samoan profile NOKIS an ISO 19115 Based Metadata System The UDK and ISO 19115 Standard Latin American Profile Núcleo Español de Metadatos (NEM) UK GEMINI Biological profile Coastal zone profile S100 Metadata profile (IHO)
ArcGIS Metadata Standards Support ArcGIS Metadata ISO 19139 (NAP) XML FGDC CSDGM XML ISO 19139 XML ISO 19139 (INSPIRE) XML
ArcCatalog
ArcGIS Is a Platform Enabling GIS Everywhere Metadata Simple Integrated Open Desktop Web Device Portal Author / Create Use Catalog / Discover / search Serve / Publish Server Online Content and Services Available in the Cloud...... and On-Premises
ArcGIS Platform and Metadata Items Carrying Metadata Portal and Online Items Mxd s, Datasets, Services Folders Metadata Formats Item Information ArcGIS Metadata Formal Metadata FGDC, ISO, DCAT Encodings esri JSON, XML, atom, Indexing / Access / Discovery GeoServices REST Spec OpenSearch OGC CS-W Products ArcGIS Desktop ArcMap, PRO ArcGIS for Server ArcGIS Online / Portal for ArcGIS ArcGIS OpenData App Geoportal Server App Widgets
Esri support for metadata Geoportal Item Info ArcMap Mxd properties ArcGIS Online ArcGIS PRO ArcGIS service meatadata Service Metadata ArcGIS OpenData WMS metadata
What s Next for ArcGIS? Investigating how to incorporate support for ISO 19115-1/19115-3 into the ArcGIS platform - 19115-3 incorporates existing ISO 19115-2 and 19110 XML schemas - Evaluating XML schemas available in new ISO repositories Continue to Improve metadata support in ArcGIS Pro and throughout the platform - Desktop -> Server -> Online/Portal - Planning next steps
Standards Data Formats Metadata Web Services
OGC Web Services Web Map Service OGC KML Web Map Tiling Specification Sensor observation Service Web Feature Service Web Processing Service Web Coverage Service 3D Portrayal service.
DGIWG and OGC Service Types and Profiles Geospaial Interoperability Reference Architecture doc Report Spring 2015 https://www.ise.gov /sites/default/files/gira.pdf http://ise.gov from Geospatial Interoperability Reference Architecture (GIRA)
ArcGIS Platform Components Core Standards Portals (AGOL, Portal) Providing & Consuming Server Providing Developer Solutions (runtime, Javascript Api) Consuming Desktop (ArcMap & PRO) Consuming Apps Consuming Metadata (ISO, FGDC, ) Data (Geopackage, geojson, gml- JPEG 2000 ) Portrayal Standards Services (wms,wfs,wcs, wps,sos,cs-w, )
JSON clients (HTTP REST) OGC clients (HTTP SOAP) Inspire clients (HTTP SOAP) Inspire CSW Inspire ATOM Inspire WFS Inspire WMS WPS WCS WFS WMTS WMS KML GeoJSON Esri REST API ArcGIS for Server ArcGIS for Inspire Geoportal http://www.esri.com/library/whitepapers/pdfs/supported-ogc-iso-standards.pdf
Standards Requirements Across the platform.. What we are hearing! Online / Portal - Register OGC services - Consume OGC services - Create hosted OGC services - Geopackage support - Allow analyses on OGC services - Support Discovery Standards OGC CS- W, OpenSearch, DCAT - Managing Secured OGC Services Metadata Support across the platform Federated searches across portals Server - Improve existing OGC support ( compliancy, update to new version, new profiles) Desktop (ArcMap / Pro) - Consume OGC services - Geopackage support Runtime and Apps - Consume OGC services - Geopackage support Javascript API - Consume OGC services
Standards Support ArcGIS Online - Support for formal Metadata (Version 3.8, July 2015) - Support for secured OGC services (Version 3.10, Nov 2015) ArcGIS PRO - Improvements to wms wmts and metadata support in PRO (PRO 1.2) - Improvements to KML support ( Pro 1.3) - Major improvements in Online (version 4.2, June 2016 ) wms, wfs ArcGIS Earth 1.1 ArcGIS for Server - Improvements to wfs 2.0 support - Achieved KML Compliancy (Server 10.4) - Supports kml, wms and wmts
Standards Support - Roadmap Support for Hosted wms, wmts and wfs support in Online (Q4 2016) and Portal ( Version 10.5) Native wfs client support in PRO (Version 1.4) WCS 2.0.1 support in ArcMap and PRO Support for wms, wmts ( Q4 2016 Quartz runtime) Support for geopackage, wfs, s57 in (Q1 2017 Quartz runtime) WCS support in Javascript API (prototype) Web App Builder widget to consume OGC OWS context document with embedded ogc services (prototype) Native netcdf Support in ArcMap (10.5) and PRO (10.4) Better metadata support across the stack Improvements to geopackage support
Procurement personnel http://www.opengeospatial.org/resource/products/compliant#esri @ 10.4 : 16 certificates
SOA Vs REST Pragmatic Interoperability http://www.programmableweb.com/news/api-anti-patterns-how-to-avoid-common-restmistakes/2010/08/13 Use the Web Luke! http://geekandpoke.typepad.com/.a/ 6a00d8341d3df553ef012875f312f9970c-pi. Some GIS Standards are still playing catchup with the Web
Standards, Specifications,. Pragmatic Interoperability geojson georss Atom Sqlite Vector tiles DCAT / geodcat -AP CSS Styling RDF No SQL xlsform Opensearch + geoextension
Open API s and Specifications Pragmatic Interoperability Shapefile (July 1998) Geodatabase XML (June 2008) GeoServices REST API (Sept 2010) FGDB API (Jan 2011) i3s (May 2015) LERC (Dec 2015) Open Data Download Stats for April 2016 Shapefile: 3,714,978 CSV: 413,037 GeoJSON: 318,052 KML: 43,689 GeoHash: 22,033
The Proof of the Pudding @martenhogeweg
Group on Earth Observations SAEOS INPE GCMD PANGAEA SeaDataNet GEO ArcGIS Online EuroGEOSS EEA-SDI IOOS EnviroGRIDS GBIF GSFC-DAAC WMO WIS
Connecting GIS and Earth Observations
Geospatial Platform and Data.gov NASA GSA Census FCC Geoplatform.gov Data.gov DOI DOT NOAA DOJ DHHS DHS EPA
Access Geoplatform Content in ArcGIS Connect to http://geoplatform.maps.arcgis.com Find National Geospatial Data Assets Use in Your GIS Projects
OGC Testbed 12 Viewer - ArcGIS, WMS, WCS, KML - WMTS, Koop FeatureServer Discovery - OpenSearch - OGC CSW 2.0.2, 3.0.0 OGC Context Document - GeoJSON encoding
Please take our Survey Your feedback allows us to help maintain high standards and to help presenters Find your event in the Esri Events App Find the session you want to review Scroll down to the bottom of the session Answer survey questions and submit