GeoSUR Program Architecture and System Design Title Creator Reviewer GeoSUR Program Architecture Development Guideline Michelle Anthony. USGS. Eric van Praag, CAF Jesus Suniaga, CAF Date 1 September 2008 Subject Publisher Type Description Rights Language GeoSUR Architecture and System Design GeoSUR Program Text The main requirements and components to be undertaken by the GeoSUR Program will be reviewed with existing standards, technologies and software implementations Corporación Andina de Fomento (CAF) EN
TABLE OF CONTENTS Introduction o Purpose of this Paper o Objectives and Requirements o Main Components of the GeoSUR Architecture o Technical Evaluations o Standards Main GeoSUR Components o Geospatial Portal o Map Services o Metadata Services Supporting Components - Additional Map Viewers o IIRSA Regional Map Viewer o National Government Map Viewers Appendix A: Map Service Software Implementations Appendix B: Metadata Creation Software Appendix C: Metadata Service Types and Implementations 2
INTRODUCTION Purpose of this Paper This white paper introduces the technical architecture and design to be implemented within the GeoSUR regional program. It will explain how GeoSUR will implement the necessary technology and standards to develop a regional program that provides the mechanism for the discovery and exchange of geospatial information for decision making within South America. Objectives and Requirements The design of the GeoSUR architecture was developed using the requirements provided by the GeoSUR project staff as well as the requirements defined within the Spatial Data Infrastructure (SDI) Cookbook. Main Components of the GeoSUR architecture The main components of the decentralized GeoSUR architecture consist of the following items: the GeoSUR portal components along with distributed map services and metadata services from the geospatial data producers. These components will connect geospatial producers and users through the following scenario: 1. Geospatial data producing agencies develop metadata (data about data) to describe their data. The metadata is hosted through a metadata service and the geospatial data is hosted through a map service. These agencies provide their metadata records to the GeoSUR portal through a harvesting mechanism to populate the metadata catalog. 3
2. Users within the GIS community can visit the GeoSUR portal to search and discover geospatial data through the search tool that queries the metadata catalog. Metadata can then be viewed and evaluated for its usefulness. The metadata also provides a link so the map service can be viewed in the GeoSUR map viewer. 3. The final step allows the GIS user the means to download or obtain the actual spatial data for their usage. Technical Evaluations Many implementations of software are available for the development of SDI systems from the commercial and Open Source communities. All mature software implementations were evaluated for the GeoSUR design and the best suited software that met the GeoSUR requirements document were selected for the final implementation. For the selection of the Portal component, a technical evaluation was completed that compared the ESRI GIS Portal Toolkit (GPT) and the GeoNetwork OpenSource portal. The results of the technical evaluation were shared with the GeoSUR technical committee to determine which software best met the GeoSUR requirements. The GeoSUR technical committee selected the use of the ESRI GPT version 9.3 software for the portal. For the implementation of the metadata and map services by each geospatial data producer, the GeoSUR architecture has been setup to be flexible and accommodating. Each geospatial data producer can use any software for the map service, metadata creation, or metadata service as long as the appropriate standards are followed. Some of the potential software implementations (commercial and open source) will be described later in this document. Standards The design of the GeoSUR architecture has been based upon compatible and mature geospatial standards that allow for the maximum technical interoperability based upon SDI standards. Geospatial standards are primarily developed by the following organizations and have been considered in the GeoSUR architecture. 4
- International Organization for Standardization (ISO) Technical Committee 211 - Open Geospatial Consortium (OGC) The following standards have been selected for acceptance in the GeoSUR architecture: Information and Content Standards - ISO 19115/19139 - Latin America Metadata Profile (LAMP) - FGDC Content Standard for Digital Geospatial Metadata (CSDGM) - OGC Styled Layer Description (SLD) - OGC Web Map Context (WMC) Service and Interface Standards - OGC Catalog Service (CS-W) specification - OGC Web Map Service (WMS) 5
MAIN GEOSUR COMPONENTS The main components of the GeoSUR system will consist of decentralized components that interact and are interoperable through standards. This will allow each participating geospatial data producer to implement and maintain their own map and metadata services. Since the GeoSUR architecture supports the standards established in the SDI Cookbook, any country or national agency who already support a regional or national SDI will be immediately compatible with the GeoSUR architecture. Geospatial Portal The GeoSUR Portal contains several main components that exist within the portal to interact for the search and discovery of geospatial data resources. - Portal Website - Harvesting Service - Metadata Catalog - Search Tool - Map Viewer The main components of the GeoSUR architecture will operate in a scenario similar to the below general description. 1. Geospatial data producer agencies will prepare their data to be served. A map service will be established to serve the data. 2. Geospatial data producer agencies will prepare metadata to describe their geospatial data sets. The metadata will include a link to the associated map service. 3. The data producer agency will login to the GeoSUR portal and register their metadata service for harvesting. 4. The GeoSUR portal will connect to the individual metadata services to harvest, validate and obtain a local copy of the metadata in the portal metadata catalog database. 5. Users of the geospatial user community will access the GeoSUR portal to locate the available geospatial data in the region. 6. Users will use the portal search tool to define their search criteria and execute the search and evaluate the returned metadata results. 6
7. The metadata can be viewed in its entirety to obtain the full background on each data set. 8. If a map service was established and documented in the metadata, the user can view the data in the portal map viewer. These data can then be viewed with other data sets. Map Services GeoSUR will initially work with national government agencies in South America that include the National Geographic Institutes and the National Environmental agencies. The GeoSUR program will request that each of these geospatial data producer establish map services to provide a visualization of their data sets for viewing and interrogating through a web browser based map viewer in the portal. These map services can be accomplished through the establishment of OGC Web Map Services (WMS). WMS is a simple HTTP interface for requesting georegistered map images from one or more distributed geospatial services. A WMS request defines the geographic layer(s) and area of interest to be processed. The response to the request is one or more geo-registered map images (returned as JPEG, PNG, etc) that can be displayed in a browser application. The interface also supports the ability to specify whether the returned images should be transparent so that layers from multiple servers can be combined or not. Each geospatial data producer agency will implement a WMS service to provide pictures of their data to a WMS based map viewer in the GeoSUR portal. There are many software implementations that can be used to establish the map services. Training by the GeoSUR program initially focused on the following software but any software can be used as long as it supports the OGC WMS specification. - Minnesota MapServer - ESRI ArcIMS The GeoSUR technical team has established a technical guide for establishing WMS services that describes the guidelines that should be followed. Appendix A describes some of the most common WMS software implementations. 7
Metadata Services The geospatial data producer agencies will also document their geospatial data (vector and raster) using globally accepted metadata standards. The metadata will be available for harvesting to the GeoSUR Portal so it can be collected and stored in a central database for future search and discovery by the geospatial community. Most of the data sets documented during the first phase of the GeoSUR program will consist of core data including topographic and environmental data sets but will not be exclusive to only these data. The GeoSUR Geospatial Portal will be configured to harvest and validate the following types of metadata: - ISO 19115/19139 - Latin America Metadata Profile (LAMP) - ISO 19115/19139 -North America Metadata Profile - FGDC Content Standard for Digital Geospatial Metadata (CSDGM) This will allow each geospatial data producer agency to select and implement the metadata editor software of their choice, as long as it supports one of the above metadata standards. Commercial and open source metadata editor software exists. The GeoSUR program provided some initial metadata training workshops where the following selection of metadata editors were discussed and used for training. Appendix B provides additional metadata software references and additional details. - GeoNetwork OpenSource - ESRI ArcCatalog - CATMDEdit Once the metadata has been developed to describe an agencies geospatial datasets, it must be provided to the GeoSUR portal for harvesting. An important step during the metadata development process is to ensure that the metadata contains a link to the associated map service. This will allow the GeoSUR portal to allow its users to immediately visualize the map service in a map viewer directly from the portal search results. The agency will login to the GeoSUR portal and register their metadata service for harvesting. The harvesting process consists of the portal connecting to metadata services to obtain a copy of the metadata and store it in a catalog database within the portal. This metadata can then be searched and discovered through a geospatial search engine on the portal. 8
Each geospatial data producer agency can select and implement the metadata service of their choice. The GeoSUR portal can harvest metadata using the following protocols: - ESRI metadata services - Z39.50 - Open Archives Initiative (OAI) - Web Accessible Folders (WAF) - CS-W Catalogs A brief description of each metadata service type can be found in Appendix C. SUPPORTING COMPONENTS - ADDITIONAL MAP VIEWERS The GeoSUR portal contains a map viewer that works with OGC services to provide interoperability to combine pictures of geospatial data from different GeoSUR servers to allow them to be viewed in the same map viewer. The GeoSUR program is also recommending that individual map viewers will be established by each participating data agency to serve their own unique user communities. The following map viewers will be established during the GeoSUR program. These can be established using any web mapping software as they will only provide a single browser based view of the data being served through it. GeoSUR Regional Map Viewer CAF will establish a map viewer using the IIRSA (Initiative for the Integration of Regional Infrastructure in South America) data and assorted infrastructure data available at CAF. This map viewer will be built using ESRI ArcGIS Server using the out of the box viewer. The data will also be served through a WMS connector. Metadata will also be developed for each data set and harvested by the GeoSUR portal. National Government Map Viewers GeoSUR will recommend that each participating agency develop a stand alone map viewer in addition to the WMS map service that provides the data to the GeoSUR portal. 9
These stand alone map viewers will be linked from the organizations website and will provide a unique interface to the geospatial data for that organization. 10
APPENDIX A: MAP SERVICE SOFTWARE IMPLEMENTATIONS Minnesota MapServer MapServer is an Open Source development environment for building spatiallyenabled internet applications. MapServer is not a full-featured GIS system, nor does it aspire to be. Instead, MapServer excels at rendering spatial data (maps, images and vector data for the web. URL: http://mapserver.gis.umn.edu/ GeoServer GeoServer is an Open Source server that connects your information to the geospatial web. With GeoServer, you can publish and edit data using open standards. Your information is made available in a large variety of formats as maps/images or actual geospatial data, and you have complete control over the look of the map. URL: http://geoserver.org/ ESRI ArcIMS ArcIMS is the solution for delivering dynamic maps and GIS data and services via the Web. It provides a highly scalable framework for GIS Web publishing that meets the needs of corporate Intranets and demands of worldwide Internet access. URL: http://www.esri.com/software/arcgis/arcims/ ESRI ArcGIS Server ArcGIS Server connects people with geographic information. Organizations, small and large, use ArcGIS Server to distribute maps and GIS capabilities over the web to improve internal workflows, communicate vital issues, and engage stakeholders. URL: http://www.esri.com/software/arcgis/arcgisserver/ 11
APPENDIX B: METADATA CREATION SOFTWARE The following metadata creation software are being recommended for use within the GeoSUR program. GeoNetwork OpenSource GeoNetwork OpenSource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides a powerful metadata editing tool and search functions as well as an embedded interactive web map viewer. URL: http://geonetwork-opensource.org/ ESRI ArcCatalog ArcCatalog is a program you use to organize and manage various datasets and documents in both file-based workspaces and geodatabases. It allows you to connect to your data source locations, browser through you workspaces, develop metadata, examine or explore the data, manage data and metadata and search for data and maps. URL: http://www.esri.com/products/ CATMDEdit CATMDEdit is a metadata editor tool that facilitates the documentation of resources, with a special focus on the description of geographic information resources. URL: http://catmdedit.sourceforge.net/ 12
APPENDIX C: METADATA SERVICE TYPES AND IMPLEMENTATIONS The following types of harvesting protocols can be used by each data producer agency to provide their metadata to the GeoSUR portal. Each is explained briefly below with a URL to more information. ESRI Metadata Services Metadata can be maintained and served to the GeoSUR portal using the ArcIMS Metadata Service software. ArcIMS Metadata Services can be used to create a central, online metadata repository that allows you to easily publish and browse metadata over the Internet. You can author your metadata through the ArcGIS ArcCatalog application using industry-standard and user-definable style templates, then publish your metadata by simply dragging and dropping it into the ArcIMS Metadata Server. Metadata is optimized for rapid and efficient searches using a variety of clients including lightweight browser-based clients and ArcGIS Desktop. URL: http://www.esri.com/software/arcgis/arcims/about/metadata-services.html Z39.50 Metadata can be served to the GeoSUR portal through a Z39.50 metadata clearinghouse. Z39.50 is a client server protocol for search and retrieving information from a remote computer database. It is widely used in library environments and has been around since the 1970s. The Isite software is an open source Z39.50 implementation that has been maintained and provided by the FGDC for geospatial metadata. URL: http://clearinghouse4.fgdc.gov/ftp/ Open Archives Initiative (OAI) OAI is an attempt to build a low barrier interoperability framework for archives containing digital content. It allows people to harvest metadata. This metadata is used to provide "value added services" often by combining different data sets. URL: http://en.wikipedia.org/wiki/open_archives_initiative 13
Web Accessible Folders (WAF) A simple way to provide your metadata to a portal is by providing your metadata files in XML format on a web accessible folder (WAF). This is a directory on the WWW where a web browser can browse the content of a directory. It may not contain a default.html or index.htm file. URL: na CS-W Catalogs The OpenGIS Catalogue Services Interface Standard (CAT) supports the ability to publish and search collections of descriptive information (metadata) about geospatial data, services and related resources. Providers of resources use catalogues to register metadata that conform to the provider's choice of an information model; such models include descriptions of spatial references and thematic information. Client applications can then search for geospatial data and services in very efficient ways. URL: http://www.opengeospatial.org/standards/cat 14