PLATA-HIS Hydrologic Information System Central Web Service Registry for the Plata basin in South America

Transcription

1 PLATA-HIS Hydrologic Information System Central Web Service Registry for the Plata basin in South America Technical proposal Silvano Pecora Head of Hydrology ARPA Emilia-Romagna, Italy November 2015

2 Table of Contents 1. Data collection and dissemination in hydrology Sharing of data and information A federated approach to water data sharing The HIS architecture The HIS components Standardized service types and data formats WIS coompliance Resource issues

3 1. Data collection and dissemination in hydrology The ultimate goal of data collection in hydrology, be it precipitation measurements, water-level recordings, discharge gaugings, groundwater monitoring and water quality sampling, is to provide a set of sufficient good quality data that can be used in decision-making in all aspects of water resources management, in the wide range of operational applications as well as in research. Decisions may be made directly from raw data measurements or based on derived statistics or on the results of many stages of modelling beyond the raw data stage, but it is the collected data that form the basis for these decisions. Data sets are of great intrinsic value as they are collected through a huge commitment of human and financial resources and often during a long period of time, and they acquire also a use value when are made available in an usable form to the final users to respond to their specific needs. The management of hydrological data is therefore important work in itself and this work must be performed effectively in order to maximize the results of the investments put in their collection and their possibility of being effectively used. WMO Hydrology and Water Resources Programme has always given high priority to issue related to data management, be it through the work of experts of the Commission for Hydrology, the publication of manual and other guidance material, the implementation of projects for improving data collection and information production (such as WHYCOS) and data rescue, the promotion of the access to and exchange of hydrological data, through the implementation of the Resolution 25 (Cg-XIII) - Exchange of Hydrological Data and Products. In support of these activities of the access to and exchange of hydrological data provided by the National Hydrological Services belonging to the Plata basin (namely, Argentina, Bolivia, Brasil, Paraguay and Uruguay), this technical proposal is intended to provide additional operational capability, in particular for in situ water observations, to be called PLATA-HIS, as a federated resource for National Hydrological Services. Figure 1 Discovery and access of hydrological data provided by the National Hydrological Services of the Plata basin 3

4 The scope of the current proposal is defined as a global registry of water data and map services catalogued using the standards and procedures of the Open Geospatial Consortium and the World Meteorological Organization. This registry as a specialist hydrological component in the WMO Information System (WIS) will be open to all users and institutions from any country or level of government, and applies to any type of water information. It is anticipated that this activity will be undertaken in collaboration with Theme 2: Data Operations and Management of the WMO Commission for Hydrology, which is active from 2012 to Sharing of data and information Resolution 25, adopted in May 1999 by the Thirteenth World Meteorological Congress, commits WMO Members to broaden and enhance, whenever possible, the free and unrestricted international exchange of hydrological data and products. This engagement is an important contribution to disaster risk reduction, human safety and well-being, and shared socio-economic benefits. In 2005, a Global Climate Observing System (GCOS) report identified standards as a key problem in data exchange in global hydrological and atmospheric networks. Thus, in 2009, WMO and the Open Geospatial Consortium (OGC) jointly formed a Hydrology Domain Working Group (HDWG) to address this issue. The OGC adopted WaterML 2.0: Part 1 - Time Series in 2012 and Part 2 Rating Curves, Gaugings and Cross Sections in 2015 as official standards (see Encoded in extensible Markup Language (XML) and based on existing OGC standards, WaterML 2.0 provides an interoperable hydrological exchange format that may be used to address a wide range of user needs. These include the exchange of data relating to: - In situ observations at hydrological (gauges, reservoirs) or climatological stations; - Forecast products (probabilistic or deterministic time series) at forecast locations; - Emergency or operator-oriented alerts (of threshold exceedance) and reports; - Time series of planned intake and release/discharge; - Groundwater observations of the water level within wells. Using WaterML 2.0, it is possible to link together local, national, regional and global water information sources as part of global water information networks. The availability of such a format will also facilitate the usability of data shared through the WMO Information System (WIS). The 2012 WMO Commission for Hydrology (CHy) recommended that WMO Members test, through pilot projects during , the use of WaterML 2.0 for the exchange of hydrological data with a view of its adoption as a joint WMO/ISO standard. The Commission also noted the importance of WaterML 2.0 and other emerging OGC standards for improving service delivery in key areas, including the WMO World Hydrological Cycle Observing System (WHYCOS) and the WMO Flood Forecasting Initiative. In 2014 the CHy Advisory Working Group has proposed the WMO Hydrological Observing System (WHOS) as the means to provide the most comprehensive hydrological component in fulfillment of the WIGOS objective of an integrated, comprehensive, and coordinated system which is comprised of the present WMO global observing systems. WHOS is conceived as a portal to facilitate access to already available on-line real-time and historical data, drawing from the water information systems of countries around the world that make their data freely and openly available, including HYCOS projects. WHOS is composed by two phases: Phase 1 provides a map interface with links to those NHSs that make their real-time and historical stage and discharge data available online (initial implementation July 2015); Phase 2 provides a fully WIS/WIGOS compliant services-oriented framework linking hydrologic data providers and users through a hydrologic information system enabling data registration, data discovery, and data access (Beta version for CHy-15 review and endorsement in December 2016 and initial implementation for EC approval in June 2018). 4

5 Figure 2 WMO Integrated Global Observing System (WIGOS) 3. A federated approach to water data sharing PLATA-HIS aims at developing CyberInfrastructure and services to support the advance of hydrologic science in the international hydrological community. Hydrologic information science involves the description of hydrologic environments in a consistent way, using data models for information integration and software tools that are either customized or based on commercial products to directly interact with the deployed data storage and communication components. PLATA-HIS will support storage of water observations data in a relational database, publication via web-services on the internet, federation with water observations data published by water agencies, and search across the various data holdings in the network. These advanced data access and analysis capability are provided through the use of web services using a standardized data formats and service types. 5

6 a) Hydrologic data request b) Hydrologic data collection c) Hydrologic data transformation d) Hydrologic data delivery Figure 3 Standardized data formats and service types in hydrologic data exchange 3.1. The HIS architecture When sharing information across scienti c communities, it becomes important to de ne a standard framework through which large quantities of multidisciplinary information can be shared, discovered and accessed. Utilizing a collection of WMO and OGC Web services, PLATA-HIS is designed as a services stack framework that shares catalog data, metadata and data with the user. The services stack framework identi es three types of services as essential to sharing water information across the Web: catalog services, metadata services and data services. These three services work together to completely index, describe and provide access to water information (e.g. time series). Catalog services provide users with an index of hydrologic metadata, metadata services identify collections of time series available over a domain of space and time, and data services provide the user with the raw data for a speci ed temporal period and spatial area. This framework is designed to publish and distribute time series, but it can be extended to include grids and coverages as well. While other interoperability studies have focused on implementing custom data streams (e.g. bridges, adaptors, etc.) between clients and server interfaces, PLATA-HIS focuses on a common data and metadata management model that leverages a suite of WMO and OGC standard Web services which can be applied to multiple scienti c communities in particular hydrologic and atmospheric sciences. 6

7 Figure 4 Discovery and access functionalities supported by broker and mediator Furthermore, PLATA-HIS can be integrated within existing data discovery frameworks (e.g. portals, gateways, etc.) by leveraging mediation and brokering services The HIS components As with any other Service Oriented Architecture, PLATA-HIS is built around two fundamental components: (1) service providers and (2) service consumers. Although service consumers directly connect to service providers to request and receive data, a third component, a service registry, is introduced to facilitate the discovery of different service providers; this can be done using various keywords, metadata and lters. As service providers introduce their services within PLATA-HIS, services are registered at the service registry. Service consumers can then search the registry to nd available services of interest. PLATA-HIS can be de ned as a collection of components which work together to store, index, access and distribute hydrologic information. The system contains servers, catalogs and applications which communicate with one another through a set of web services. Web services are the set of protocols and speci ed functions that exchange hydrologic metadata and data (i.e. time series) through the web using a common standardized language, The individual components of PLATA-HIS each serve an important role in the data discovery and fetching process. Data providers are the principal locations for storing large volumes of hydrologic data, speci cally time series. Within the provider itself, data and metadata are managed in a database and then exposed through its Principal GISC (Brasilia) as well as through a suite of web services so that remote users can then access the data through the web. 7

8 Figure 5 - Data provider in the Plata basin Another component of PLATA-HIS is the hydrologic metadata catalog. The registry is the component of PLATA-HIS which facilitates the discovery of hydrologic data that has already been published by the data providers as required in the Manual on WIS (WMO No.1060) section Describe information with metadata. Within PLATA-HIS, data providers are the primary repositories for hydrologic data, while the registry is the primary repository for hydrologic data services. The registry provides an interface where users can search registered data services by specifying keywords and metadata which describe the hydrologic data of interest. The registry is like a Google for discovering hydrologic time series information. Data publishers can register their data on the registry and provide brief descriptions of the datasets they want to share. This is an important aspect of PLATA-HIS because it allows for data to be organized and discovered in an efficient, structured and methodical process. 8

9 Figure 6 - Registry applied with a federated approach to the Plata basin The third and final component of PLATA-HIS is the web or desktop client. The client is the component of PLATA-HIS that allows for the harvesting of hydrologic information at the locality of one s own computer or analytical system as per Manual on WIS (WMO No.1060) section Provide access to information. The client is a platform located on the user s machine and communicates with both data services and the registry. Users can directly download hydrologic information from data providers if they already know of their existence or can search the registry for data that they might not know about. Once the data of interest has been discovered, users can download the information onto their local databases. 9

10 Figure 7 - Desktop client searching and downloading data via web services from the Plata HIS Figure 8 - Web client searching and downloading data via web services from the Plata HIS 10

11 3.3. Standardized service types and data formats There are three components to the services stack framework which work together to provide a system in which data consumers can readily discover and access both time series and coverage data using spatial, temporal and semantic lters: Catalog Services, Metadata Services and Data Services. At the core of the services stack framework lie the metadata services which act as middleware between the catalog services and data services. Data services ultimately provide the user with the data they are searching for, whereas catalog services allow users to perform federated searches across multiple data providers. Metadata services link both these layers together by being registered at the catalog level and providing all the information needed to access information at the data level. Data services are at the bottom of the services stack framework and are responsible for providing users with the data sought after. These services follow the Sensor Observation Service (SOS) and Web Coverage Service (WCS) speci cations outlined by the OGC. For hydrologic information, time series data is obtained through a SOS implementation encoded in WaterML 2.0. Similarly, for atmospheric science information, coverages can be obtained through a WCS implementation as a netcdf le. Each service allows users to specify spatial, temporal and semantic lters to obtain subsets of the larger datasets supplied by the data service. Metadata is an essential component of the data sharing process. Not only does it facilitate the search and discovery of information within one s own research community but also helps foster interoperability between research communities. Metadata is used to describe a set of data that share a common ground to others who are not directly familiar with the information. The metadata issue is very well documented and has led many scienti c communities to adopt metadata standards created by the International Organization for Standardization (ISO). Speci cally, the ISO standard provides a manner in which geographic metadata can be published across web based information systems. Figure 9 Catalog interfaces in the Plata HIS 11

12 Catalog services aid in the management, discovery and distribution of metadata describing geographic datasets and services. Within the PLATA-HIS services stack framework, catalog services function as the interface through which data consumers discover indexed metadata services published as WFS. As part of the standard suite of OGC services, Catalog Services for the Web (CSW) are the Web services that focus on the management and indexing of geographic metadata. The OGC designed CSW to help data consumers search through a set of matching resources. As such, CSW allow data publishers to register and index a set of metadata services with a variety of different metadata pro les as de ned by the ISO (e.g. ISO 19115/19119). Using a CSW interface, data publishers are able to share their catalog of metadata with search clients as well as other catalogs. 4. WIS coompliance The WMO Information system (WIS) is the single coordinated global infrastructure responsible for the telecommunications and data management functions. It is the pillar of the WMO strategy for managing and moving weather, climate and water information in the 21st century. WIS provides an integrated approach suitable for all WMO Programmes to meet the requirements for routine collection and automated dissemination of observed data and products, as well as data discovery, access and retrieval services for all weather, climate, water and related data produced by centres and Member countries in the framework of any WMO Programme. WIS was designed to dramatically extend WMO Members' ability to collect and disseminate data and products. Owned and operated by Members, it will be the core information system utilized by the WMO community, providing linkages for all WMO and supported programmes associated with weather, climate, water, and related natural disasters. In the context of its role in WIS, PLATA-HIS will publish discovery metadata in the GISC Brasilia allowing WMO users to access a range of services on PLATA-HIS. By publishing in GISC Brasilia, the discovery metadata will also be available to the GEOSS community through the GEO Data Portal which includes all WMO discovery metadata. A potential future phase of the PLATA-HIS is to pilot possible extensions of WIS functions and services tailored to the needs of the hydrological community, but potentially beneficial to other programmes, including databases, simulation models, web-gis and other tools. 5. Resource issues This technical proposal will be undertaken within existing resources. However, it is recognized that eventually the water data sharing system could require additional resources to ensure sustained technological support, and to expand the objectives of the proposal which could include the collection and sharing of additional parameters, as well as the development of models to support delivery of products and services through related WMO efforts. 12