Arctic Climate and Weather Forecast Model Visualization Scenario Engineering Report GEOSS Architecture Implementation Pilot Version 1.

Transcription

1 Arctic Climate and Weather Forecast Model Visualization Scenario GEOSS Architecture Implementation Pilot Version 1.0 Content developed by the GEO Architecture Implementation Pilot Licensed under a Creative Commons Attribution 3.0 License

2 Revision History Version Date Editor and Content providers Comments /07/2010 Yuqi Bai Section 1.1, 1.2, 3.1, 3.2, 3.3, /08/2010 Xuanang Cheng Section 5.1 2)3), /09/2010 Doug Nebert Section 5.3, 5.4, 6, Revised Draft /13/2010 Yuqi Bai Figure 3a, Revised Draft /21/2010 Ben Domenico Edit Section /28/2010 Doug Nebert Final edits and accept comments/changes Document Contact Information If you have questions or comments regarding this document, you can contact: Name Organization Contact Information Douglas D. Nebert USGS/ FGDC ddnebert@usgs.gov Liping Di CSISS, GMU ldi@gmu.edu Xuanang (Roman) Cheng CSISS, GMU xcheng@gmu.edu Genong (Eugene) Yu CSISS, GMU gyu@gmu.edu Yuqi Bai CSISS, GMU ybai1@gmu.edu Ben Domenico UCAR UNIDATA ben@unidata.ucar.edu Tyler Erickson MTU MTRI tyler.erickson@mtu.edu Page 2

3 Table of Contents Contents 1. Introduction Scope of this document GEOSS AIP 4 2. Community SBA Objectives 4 3. Scenario: Climate Change and ArcSDI Actors Context and pre-conditions 3.3 Scenario Events Post-Conditions Special Requirements 7 4. System Model of the Scenario 7 5. Implementation Deployed Components Use of the GCI Demonstrations Future plans for deployment Discussions and Lessons Learned References 12 Page 3

4 Arctic Climate and Weather Forecast Model Visualization Scenario 1. Introduction 1.1 Scope of this document This AIP-3 ER will describe the user scenario entitled "Arctic Climate and Weather Forecast Model Visualization Scenario." This scenario demonstrated publishing, discovery, and visualization of weather and climate data through GEOSS, highlighted value of geospatial standards common to Spatial Data Infrastructures as layers of interest to a conceptual Arctic SDI, and supported end-user desktop visualization of complex climate and weather models for decision support in the Arctic region. The architecture, workflow of the developed system is presented; the standards-based GEOSS components and services are highlighted, along with our lessons learned. 1.2 GEOSS AIP The GEOSS Architecture Implementation Pilot (AIP) task develops process and infrastructure components for the GCI and the broader GEOSS architecture as a means of coordinating cross-disciplinary interoperability deployment. The AIP Task provides phased delivery of components to GEOSS operations, with each phase consisting of: architecture refinement based on user interactions; component deployment and interoperability testing; and SBAfocused demonstrations. This (ER) is a key result of the third phase of AIP. AIP-3 was conducted from January 2010 to December A separate ER describes the overall process and results of AIP-3 and thereby provides a context for this Community SBA ER Community SBA Objectives The objectives of this scenario are 1) to demonstrate the availability of climate and weather forecast data through GEOSS, 2) to visualize climate and weather phenomena in formats and terms more meaningful to non-expert users, and 3) to provide a viewing scenario amenable to use in the Arctic or polar environment (suitable projection via globe view of KML) along with other geographic data for context. The primary audience would be planners and logistics individuals interested in arctic climate and weather data. 3. Scenario: Climate Change and ArcSDI 3.1 Actors The main involved actors in this Scenario are: Analyst: The end user of the GEOSS system, who consults the GEO Web Portal about the availability of mapped weather and climate forecast data for his area of interest from the present through Real-Time and Forecast Weather Data Service Provider: o Unidata Motherlode Climate Change Model Data Map Provider: o Intergovernmental Panel on Climate Change (IPCC) Weather Station Climatology Data Provider: o CSISS/GMU from NOAA climatology CD 1 A listing of all AIP-3 s: Page 4

5 3.2 Context and pre-conditions These resources were registered in the GEOSS Component and Service Registry, and are discoverable through GEO Portal. 1. Current Weather Forecast WMS and WCS interface (UCAR) 2. KML to wrap IPCC-data.gov/maps long-term climate model WMS services (GMU) 3. Station Climatology KML from NOAA NCDC data (GMU) 3.3 Scenario Events Table 1 Steps in the Community Scenario Step Description Trans. Tech Use Case 0 Analyst in the arctic region consults the GEO Web Portal about the availability of mapped weather and climate forecast data for his area of interest from the present through Search for Resources General search with a geographic footprint over near-shore waters in northern Europe, and content keywords of 'climate weather forecast.' Their interest is in identifying surface temperature changes that may affect permafrost conditions, with impact on surface infrastructure maintenance. 1 In the right pane, a list of search results includes the "Unidata Motherlode Real-Time Weather Data Services." 5. Present Services and Alerts Page 5

6 2 The analyst clicks on the brief metadata, and requests and views a "Full Description" metadata record. 3 The analyst returns to the prior URL "Summary Description" and sees a KML file link present. 4 The analyst clicks on the KML link and is prompted to save the file or launch an installed program, such as Google Earth - selects launch in GE. 5 Analyst zooms to northern Europe in Google Earth, illustrating that a circumpolar view is possible. 6 The analyst browses the available model parameters and time periods in GE, opening the appropriate 'folders' in the legend control to select surface temperature for all time periods, with an interest in the month of July. 7 The analyst views an animated sequence of temperature with decadal frames, stops the animation on a period of interest (2050). 8 The analyst is now interested in knowing what weather station climatologies exist in northern Europe. Using geoportal.org, they search for "monthly climate" 9 Select the registered Component for the monthly climate KML file and also launch in GE. 10 Click on stations to reveal basic station information, click on embedded hyperlink to display the climate graph for the station to answer the question of what current "normal" temperatures should be for July. 11 Returning to a more immediate facilities management problem, the analyst is interested in knowing current weather forecasts for the same area of interest. Using geoportal.org, they search for "weather forecast KML" and locate the Motherlode GFS model reference. 12 The analyst launches the discovered data link in the portal as KML and loads to Google Earth. 13 Analyst selects Temperature (Surface) and Precipitation (Total) to see what the forecast conditions are, including the ability to identify and then animate a parameter over a selected range of time. 14 Analyst can save these changing feed references for quick and updated reload in the future. 7. Exploit Data Visually and Analytically 7. Exploit Data Visually and Analytically 6. Interact with Services 7. Exploit Data Visually and Analytically 7. Exploit Data Visually and Analytically 7. Exploit Data Visually and Analytically 4. Search for Resources 5. Present Services and Alerts 6. Interact with Services 4. Search for Resources 5. Present Services and Alerts 6. Interact with Services 7. Exploit Data Visually and Analytically 3.4 Post-Conditions For each of the three sub-scenarios, the end-state is that the user is able to interact with a visualization of the data, largely achieved through applying an OGC KML wrapper around either a point location-based data series (station climatology database) or around model-driven OGC Web Map Services. This broadens the scope of accessibility beyond a limited number of specialized WMS clients and to the broader reach of KML viewers and globes. This means to visualize and discover data in a globe viewer is especially valuable in the polar regions where there is Page 6

7 systematic distortion of the landscape in more traditional cylindrical projections that conform at lower latitudes. 3.5 Special Requirements 4. System Model of the Scenario Figure 1 System Model of the Scenario 5. Implementation 5.1 Deployed Components 1) Current Weather Forecast WMS and WCS interface (UCAR) The University Corporation for Atmospheric Research (UCAR) Unidata program has implemented a THREDDS Data Server (TDS) called motherlode that provides near real-time access to global weather forecast model output from the National Centers for Environmental Prediction (NCEP) using OGC Web Map Service and Web Coverage Service protocols. These models provide forecast values for multiple atmospheric parameters at regular time steps at different levels in the atmosphere a complex multidimensional solution that presents challenges for rendering and visualization. The NCEP GFS (Global Forecast System) model is run every six hours. A python program was written to develop an OGC KML wrapper to name and organize selected global weather forecast outputs into the folder metaphor by parameter and time. This organizational approach enables KML animation of the time series data through repeated WMS calls for a global image for each time stop and parameter combination, on-the-fly. This solution allows casual users to view symbolized atmospheric phenomena globally (including coverage in the arctic region) in time series using KML clients and enabling display with additional data layers for context. Additionally, the model outputs are, of course, available as OGC WMS layers and WCS coverages for interaction by other clients. The model parameters (layers) were harvested into a CSW catalogue that Page 7

8 was registered in the GEOSS CSR for promotion through Clearinghouse. The multi-dimensional and multitemporal issues present a continuing challenge in describing and discovering dynamic weather data. Two specific challenges were revealed by this exercise. First, the real-time nature of data on the motherlode TDS illustrates the need for a means of specifying time relative to the present rather than specific absolute times. Such a facility is needed both in the WMS/WCS protocols as well as in KML. The lack of such a capability has made it necessary to generate a set of KML scripts on a regular schedule rather than on an as-requested schedule. The second issue is the sheer volume and volatile nature of the data served by motherlode. Harvesting metadata for service in catalogs can generate huge XML files that are, in fact, out of date by the time the harvesting is complete. This points to a need to be able to harvest discovery metadata at a collection level rather than at the individual dataset level. But that means the client has to be able to drill down to be able to get to individual datasets. This is often referred to as the granularity issue and is solved in the case of the TDS by a hierarchical set of catalogs of catalogs that is, in effect, a remote file system. 2) KML to wrap the Intergovernmental Panel on Climate Change (IPCC) Third Assessment Report long-term climate model WMS services (GMU) Long-term climate model data published with the IPCC Third Assessment Report were available through Web Map Coverage (WMS) service interface operated by the IPCC [2]. These WMS services were wrapped in KML files by the GMU team for visualization in Google Earth or other KML viewer that would create less distortion in Polar Regions. IPCC Third Assessment Report provides long-term ( ) predicted climate data parameters based on six different climate models. These climate data were available for multiple change scenarios and include up to 20 parameters per model, e.g. 200 hpa Mean Temperature, 850 hpa u-wind, Mean Temperature. They are accessible via WMS requests by specifying key values including model, scenario, time period, and parameter. One sample WMS request is listed as follows, it requests for January 200 hpa Mean Temperature data in time period based on CCSR/NIES model, SRES Scenario A2: =GetLegend&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=nies99%3Aa2%2Ftmp200& TIME= T0%3A0%3A0.0Z&SRS=EPSG%3A4326 Since the output of each WMS request is simply an image file for a specified geography (in this case, global), it could be wrapped in KML file for visualization in Google Earth or other KML viewer software as an image overlay. By specifying geographic parameters in the KML, the output of WMS could be reprojected in Google Earth onto a globe automatically. The Capabilities file of the IPCC Third Assessment Report WMS server was extracted and analyzed by Java program developed by GMU team in order to generate all possible WMS requests and wrap them into nested KML files, using the folder metaphor to organize the presentation of the models, scenarios, and parameters in a logical hierarchy, with the final images ordered and named to support time series animation in KML. Furthermore, a java servlet program was also developed by GMU team to generate legend file for each WMS request so that they could be displayed together in Google Earth. The KML file was then uploaded on one of GMU servers and registered with GEOSS Component and Service Registry (and GEOSS Clearinghouse) so that GEOSS community users could access it online. The KML file would be automatically launched on computers with Google Earth or other KML viewer software installed to recognize the KML file type. In this way, long-term animated climate data generated from IPCC Third Assessment Report could be visualized on-the-fly by user s request. Page 8

9 Figure 2 IPCC Third Assessment Report Long-Term Climate Data Visualization In Google Earth 3) Station Climatology KML from NOAA NCDC data (GMU) A CD-ROM describing the global standard climate normals was acquired from NOAA NCDC as a source for station climatology data (reference needed) [1]. This CD-ROM contained a collection of DOS files which described the global standard climate normals for over 4000 stations worldwide computed by more than 135 countries and territories. The GMU team processed the content of the CD by program and loaded the climate data into a MySQL database. A PHP program was also deployed to read data from the database and generate statistical climate graphs of each station by request. A Java program was then developed to read and generate KML file containing information about all weather stations in Arctic region. The KML file was designed to show basic information and climate graph (using PHP program) for each station. The KML file was then uploaded on one of GMU servers and registered with GEOSS Component and Service Register System so that GEOSS community users could access it online. The KML file would be loaded automatically on computers with Google Earth or other KML viewer software installed. In this way, historical weather station climatology in Arctic region can be visualized on-the-fly by user s request. Page 9

10 NOAA Data CD Parse Google Earth Load into Station Climatology KML Generate KML Java Program Generate Database Insertion SQL Commands Generate Analysis graph References PHP Query Station Climatology Record Database Ingested into Figure 3a System Workflow for Generating Station Climatology KML from NOAA NCDC CD at CSISS/GMU Figure 3b WMO Weather Stations of Canada Showing in Google Earth Page 10

11 Figure 3c Station Climatology KML showing in Google Earth (Lynn Lake, MB, Canada) 5.2 Use of the GCI The following GCI components are directly utilized in this scenario: Component and Service Registry (CSR) for community registration of the KML resources, the GEOSS Clearinghouse as search engine, and the GEO Portal to provide the user interface for search. A deficiency in the GEO Portal implementation was noted in the search and presentation of suitable metadata. In all cases, a familiar resource format (KML) was being sought for presentation and access. Although there are multiple URLs in the underlying metadata, the portal presentation of the basic record only showed the information link. It was discovered that if resources were registered as services in the CSR, that this actionable link would not be presented to the user except through the Full Metadata view. To improve usability, the recognition that a resource has a URL that could be acted upon drawn on a map or globe, available to launch a desktop application should be embedded in the viewer result as links, buttons, or similar options. Otherwise, most users would not know what to do with the one or more links provided. It is recommended that the Portal provide the means to detect WMS, KML, and similar resource types, and offer to render them or pass them along to helper applications on the desktop. 5.3 Demonstrations The scenario was described in a six-minute video linked from the following page as Biodiversity: Arctic SDI : Page 11

12 5.4 Future plans for deployment All three components and the data services that lie behind them (IPCC WMS and UNIDATA Motherlode THREDDS services) are intended to stay online indefinitely. These resources have been registered as Components and Services in the GEOSS for general public access. There are no restrictions on the use of these visualizations. In addition to the visualization services, actual data services are available for IPCC data as model data downloads and for the general climate models hosted by UNIDATA, available as netcdf via OGC WCS. 6. Discussions and Lessons Learned The first issue is a need in access service interfaces to specify time relative to the present so that a client can make requests for the "latest" or "last ten" instances of a given type of data. In the case of forecast model output, it is often useful to request data that applies in the future, e.g., the forecast temperature for the next 48 hours. The lack of such a capability makes it difficult or impossible to get the required data from a server which is making available hundreds of new data products each second (weather radar for example). This gap also makes it cumbersome to create automated scripts that request the latest real-time maps. This came to light in our efforts to create KML scripts that request maps of the latest weather data from WMS servers for display in Google Earth. A second difficulty is sometimes referred to as the "granularity issue." In cases where there are large collections of datasets on a server, it has been known for sometime that all the OGC "GetCapabilities" requests generate long lists of available layers, coverages, observations, and features. The lists can be so long as to be of little practical use. At one time, it was thought that using catalogs would be the solution, but our experience has been that this just shifts the problem into the domain of the catalog system which has to harvest the long lists and then serves them up as responses to search queries. A facility partitioning these long lists into groups, or files, or folders, or collections, or categories is needed. In addition, a mechanism is needed for drilling down from a collection level entry to the actual dataset level where the data may be retrieved by the client. Unfortunately, the concept of "collections" was left out of the most recent proposed version of the Observations and Measurements specs. At some point, this needs to be revisited. From a deployment point of view, the visualization analysis of near real-time weather conditions and forecasts presented some computational challenges to the selected solution. The value of the model interface is in the diverse number of parameters and time steps available to the end user for multiple time runs of the models. However, one side effect is that rendered snapshots of the selected variables must be generated on demand for the full period of interest to support the animation function in Google Earth. Because these views are generated on-demand, the inability to deliver these rapidly often caused the user to have to run the loop several times in order to fetch all time slices. This side-effect is not a problem with the standards, per se, but of the underlying server software being able to respond quickly, to pre-cache, or otherwise anticipate the request for animation generation. This also indicates one possible upgrade on the Google Earth side to enable the success retrieval of each KML when presenting the series as an animation. 7. References [1] Global Climate Normals [2] IPCC Data Distribution Center Visualisation. [3] UNIDATA Motherlode Data Server Page 12