Real-time and Archived Antarctic. Interactive Processing Tools

Size: px

Start display at page:

Download "Real-time and Archived Antarctic. Interactive Processing Tools"

Olivia Stafford
10 years ago
Views:

1 Real-time and Archived Antarctic Meteorological Data via a Synergy of Interactive Processing Tools Mark W. Seefeldt Department of Atmospheric and Oceanic Sciences University it of Colorado at Boulder, Boulder, CO Tom Yoksas Unidata Program Center, UCAR Community Programs University Corporation for Atmospheric Research, Boulder, CO Matthew A. Lazzara Antarctic Meteorological Research Center, Space Science and Engineering Center University of Wisconsin Madison, Madison, WI

Unidata Program Center, UCAR Community Programs University Corporation for Atmospheric Research, Boulder, CO Matthew A.

2 Overview Outline Real-time Data Collection Distribution: Antarctic-IDD / Local Data Manager (LDM) Servers: RAMADDA THREDDS Data Server (TDS) Abstract Data Distribution Environment (ADDE) Interactive Processing and Display: Integrated Data Viewer (IDV) McIDAS-V McIDAS-X

Data Server (TDS) Abstract Data Distribution Environment (ADDE)

3 Overview During the last 15+ years the availability of real-time meteorological data over Antarctica has been steadily increasing

4 Overview During the last 5 years a community has been established to assist in the dissemination of the real-time data using a unified network The backbone of this network, the Antarctic-IDD, uses Unidata s Local Data Manager (LDM) data sharing software Current efforts aim at employing additional Unidata tools to provide free access to the data to the broader Meteorological and Earth science community

Unidata s Local Data Manager (LDM) data sharing software Current efforts aim at employing additional

5 Internet Data Distribution Systems Project-specific IDDs: North American IDD IDD-Brasil Antarctic-IDD US National Weather Service NEXRAD Level II & III SURA Coastal Ocean Observing and Prediction (SCOOP) Linked Environments for Atmospheric Discovery y( (LEAD) THORPEX Interactive Grand Global Ensemble (TIGGE) Demonstrated bi-directional relay of 8.4 GB/hr between NCAR/SCD and ECMWF for three week period in January Peak relay of 25+ GB/hr from ECMWF to NCAR/SCD Demonstrated ability to relay small ( KB) and large (10-60 MB) products with very low latencies Governments of South Korea, Spain,... NASA, EPA, Private Industry, Source LDM LDM LDM Source LDM LDM Internet LDM LDM LDM Source LDM

of 8.4 GB/hr between NCAR/SCD and ECMWF for three week period in January Peak relay of 25+ GB/hr from ECMWF to NCAR/SCD Demonstrated ability to relay small (100-300 KB) and

6 Primary Participants NCAR-MMM Antarctic Mesoscale Prediction System (AMPS) Kevin Manning (and Jordan Powers) University of Wisconsin-Madison Antarctic Meteorology Research Center (AMRC) Matthew Lazzara Ohio State University Byrd Polar Research Center (BPRC) Dan Steinhoff University of Colorado at Boulder Melissa Richards (and Mark Seefeldt) SPAWAR System Center Charleston Robert Vehorn (and others ) Columbia (National) Scientific Balloon Facility (CSBF) Robert Mullenax Unidata Program Center/ UCAR (Unidata) Tom Yoksas

(BPRC) Dan Steinhoff University of Colorado at Boulder Melissa Richards (and Mark Seefeldt) SPAWAR System Center Charleston Robert

7 Data Available in the Antarctic-IDD Numerical Weather Prediction: NCAR-MMM Antarctic Mesoscale Prediction System (WRF) GRIB files for each domain, PNG files of pre-generated displays - AMRC NOAAPort-delivered GFS model fields in McIDAS GRID format Satellite Observations: AMRC Antarctic composite (infra-red, visible, water vapor) McIDAS AREA files, JPG images (some polar orbiting satellite) Surface / Upper Air Observations: AMRC Automatic Weather Station Observations text files from retrieval sites, Meteorological Data (MD) files (raob, ship, airep, metar, etc, etc.) AMRC station observations and upper-air plots, GIF images SPAWAR - McMurdo AWS and air field observations Other data to be included? (Always improving )

images (some polar orbiting satellite) Surface / Upper Air Observations: AMRC Automatic Weather Station Observations text files from retrieval sites, Meteorological Data (MD) files

8 Antarctic-IDD Ad data distribution tib ti network kfor operations and research Employs Unidata s Local Data Manger (LDM software) Upwards of 1.3 GB/hr flowing through the LDM EXP feed

9 Antarctic-IDD The Antarctic-IDD ti was established in 2005 A collection of successes and failures have been faced Firewall, security, and hardware issues limited the availability of data from NCAR-MMM Recent access through h Unidata s motherlode system has dramatically increased the capability of distribution The current topology is relatively limited with two critical point of failure locations (Unavoidable sources are limits) Products from SPAWAR have been hindered by security and firewall issues Products to and from Antarctica are often limited by the Internet bandwidth to/from McMurdo (Security.)

distribution The current topology is relatively limited with two critical point of failure locations (Unavoidable sources are limits) Products from

10 Antarctic-IDD Interested new partners can easily be added to the Antarctic-IDD data sharing network contacts: Matthew Lazzara: Tom Yoksas: General Information: Provision of web-based data access services Once the link to NCAR-MMM was established, Unidata implemented human-interactive ti and programmatic access to the suite of data available in the Antarctic-IDD data through its RAMADDA and THREDDS Data Server technologies Current efforts aim at employing additional Unidata tools to provide free access to the data to the broader community

edu General Information: http://amrc.ssec.wisc.edu/antidd.html http://www.unidata.ucar.

11 Repository for Archiving, Managing and Accessing Diverse DAta (RAMADDA) Antarctic-IDD data are currently being made available via RAMADDA on the Unidata demonstration data server motherlode ( th d / RAMADDA: a new development effort in Unidata a Java-base server that runs under Tomcat or can be run as a standalone application provides a publishing platform and content management system for Earth science and Meteorological data implements a front end to THREDDS Data Server functionality RAMADDA provides new opportunities for data access: preview functions collections search facility federated servers provide transparent access to geographicallydistributed data holdings

edu/repository) th d / RAMADDA: a new development effort in Unidata a Java-base server that runs under Tomcat or can be run as a standalone application provides a publishing

12 Thematic Real-time Environmental Distributed Data Services (THREDDS) RAMADDA provides data access through THREDDS Data Server infrastructure THREDDS Data Server: a web-based server which provides metadata and data access uses a variety of data access protocols including OPeNDAP, McIDAS ADDE, and HTTP developed, distributed and supported by Unidata written in Java and easily implemented by the Tomcat server Free and open access to the data is now available to users around the world using RAMADDA/THREDDSenabled applications: Integrated Data Viewer (IDV, Unidata) McIDAS-V (McV, U. Wisconsin/SSEC)

HTTP developed, distributed and supported by Unidata written in Java and easily implemented by the Tomcat server Free and open access to the data is

13 RAMADDA The Antarctic-IDD data collection can be accessed from Unidata s prototype RAMADDA server:

14 LDM + RAMADDA Data products are filed by the LDM and cataloged by RAMADDA to include basic metadata information:

metadata information: http://motherlode.ucar.

15 RAMADDA Preview/browse is provided for a variety of data types including model output, satellite imagery and observations:

imagery and observations: http://motherlode.ucar.

16 Data Analysis and Display via the Integrated Data Viewer (IDV) and McIDAS-V (McV) A Java based software framework for analyzing and visualizing geoscience data (based on VisAD) Provides the ability to display and work with: satellite imagery, radar data gridded model output surface and upper-air observations and much more Can create a variety of displays: 2-D horizontal contours and color-filled contours 3-D iso-surfaces vertical cross sections interactive data probing and much more

output surface and upper-air observations and much more Can create a variety of displays: 2-D horizontal contours and color-filled contours

17 Integrated Data Viewer (IDV) & McIDAS-V (McV) Antarctic-IDD data can be accessed directly by IDV & McV through RAMADDA, TDS and McIDAS ADDE services

18 IDV and McV Analysis/visualization of remote data holdings make exploration simple, quick and transparent Antarctic-IDD data access is built-in IDV (McV?) Can publish data/display on a RAMADDA server

19 Summary Real-time Antarctic meteorology data are available as a part of the Antarctic-IDD network: AMRC satellite imagery, AMPS model output, surface and upper-air observations RAMADDA plus THREDDS Data Server & McIDAS ADDE services provide remote programmatic access real-time Antarctic-IDD data Integrated Data Viewer and McIDAS-V can be used to visualize and analyze the real-time Antarctic meteorology data provided in the Antarctic-IDD AMRC plans a RAMADDA server (with SSEC Data Center) Questions Mark Seefeldt [email protected] Tom Yoksas [email protected] Matthew Lazzara [email protected]

Data Viewer and McIDAS-V can be used to visualize and analyze the real-time Antarctic meteorology data provided in the Antarctic-IDD AMRC plans a RAMADDA

13.2 THE INTEGRATED DATA VIEWER A WEB-ENABLED APPLICATION FOR SCIENTIFIC ANALYSIS AND VISUALIZATION

13.2 THE INTEGRATED DATA VIEWER A WEB-ENABLED APPLICATION FOR SCIENTIFIC ANALYSIS AND VISUALIZATION Don Murray*, Jeff McWhirter, Stuart Wier, Steve Emmerson Unidata Program Center, Boulder, Colorado 1.