Scientific Data Management and Dissemination

Size: px

Start display at page:

Download "Scientific Data Management and Dissemination"

Jerome McCoy
9 years ago
Views:

1 Federal GIS Conference February 9 10, 2015 Washington, DC Scientific Data Management and Dissemination John Fry Solution Engineer, Esri [email protected]

2 Agenda Background of Scientific Data Management through the Mosaic Dataset Raster Functions Sharing with ArcGIS Server - Image Services - WMS Consuming - WebMaps - WebApps Questions

3 Communicating Science to Decision Makers Winds in the Area of Interest will be sustained at 20kts with gusts to 35kts. ArcGIS Waves in the Area of Interest will between 2.5 m and 3 m. What does that mean??? You can conduct X operation at these locations, but not at these locations. Will my mission be a success? What platform can I use?

4 Communicating Science to Decision Makers Reduction in upwelling and mixing not beneficial in these locations. Stratification occurring at these locations ArcGIS What does that mean??? Will it be too costly to fix? What techniques should I use? Pollutant runoff in surrounding regions needs to be addressed Fish yields will be low in these areas

Demo Visualize a final product -http://dtc-sci01.esri.

5 Demo Visualize a final product - Esri FedGIS 2015 Scientific Data Management and Dissemination

6 Scientific Data Oceanographic data Sea temperature Salinity Ocean current Meteorological data Temperature Humidity Wind Land Soil moisture NDVI Land cover

7 1. Latitude (y) 2. Longitude (x) 2D

8 1. Latitude (y) 2. Longitude (x) 3. Height (z) 3D

9 1. Latitude (y) 2. Longitude (x) 3. Time (t) 3D

10 1. Latitude (y) 2. Longitude (x) 3. Depth (d) 4. Time (t) 4D

Scientific formats in ArcGIS 10.3 GRIB (General Regularly-distributed Information in Binary form) - A gridded format used operationally worldwide by most meteorological centers.

11 Scientific formats in ArcGIS 10.3 GRIB (General Regularly-distributed Information in Binary form) - A gridded format used operationally worldwide by most meteorological centers. netcdf (Network Common Data Form) - A self-describing, machine-independent data format that support the creation, access, and sharing of array-oriented scientific data. HDF (Hierarchical Data Format) - A scientific format that supports a proliferation of different data models, including multidimensional arrays, raster images, and tables. Variable / Dimension Temperature / Time Salinity / Depth Pressure / Altitude

Fundamentals of netcdf data storage ArcGIS Help - Fundamentals of netcdf data storage Three-dimensional (3D) data, like temperature over an area varying with time, or

12 Fundamentals of netcdf data storage ArcGIS Help - Fundamentals of netcdf data storage Three-dimensional (3D) data, like temperature over an area varying with time, or four-dimensional (4D) data, like temperature over an area varying with time and altitude, is stored as a series of twodimensional arrays. NetCDF Conventions - Unidata

13 CF Convention Climate and Forecast (CF) Convention - Initially developed for: - Climate and forecast data - Atmosphere, surface and ocean model-generated data - Also for observational datasets CF is now the most widely used conventions for geospatial netcdf data. Current version 1.6 According to CF conventions, variables representing time must always explicitly include the units attribute. You can use Compliance checker utility to check a netcdf file. CF-Convention Compliance Checker for netcdf Format

14 N-dimensional capability in ArcGIS 10.3 Visualization Observations Modeling Processing on-the-fly Raster Functions Dissemination Mosaic Data Set Server Native METOC Data Formats Exploitation netcdf GRIB HDF ArcGIS Desktop

15 Raster Mosaic Dataset for N-dimensional Data (ArcGIS 10.3) Resolves Traditional Raster Data Management Issues Processing Time Reduced exponentially Overlapping Imagery Pick Best raster Storage Reduces storage by removing redundancy Multi-resolution Data No need to sample up or down Disparate Datasets Handle large NoData areas Maintenance Add Rasters as required Data Quality Reduces resampling Maintain Metadata Appended directly upon registration

on-the-fly processing Serve as Multidimensional o Image Service o Map

16 Multidimensional Mosaic Datasets Define variables when adding Rasters Each Row is a 2D Raster with variables and dimension values Define on-the-fly processing Serve as Multidimensional o Image Service o Map Service o WMS Aggregate (mosaic) spatial, time, and vertical dimensions

17 Mosaic Dataset - Visualization Visualize variables at any time and vertical dimension Visualize flow direction and magnitude variables Visualize temporal change of a variable

18 Multidimension Tools in ArcGIS Multidimension Tools Available with ArcGIS for Desktop Multidimension Supplemental Tools Download from ArcGIS Online

19 Demo OpenDap to NetCDF tool Describe the multidimensional dataset Create a Mosaic Dataset Add Rasters to Mosaic Dataset Vector Rendering of raster Esri FedGIS 2015 Scientific Data Management and Dissemination

20 Multidimensional Mosaic Dataset On-The-Fly Processing Modeling with Raster Function Templates (RFT) o o o o

21 Multidimensional Mosaic Dataset - Analysis Perform analysis using GP tools Use a slice in a Geoprocessing tool - Mosaic dataset path\objectid=2 - Mosaic dataset path\name= elev Use Make Mosaic Layer tool - Select one or set of slices - Use mosaic method to get the statistics of a set of slices - Minimum, maximum, mean

22 DEMO Adding functions to mosaic datasets Esri FedGIS 2015 Scientific Data Management and Dissemination

Publishing a WMS on ArcGIS Server (Started at 10.2.

23 Publishing a WMS on ArcGIS Server (Started at ) Enable WMS capabilities on Service Editor or Server Manager Multidimensional capability

24 From Support for serving Multi-Dimensional Data using the WMS 1.3 implementation in ArcGIS Server Use Case : The Naval Oceanographic Office provide data to operational units around the world, with many forecasts and multi-dimensional products. - Global NCOM model 60,000 2D map layers a day - Keeps a history of 3 base times worth of data - Multi-Dimensional handling makes it easier - Now with support of for TIME, ANALYSIS_TIME, and ELEVATION, number of layers could be brought down to a manageable 1200 every 18 hours.

25 WMS Get Capabilities Multi-Dimensional Support

com/esri/wmsmult idimensionalesriviewer http://www.

26 N-dimensional WMS in a Web Application Time slider Elevation slider Available on Github idimensionalesriviewer m.html?id=4d8f79b89d0046a5bf0 619decc369d22

27 Demo Sharing on ArcGIS Online WMS Viewer WebMaps StoryMaps Esri FedGIS 2015 Scientific Data Management and Dissemination

28 Things to Consider Embrace the Common Data Model (netcdf, HDF etc.) Use Data and metadata standards (OGC, CF etc) Provide mechanism so that we can access scientific data using a single set of APIs. and can expect data to be CF complainant Make your data spatial (by specifying geographic or a projected coordinate system) Clearly define workflow and requirements Create sample tools / templates where possible ArcGIS Online / Solutions Pages / GitHub

29 More Information Sessions: - Imagery and Raster Data Management and Dissemination (Monday, 9 Feb 2015, 3:00 pm 4:00pm, 101) - Imagery and Raster Data Management and Dissemination (Tuesday, 10 Feb 2015, 1:30pm - 2:30pm, Theater 2 Hall D) Tools: - Multidimension supplemental tools ArcGIS Online - CF-Convention Compliance Checker for netcdf Format - WMS Viewer - Help:

30 Federal GIS Conference February 9 10, 2015 Washington, DC Don t forget to complete a session evaluation form!

31 Federal GIS Conference February 9 10, 2015 Washington, DC Print your customized Certificate of Attendance! Printing stations located on L St. Bridge, next to registration

32 Federal GIS Conference February 9 10, 2015 Washington, DC GIS Solutions EXPO, Hall D Monday, 12:30pm 6:30pm Tuesday, 10:45 AM 4:00 PM Exhibitors Hands-On Learning Lab Technical & Extended Support Demo Theater Esri Showcase

33 Federal GIS Conference February 9 10, 2015 Washington, DC Networking Reception: National Museum of American History Tuesday, 6:30 PM 9:30 PM Bus Pickup located on L Street

34 Federal GIS Conference February 9 10, 2015 Washington, DC Interested in diving deeper into Esri technology? Add a day to your Fed GIS experience and register to attend the Esri DevSummit Washington DC. Stop by the registration counter to sign up.

PART 1. Representations of atmospheric phenomena

PART 1. Representations of atmospheric phenomena PART 1 Representations of atmospheric phenomena Atmospheric data meet all of the criteria for big data : they are large (high volume), generated or captured frequently (high velocity), and represent a