Federal GIS Conference February 9 10, 2015 Washington, DC Scientific Data Management and Dissemination John Fry Solution Engineer, Esri jfry@esri.com
Agenda Background of Scientific Data Management through the Mosaic Dataset Raster Functions Sharing with ArcGIS Server - Image Services - WMS Consuming - WebMaps - WebApps Questions
Communicating Science to Decision Makers Winds in the Area of Interest will be sustained at 20kts with gusts to 35kts. ArcGIS Waves in the Area of Interest will between 2.5 m and 3 m. What does that mean??? You can conduct X operation at these locations, but not at these locations. Will my mission be a success? What platform can I use?
Communicating Science to Decision Makers Reduction in upwelling and mixing not beneficial in these locations. Stratification occurring at these locations ArcGIS What does that mean??? Will it be too costly to fix? What techniques should I use? Pollutant runoff in surrounding regions needs to be addressed Fish yields will be low in these areas
Demo Visualize a final product -http://dtc-sci01.esri.com/deadzonestorymap/ Esri FedGIS 2015 Scientific Data Management and Dissemination
Scientific Data Oceanographic data Sea temperature Salinity Ocean current Meteorological data Temperature Humidity Wind Land Soil moisture NDVI Land cover
1. Latitude (y) 2. Longitude (x) 2D
1. Latitude (y) 2. Longitude (x) 3. Height (z) 3D
1. Latitude (y) 2. Longitude (x) 3. Time (t) 3D
1. Latitude (y) 2. Longitude (x) 3. Depth (d) 4. Time (t) 4D
Scientific formats in ArcGIS 10.3 GRIB (General Regularly-distributed Information in Binary form) - A gridded format used operationally worldwide by most meteorological centers. netcdf (Network Common Data Form) - A self-describing, machine-independent data format that support the creation, access, and sharing of array-oriented scientific data. HDF (Hierarchical Data Format) - A scientific format that supports a proliferation of different data models, including multidimensional arrays, raster images, and tables. Variable / Dimension Temperature / Time Salinity / Depth Pressure / Altitude
Fundamentals of netcdf data storage ArcGIS Help - Fundamentals of netcdf data storage Three-dimensional (3D) data, like temperature over an area varying with time, or four-dimensional (4D) data, like temperature over an area varying with time and altitude, is stored as a series of twodimensional arrays. NetCDF Conventions - Unidata
CF Convention Climate and Forecast (CF) Convention - http://cfconventions.org/ Initially developed for: - Climate and forecast data - Atmosphere, surface and ocean model-generated data - Also for observational datasets CF is now the most widely used conventions for geospatial netcdf data. Current version 1.6 According to CF conventions, variables representing time must always explicitly include the units attribute. You can use Compliance checker utility to check a netcdf file. CF-Convention Compliance Checker for netcdf Format
N-dimensional capability in ArcGIS 10.3 Visualization Observations Modeling Processing on-the-fly Raster Functions Dissemination Mosaic Data Set Server Native METOC Data Formats Exploitation netcdf GRIB HDF ArcGIS Desktop
Raster Mosaic Dataset for N-dimensional Data (ArcGIS 10.3) Resolves Traditional Raster Data Management Issues Processing Time Reduced exponentially Overlapping Imagery Pick Best raster Storage Reduces storage by removing redundancy Multi-resolution Data No need to sample up or down Disparate Datasets Handle large NoData areas Maintenance Add Rasters as required Data Quality Reduces resampling Maintain Metadata Appended directly upon registration
Multidimensional Mosaic Datasets Define variables when adding Rasters Each Row is a 2D Raster with variables and dimension values Define on-the-fly processing Serve as Multidimensional o Image Service o Map Service o WMS Aggregate (mosaic) spatial, time, and vertical dimensions
Mosaic Dataset - Visualization Visualize variables at any time and vertical dimension Visualize flow direction and magnitude variables Visualize temporal change of a variable
Multidimension Tools in ArcGIS Multidimension Tools Available with ArcGIS for Desktop Multidimension Supplemental Tools Download from ArcGIS Online
Demo OpenDap to NetCDF tool Describe the multidimensional dataset Create a Mosaic Dataset Add Rasters to Mosaic Dataset Vector Rendering of raster Esri FedGIS 2015 Scientific Data Management and Dissemination
Multidimensional Mosaic Dataset On-The-Fly Processing Modeling with Raster Function Templates (RFT) o o o o
Multidimensional Mosaic Dataset - Analysis Perform analysis using GP tools Use a slice in a Geoprocessing tool - Mosaic dataset path\objectid=2 - Mosaic dataset path\name= elev Use Make Mosaic Layer tool - Select one or set of slices - Use mosaic method to get the statistics of a set of slices - Minimum, maximum, mean
DEMO Adding functions to mosaic datasets Esri FedGIS 2015 Scientific Data Management and Dissemination
Publishing a WMS on ArcGIS Server (Started at 10.2.1) Enable WMS capabilities on Service Editor or Server Manager Multidimensional capability
From Support for serving Multi-Dimensional Data using the WMS 1.3 implementation in ArcGIS Server 10.2.1 Use Case : The Naval Oceanographic Office provide data to operational units around the world, with many forecasts and multi-dimensional products. - Global NCOM model 60,000 2D map layers a day - Keeps a history of 3 base times worth of data - Multi-Dimensional handling makes it easier - Now with support of for TIME, ANALYSIS_TIME, and ELEVATION, number of layers could be brought down to a manageable 1200 every 18 hours.
WMS Get Capabilities Multi-Dimensional Support
N-dimensional WMS in a Web Application Time slider Elevation slider Available on Github https://github.com/esri/wmsmult idimensionalesriviewer http://www.arcgis.com/home/ite m.html?id=4d8f79b89d0046a5bf0 619decc369d22
Demo Sharing on ArcGIS Online WMS Viewer WebMaps StoryMaps Esri FedGIS 2015 Scientific Data Management and Dissemination
Things to Consider Embrace the Common Data Model (netcdf, HDF etc.) Use Data and metadata standards (OGC, CF etc) Provide mechanism so that we can access scientific data using a single set of APIs. and can expect data to be CF complainant Make your data spatial (by specifying geographic or a projected coordinate system) Clearly define workflow and requirements Create sample tools / templates where possible ArcGIS Online / Solutions Pages / GitHub
More Information Sessions: - Imagery and Raster Data Management and Dissemination (Monday, 9 Feb 2015, 3:00 pm 4:00pm, 101) - Imagery and Raster Data Management and Dissemination (Tuesday, 10 Feb 2015, 1:30pm - 2:30pm, Theater 2 Hall D) Tools: - Multidimension supplemental tools ArcGIS Online - CF-Convention Compliance Checker for netcdf Format - WMS Viewer - https://github.com/esri/wmsmultidimensionalesriviewer Help: http://desktop.arcgis.com/en/desktop/latest/manage-data/netcdf/what-is-netcdfdata.htm
Federal GIS Conference February 9 10, 2015 Washington, DC Don t forget to complete a session evaluation form!
Federal GIS Conference February 9 10, 2015 Washington, DC Print your customized Certificate of Attendance! Printing stations located on L St. Bridge, next to registration
Federal GIS Conference February 9 10, 2015 Washington, DC GIS Solutions EXPO, Hall D Monday, 12:30pm 6:30pm Tuesday, 10:45 AM 4:00 PM Exhibitors Hands-On Learning Lab Technical & Extended Support Demo Theater Esri Showcase
Federal GIS Conference February 9 10, 2015 Washington, DC Networking Reception: National Museum of American History Tuesday, 6:30 PM 9:30 PM Bus Pickup located on L Street
Federal GIS Conference February 9 10, 2015 Washington, DC Interested in diving deeper into Esri technology? Add a day to your Fed GIS experience and register to attend the Esri DevSummit Washington DC. Stop by the registration counter to sign up.