Insights from 40 years of data services at NCAR s Research Data Archive. Douglas Schuster and Steven Worley
|
|
|
- Kathleen Stevenson
- 7 years ago
- Views:
Transcription
1 Insights from 40 years of data services at NCAR s Research Archive Douglas Schuster (schuster@ucar.edu) and Steven Worley (worley@ucar.edu)
2
3 Highlights What is the? Evolution of Services User Registration Policies Future Directions
4 What is the? Purposes Support climate & weather research at NCAR and UCAR Universities Collections Ocean & atmosphere observations, analyses, reanalyses, operational NWP outputs Basic Metrics Established in 1960s 600+ datasets, 8M files, 1.8 PB +70 datasets growing daily monthly 4
5 What is the? Science educated staff Expert consultants and data engineers Free and open access
6 Evolution of services General Goals Securely maintain community reference information for a long period Facilitate easy discovery of holdings Enable simple access to holdings Provide the community with the capability to build upon or reproduce the research of others 6
7 Evolution of services 1960s to early 1990s Specialist Driven / Storage Discovery Delivery 7
8 Evolution of services Mid 1990s to Early 2000s Specialist Driven / Storage Discovery Delivery 8
9 Evolution of services Mid 2000s to Present Driven / Storage Discovery Delivery 9
10 Current Services External Users ~10K/year Geoportal! Control and DB Internal Users ~100s/year Download GCMD Journal Article Faceted Search Tools OPeNDAP, WCS, WMS OAI-PMH! Web Interface and Services THREDDS Web and Server Processing Central File System 500 TB of set NCAR/CISL High Performance Computing Command line apps Software HPSS Tape Library 1.8 PB of set Legend /User Interactions
11 Discovery Pathways External Users ~10K/year Geoportal! Control and DB Internal Users ~100s/year GCMD OAI-PMH! Distributed search Web and Server NCAR/CISL High Performance Computing Download Central File System 350 TB of set HPSS Tape Library 1.2 PB of set Legend /User Interactions
12 Discovery Pathways External Users ~10K/year Download Journal/DOI based discovery Journal Article Web Interface and Services Web and Server Control and DB Central File System 350 TB of set NCAR/CISL High Performance Computing HPSS Tape Library 1.2 PB of set Legend /User Interactions Internal Users ~100s/year
13 Discovery Pathways -Server Generated Citation
14 Discovery Pathways External Users ~10K/year Download In-house developed web interface search Faceted Search Web Interface and Services Web and Server Control and DB Central File System 350 TB of set NCAR/CISL High Performance Computing HPSS Tape Library 1.2 PB of set Legend /User Interactions Internal Users ~100s/year
15 Discovery Pathways THREDDS catalog search
16 Discovery Pathways -Advise User of Best set Collection(s) to Support Needs
17 Pathways External Users ~10K/year Download Web interface/service data request submission and data download Web Interface and Services Tools Web and Server Processing Control and DB Central File System 500 TB of set NCAR/CISL High Performance Computing HPSS Tape Library 1.8 PB of set Legend /User Interactions Internal Users ~100s/year
18 Pathways External Users ~10K/year Control and DB Internal Users ~100s/year Remote data analysis software access through TDS service endpoints Download OPeNDAP, WCS, WMS THREDDS Web and Server Processing Central File System 500 TB of set NCAR/CISL High Performance Computing HPSS Tape Library 1.2 PB of set Legend /User Interactions
19 Pathways -Unidata TDS set Collection Virtual Aggregations Issue: Clients lack authentication support User Made Products Client Tools TDS 500 TB on Disk Plots Statistical Analyses Unidata IDV NASA Panoply Gridded set Collection Virtual Timeseries Aggregation File Gridded set Collection GRIB1 File GRIB1 File GRIB1 File netcdf File GRIB2 File GRIB2 File GRIB2 File s netcdf File netcdf File NetCDF Output files Spreadsheets NCAR Command Language Web Browser Observation set Collection Virtual Timeseries Aggregation File In-Situ Observation set Collection BUFR File BUFR File BUFR File Operational Under Development
20 Pathways External Users ~10K/year Download OPeNDAP, WCS, WMS THREDDS Web and Server Control and DB Direct data analysis on HPC systems Central File System 500 TB of set NCAR/CISL High Performance Computing Software HPSS Tape Library 1.8 PB of set Legend /User Interactions Internal Users ~100s/year
21 Future Pathways External Users ~10K/year Download OPeNDAP, WCS, WMS Control Direct data analysis Private and DB Cloud for select external users? THREDDS Web and User ServerConfigured VM Central File System 500 TB of set NCAR/CISL Private Cloud HPSS Tape Library 1.8 PB of set Legend /User Interactions Internal Users ~100s/year
22 Format management and usage Archive files stored in native data formats GRIB-1, GRIB-2, netcdf, HDF, BUFR, various binary and ASCII formats Archive may be restructured, depending on end-user research needs Time-series for single variable, or synoptic-time arrangement Climate research versus weather research preference! Prefer on-demand format conversion over maintaining datasets in multiple formats Offer format transformation as an output option Leverage protocols (e.g. OPeNDAP) that make data-format transparent to the end user
23 Current Services Overview Usage metrics In PB (virtual) processing 4000 users received data subsetting and format conversion services 45K requests were processed and 380 TB of data delivered 7300 users downloaded 750TB of data from archive files Metrics are now being collected to determine user data discovery pathways and TDS use
24 Policies (User Registration, /Use, Licensing, Limits, T&Cs) User Profile provided during registration ( only) address First and Last Name Organization Country Agree to UCAR Corporate Terms of Use Special Authorizations (access to restricted datasets) Implement requirements per third party policy agreements Enable to hold and serve unique research products Analyze metrics as a basis for scientific impact assessment and economic justification Free and open access to all archive holdings We reserve the right to request payment Why?
25 Policies (User Registration, /Use, Licensing, Limits, T&Cs) Future Might instantiate access control to services in tiered levels dependent on resource strain. Highest priority for NCAR and NSF supported education and research Broadly support all US users Use unused capacity for world-wide service
26 Future Directions Improve data access General GLOBUS connect access for all holdings Expand OPeNDAP and OGC based access ESGF integration of archives with climate model data brokering e.g. distributed satellite data matchup service with marine surface observations (ICOADS) Improve data discovery More data journal articles and DOIs Additional metadata standards available through harvestable endpoints ISO Use web standards (e.g. defined by Schema.org) to support discoverability by commercial search engines Support single-sign-on identity management Tutorial videos to enhance consulting with user forum (,YouTube, edx.org) More hosted data processing Improved multi-node/mpi data preparation Sub-setting, format conversion, customized file structures Private tiered cloud structure for non-hpc users analysis (analytics, regridding, user code execution environments) Commercial partnerships public cloud Focus on new communities general science interested public, more visualization,
27 Questions? Doug Schuster
The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team
The Arctic Observing Network and its Data Management Challenges Florence Fetterer (NSIDC/CIRES/CU), James A. Moore (NCAR/EOL), and the CADIS team Photo courtesy Andrew Mahoney NSF Vision What is AON? a
The THREDDS Data Repository: for Long Term Data Storage and Access
8B.7 The THREDDS Data Repository: for Long Term Data Storage and Access Anne Wilson, Thomas Baltzer, John Caron Unidata Program Center, UCAR, Boulder, CO 1 INTRODUCTION In order to better manage ever increasing
NOAA Environmental Data Management Update for Unidata SAC 2014-10-08
NOAA Environmental Management Update for Unidata SAC 2014-10-08 Jeff de La Beaujardière, PhD NOAA Management Architect +1 301-713-7175 NOAA data are unique, valuable, and irreplaceable Many observing systems:
PART 1. Representations of atmospheric phenomena
PART 1 Representations of atmospheric phenomena Atmospheric data meet all of the criteria for big data : they are large (high volume), generated or captured frequently (high velocity), and represent a
Outcomes of the CDS Technical Infrastructure Workshop
Outcomes of the CDS Technical Infrastructure Workshop Baudouin Raoult Baudouin.raoult@ecmwf.int Funded by the European Union Implemented by Evaluation & QC function C3S architecture from European commission
1. Overview and Status Update (Satoko) : 10min. 2. Demonstration (Yoshi) : 20min. 3. New Architecture (Yoshi): 15min. 4. Q&A, Discussion (All) : 15min
1. Overview and Status Update (Satoko) : 10min 2. Demonstration (Yoshi) : 20min 3. New Architecture (Yoshi): 15min 4. Q&A, Discussion (All) : 15min 1 CEOS Water Portal Overview and Status Update WGISS-37
Integrated Data Viewer (IDV) a visualization framework Yuan Ho Unidata Program Center Boulder, CO Presentation Outline Integrated Data Viewer (IDV) overview The IDV features IDV examples and customized
Data-Intensive Science and Scientific Data Infrastructure
Data-Intensive Science and Scientific Data Infrastructure Russ Rew, UCAR Unidata ICTP Advanced School on High Performance and Grid Computing 13 April 2011 Overview Data-intensive science Publishing scientific
WCS 2.0 Extension for netcdf-cf; CSW and THREDDS Interoperability
GEOSS Workshop XLIII: Sharing Climate Information & Knowledge NCAR Center Green, Boulder Colorado, 23 Sept 2011, 8:00am 5:30pm WCS 2.0 Extension for netcdf-cf; CSW and THREDDS Interoperability S.Nativi
Advancement of the NOMADS for Observational Data and Model Intercomparisons and the Establishment of a NCDC NOMADS Team and HelpDesk
Advancement of the NOMADS for Observational Data and Model Intercomparisons and the Establishment of a NCDC NOMADS Team and HelpDesk A NOMADS Collaboration White Paper 5/5/2003 This white paper is intended
13.2 THE INTEGRATED DATA VIEWER A WEB-ENABLED APPLICATION FOR SCIENTIFIC ANALYSIS AND VISUALIZATION
13.2 THE INTEGRATED DATA VIEWER A WEB-ENABLED APPLICATION FOR SCIENTIFIC ANALYSIS AND VISUALIZATION Don Murray*, Jeff McWhirter, Stuart Wier, Steve Emmerson Unidata Program Center, Boulder, Colorado 1.
The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA
The distribution of marine OpenData via distributed data networks and Web APIs. The example of ERDDAP, the message broker and data mediator from NOAA Dr. Conor Delaney 9 April 2014 GeoMaritime, London
Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness.
Using Message Brokering and Data Mediation to use Distributed Data Networks of Earth Science Data to Enhance Global Maritime Situational Awareness. Conor Delaney, Alfredo Alessandrini & Harm Greidanus
SURFsara Data Services
SURFsara Data Services SUPPORTING DATA-INTENSIVE SCIENCES Mark van de Sanden The world of the many Many different users (well organised (international) user communities, research groups, universities,
NASA's Strategy and Activities in Server Side Analytics
NASA's Strategy and Activities in Server Side Analytics Tsengdar Lee, Ph.D. High-end Computing Program Manager NASA Headquarters Presented at the ESGF/UVCDAT Conference Lawrence Livermore National Laboratory
André Karpištšenko, Co-Founder & Chief Scientist, Marinexplore Strata, 2014.02.11
marineos André Karpištšenko, Co-Founder & Chief Scientist, Marinexplore Strata, 2014.02.11 The Ocean's Big Data Platform marineos: a platform for organizing, analyzing and distributing machine data marineos
Real-time and Archived Antarctic. Interactive Processing Tools
Real-time and Archived Antarctic Meteorological Data via a Synergy of Interactive Processing Tools Mark W. Seefeldt Department of Atmospheric and Oceanic Sciences University it of Colorado at Boulder,
CEOS Water Portal Status Update
CEOS Water Portal Status Update Satoko Horiyama MIURA / Atsushi KAWAI JAXA/Mission Operations System Office WGISS-31 Introduction CEOS Water Portal is ; A web based portal system evolved from Distributed
Nevada NSF EPSCoR Track 1 Data Management Plan
Nevada NSF EPSCoR Track 1 Data Management Plan August 1, 2011 INTRODUCTION Our data management plan is driven by the overall project goals and aims to ensure that the following are achieved: Assure that
Structure? Integrated Climate Data Center How to use the ICDC? Tools? Data Formats? User
Integrated Climate Data Center? Data Formats?? Tools???? visits Structure???? User Contents Which Data Formats do we offer? What is the Structure of our data center? Which Tools do we provide? Our Aims
Introduction to IODE Data Management. Greg Reed Past Co-Chair IODE
Introduction to IODE Data Management Greg Reed Past Co-Chair IODE Outline Background - Introduction to IOC and IODE - IODE activities Oceanographic data management - End to end data management - Data stewardship
Using standards for ocean data
Pan-European infrastructure for marine and ocean data management Using standards for ocean data By Dick M.A. Schaap SeaDataNet Technical Coordinator Vienna, April 09 Supported by the European Commission
THREDDS. THematic Real-time Environmental Distributed Data Services. Connecting people, documents and data
THREDDS THematic Real-time Environmental Distributed Data Services Connecting people, documents and data Ben Domenico, John Caron, Ethan Davis, Robb Kambic, Stefano Nativi Unidata Program Center and University
A Project to Create Bias-Corrected Marine Climate Observations from ICOADS
A Project to Create Bias-Corrected Marine Climate Observations from ICOADS Shawn R. Smith 1, Mark A. Bourassa 1, Scott Woodruff 2, Steve Worley 3, Elizabeth Kent 4, Simon Josey 4, Nick Rayner 5, and Richard
The ORIENTGATE data platform
Research Papers Issue RP0195 December 2013 The ORIENTGATE data platform SCO Scientific Computing and Operations Division By Alessandra Nuzzo University of Salento and Scientific Computing and Operations
Data Management Perspective for an International Arctic Observing Network: Preserving the Legacy Promoting Exchange
Data Management Perspective for an International Arctic Observing Network: Preserving the Legacy Promoting Exchange James A. Moore 1 Molly McCammon 2 Rob Bochenek 3 1 National Center for Atmospheric Research,
Scientific Data Management and Dissemination
Federal GIS Conference February 9 10, 2015 Washington, DC Scientific Data Management and Dissemination John Fry Solution Engineer, Esri jfry@esri.com Agenda Background of Scientific Data Management through
StormSurgeViz: A Visualization and Analysis Application for Distributed ADCIRC-based Coastal Storm Surge, Inundation, and Wave Modeling
: A Visualization and Analysis Application for Distributed ADCIRC-based Coastal Storm Surge, Inundation, and Wave Modeling Brian Blanton Renaissance Computing Institute University of North Carolina at
Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES
Norwegian Satellite Earth Observation Database for Marine and Polar Research http://normap.nersc.no USE CASES The NORMAP Project team has prepared this document to present functionality of the NORMAP portal.
Environmental Data Management:
Environmental Data Management: Challenges & Opportunities Mohan Ramamurthy, Unidata University Corporation for Atmospheric Research Boulder CO 25 May 2010 Environmental Data Management Workshop Silver
Satellite Products and Dissemination: Visualization and Data Access
Satellite Products and Dissemination: Visualization and Data Access Gregory Leptoukh GES DISC, NASA GSFC Dana Ostrenga GES DISC, NASA GSFC Introduction The Goddard Earth Sciences Data and Information Services
Web-based spatio-temporal visualization and analysis of the Siberian Earth System Science Cluster (SIB-ESS-C)
Web-based spatio-temporal visualization and analysis of the Siberian Earth System Science Cluster (SIB-ESS-C) Roman Gerlach Supervisor: Prof. C. Schmullius (Dept. of Geography, Friedrich-Schiller-University
NASA s Big Data Challenges in Climate Science
NASA s Big Data Challenges in Climate Science Tsengdar Lee, Ph.D. High-end Computing Program Manager NASA Headquarters Presented at IEEE Big Data 2014 Workshop October 29, 2014 1 2 7-km GEOS-5 Nature Run
EUMETSAT EO Portal. End User Image Access using OGC WMS/WCS services. EUM/OPS/VWG/10/0095 Issue <1> <14/01/2010> Slide: 1
EUMETSAT EO Portal End User Image Access using OGC WMS/WCS services Slide: 1 Overview Introduction: status of data access and visualization EUMETSAT datasets Architecture Web Map Service implementation
Globus Research Data Management: Introduction and Service Overview
Globus Research Data Management: Introduction and Service Overview Kyle Chard chard@uchicago.edu Ben Blaiszik blaiszik@uchicago.edu Thank you to our sponsors! U. S. D E P A R T M E N T OF ENERGY 2 Agenda
MyOcean Copernicus Marine Service Architecture and data access Experience
MyOcean Copernicus Marine Service Architecture and data access Experience Sophie Besnard CLS, Toulouse, France February 2015 MyOcean Story MyOcean Challenge & Success MyOcean Service MyOcean System MyOcean
ARM Data Center Experience in Linking Big Data
ARM Data Center Experience in Linking Big Data Giri Palanisamy ARM www.arm.gov Climate Change Science Institute Oak Ridge National Laboratory, USA Presented at: CENDI-NFAIS December 11, 2012 Climate Change
NATIONAL CLIMATE CHANGE & WILDLIFE SCIENCE CENTER & CLIMATE SCIENCE CENTERS DATA MANAGEMENT PLAN GUIDANCE
NATIONAL CLIMATE CHANGE & WILDLIFE SCIENCE CENTER & CLIMATE SCIENCE CENTERS DATA MANAGEMENT PLAN GUIDANCE Prepared by: NCCWSC/CSC Data Management Working Group US Geological Survey February 26, 2013 Version
Big Data Services at DKRZ
Big Data Services at DKRZ Michael Lautenschlager and Colleagues from DKRZ and Scientific Computing Research Group MPI-M Seminar Hamburg, March 31st, 2015 Big Data in Climate Research Big data is an all-encompassing
How To Write An Nccwsc/Csc Data Management Plan
Guidance and Requirements for NCCWSC/CSC Plans (Required for NCCWSC and CSC Proposals and Funded Projects) Prepared by the CSC/NCCWSC Working Group Emily Fort, Data and IT Manager for the National Climate
George Mason University (GMU)
George Mason University (GMU) Center for Spatial Information Science and Systems Organization (CSISS) 4400 University Drive, MSN 6E1 George Mason University Fairfax, VA 22030, USA Telephone: +1 703 993
HYCOM Data Management -- opportunities & planning --
HYCOM Data Management -- opportunities & planning -- Steve Hankin (NOAA/PMEL) Ashwanth Srinivasan, (RSMAS) Peter Cornillon, OPeNDAP PI (URI) Mike Clancy, US GODAE Server (FNMOC) Jon Callahan, Kevin O Brien
The ORIENTGATE data platform
Seminar on Proposed and Revised set of indicators June 4-5, 2014 - Belgrade (Serbia) The ORIENTGATE data platform WP2, Action 2.4 Alessandra Nuzzo, Sandro Fiore, Giovanni Aloisio Scientific Computing and
Big Data in OpenTopography
Big Data in OpenTopography Vishu Nandigam San Diego Supercomputer Center NSF Big Data in Educa
Open Source Visualisation with ADAGUC Web Map Services
Open Source Visualisation with ADAGUC Web Map Services Maarten Plieger Ernst de Vreede John van de Vegte, Wim Som de Cerff, Raymond Sluiter, Ian van der Neut, Jan Willem Noteboom 1 ADAGUC project Cooperative
REACCH PNA Data Management Plan
REACCH PNA Data Management Plan Regional Approaches to Climate Change (REACCH) For Pacific Northwest Agriculture 875 Perimeter Drive MS 2339 Moscow, ID 83844-2339 http://www.reacchpna.org reacch@uidaho.edu
Driving Earth Systems Collaboration across the Pacific
Driving Earth Systems Collaboration across the Pacific Tim F. Pugh Centre for Australian Weather and Climate Research Australian Bureau of Meteorology What s our business? Earth System Science Study of
A standards-based open source processing chain for ocean modeling in the GEOSS Architecture Implementation Pilot Phase 8 (AIP-8)
NATO Science & Technology Organization Centre for Maritime Research and Experimentation (STO-CMRE) Viale San Bartolomeo, 400 19126 La Spezia, Italy A standards-based open source processing chain for ocean
CEOS Water Portal Status Update
CEOS Water Portal Status Update WGISS-33 Tokyo, Japan April 23-27, 2012 JAXA/Mission Operations System Office Satoko Miura/ Atsushi Kawai Update since WGISS-32 New features Compare service for MOLTS data
11-12 June 2015, Bari-Italy. Stefano Nativi CNR-IIA
11-12 June 2015, Bari-Italy Stefano Nativi CNR-IIA Coordinating an Observation Network of Networks EnCompassing satellite and IN-situ to fill the Gaps in European Observations GEOSS Information System
Metadata for Data Discovery: The NERC Data Catalogue Service. Steve Donegan
Metadata for Data Discovery: The NERC Data Catalogue Service Steve Donegan Introduction NERC, Science and Data Centres NERC Discovery Metadata The Data Catalogue Service NERC Data Services Case study:
HYCOM Meeting. Tallahassee, FL
HYCOM Data Service An overview of the current status and new developments in Data management, software and hardware Ashwanth Srinivasan & Jon Callahan COAPS FSU & PMEL HYCOM Meeting Nov 7-9, 7 2006 Tallahassee,
University Bremen (UniHB) PANGAEA
SeaDataNet 2 New Participants University Bremen (UniHB) PANGAEA Stéphane PESANT (spesant@marum.de) OUTLINE 1. HISTORY 2. PEOPLE 3. INFRASTRUCTURE 4. DATA COLLECTIONS 5. METADATA & DATA SUBMISSION 6. METADATA
The CEOP Model Data Archive at the World Data Center for Climate as part of the CEOP Data Network
The CEOP Model Data Archive at the World Data Center for Climate as part of the CEOP Data Network CEOP / IGWCO Joint Meeting Feb. 28th Mar. 4th, 2005, Tokyo Michael Lautenschlager Frank Toussaint, Hans
IDL. Get the answers you need from your data. IDL
Get the answers you need from your data. IDL is the preferred computing environment for understanding complex data through interactive visualization and analysis. IDL Powerful visualization. Interactive
Globus Auth. Steve Tuecke. The University of Chicago
Globus Auth Enabling an extensible, integrated ecosystem of services and applications for the research and education community. Steve Tuecke The University of Chicago Cloud has transformed how platforms
GIS Initiative: Developing an atmospheric data model for GIS. Olga Wilhelmi (ESIG), Jennifer Boehnert (RAP/ESIG) and Terri Betancourt (RAP)
GIS Initiative: Developing an atmospheric data model for GIS Olga Wilhelmi (ESIG), Jennifer Boehnert (RAP/ESIG) and Terri Betancourt (RAP) Unidata seminar August 30, 2004 Presentation Outline Overview
DATA MANAGEMENT (DM) - (e.g. codes, monitoring) software offered to WMO Members for exchange
DATA MANAGEMENT (DM) - (e.g. codes, monitoring) software offered to s for exchange Notified to WMO 10/11/2009 United States A library of functions and subroutines which can be used to encode and decode
GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata
GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata Ing. Simone Giannecchini, GeoSolutions Ing. Emanuele Tajariol, GeoSolutions Outline GeoNetwork Introduction
Science Gateway Services for NERSC Users
Science Gateway Services for NERSC Users Shreyas Cholia NERSC User Group Meeting October 7, 2009 Science Gateways at NERSC Web access methods to NERSC resources Much is possible beyond yesterday s ssh+pbs
Development of Earth Science Observational Data Infrastructure of Taiwan. Fang-Pang Lin National Center for High-Performance Computing, Taiwan
Development of Earth Science Observational Data Infrastructure of Taiwan Fang-Pang Lin National Center for High-Performance Computing, Taiwan GLIF 13, Singapore, 4 Oct 2013 The Path from Infrastructure
OGC at KNMI: Current use and plans Available products
OGC at KNMI: Current use and plans Available products Maarten Plieger KNMI R&D Ernst de Vreede, John van de Vegte, Wim Som de Cerff, Ian van der Neut, Raymond Sluiter, Jan Willem Noteboom Contents 1. ADAGUC
National Snow and Ice Data Center A brief overview and data management projects
Cooperative Institute for Research in Environmental Sciences University of Colorado National Snow and Ice Data Center A brief overview and data management projects Walt Meier, Mark Parsons, and Mark Serreze
GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata
GeoNetwork, The Open Source Solution for the interoperable management of geospatial metadata Ing. Emanuele Tajariol, GeoSolutions Ing. Simone Giannecchini, GeoSolutions GeoSolutions GeoSolutions GeoNetwork
NCDC Strategic Vision
NOAA s National Climatic Data Center World s Largest Archive of Climate and Weather Data Presented to: Coastal Environmental Disasters Data Management Workshop September 16, 2014 Stephen Del Greco Deputy
ArcGIS Platform. An Integrated System. Portal
Platform An Integrated System Portal An Integrated Web GIS Platform Knowledge Workers Executive Access Public Engagement Work Anywhere Enterprise Integration Providing Mapping, Analysis, Data Management,
SRS BIO OPTICAL WORKFLOW
SRS BIO OPTICAL WORKFLOW Version 2.0 22 nd March 2013 Data Workflows emii, the data management facility for IMOS, has developed workflows for each IMOS sub facility to describe the flow of IMOS data from
Enhanced Research Data Management and Publication with Globus
Enhanced Research Data Management and Publication with Globus Vas Vasiliadis Jim Pruyne Presented at OR2015 June 8, 2015 Presentations and other useful information available at globus.org/events/or2015/tutorial
SAMOS, GOSUD. and the Observing System Monitoring Center. Second Joint GOSUD/SAMOS Workshop Seattle, Washington, USA 10-12 June 2008
SAMOS, GOSUD and the Observing System Monitoring Center Second Joint GOSUD/SAMOS Workshop Seattle, Washington, USA 10-12 Steve Hankin NOAA/PMEL (presenter) Derrick Snowden NOAA/OCO Kevin O Brien JISAO/UW
Data Lab Operations Concepts
Data Lab Operations Concepts 1 Introduction This talk will provide an overview of Data Lab components to be implemented Core infrastructure User applications Science Capabilities User Interfaces The scope
Research Data Collection Data Management Plan
Research Data Collection Data Management Plan This document records the whole process of the data collections, including pre- preparation data,
Model examples Store and provide Challenges WCS and OPeNDAP Recommendations. WCS versus OPeNDAP. Making model results available through the internet.
Making model results available through the internet. Fedor Baart, Gerben de Boer, Wim de Haas, Gennadiy Donchyts, Marc Philippart, Maarten Plieger September 14, 2011 Introduction Fedor Baart PhD thesis:
NOMADS. Jordan Alpert, Jun Wang NCEP/NWS. Jordan C. Alpert jordan.alpert@noaa.gov. where the nation s climate and weather services begin
An Application for High Availability NOMADS Jordan Alpert, Jun Wang NCEP/NWS Jordan C. Alpert jordan.alpert@noaa.gov DMIT Workshop, Silver Spring, MD 8/13-14//2009 where the nation s climate and weather
Levels of Archival Stewardship at the NOAA National Oceanographic Data Center: A Conceptual Model 1
Levels of Archival Stewardship at the NOAA National Oceanographic Data Center: A Conceptual Model 1 Levels of Archival Stewardship at the NOAA National Oceanographic Data Center: A Conceptual Model Dr.
A DISTRIBUTED CATALOG AND DATA SERVICES SYSTEM FOR REMOTE SENSING DATA
A DISTRIBUTED CATALOG AND DATA SERVICES SYSTEM FOR REMOTE SENSING DATA Ramachandran Suresh *, Liping Di *, Kenneth McDonald ** * NASA/RITSS 4500 Forbes Blvd, Lanham, MD 20706, USA suresh@rattler.gsfc.nasa.gov
NetCDF and HDF Data in ArcGIS
2013 Esri International User Conference July 8 12, 2013 San Diego, California Technical Workshop NetCDF and HDF Data in ArcGIS Nawajish Noman Kevin Butler Esri UC2013. Technical Workshop. Outline NetCDF
CDI/THREDDS Interoperability: the SeaDataNet developments. P. Mazzetti 1,2, S. Nativi 1,2, 1. CNR-IMAA; 2. PIN-UNIFI
CDI/THREDDS Interoperability: the SeaDataNet developments P. Mazzetti 1,2, S. Nativi 1,2, 1. CNR-IMAA; 2. PIN-UNIFI Outline Interoperability Issues in SeaDataNet A broker solution for CDI/THREDDS interoperability
Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future
Big Data at ECMWF Providing access to multi-petabyte datasets Past, present and future Baudouin Raoult Principal Software Strategist ECMWF Slide 1 ECMWF An independent intergovernmental organisation established
McIDAS-V: An open source data analysis and visualization tool for multiand hyperspectral satellite data ITSC-XVI, Angra do Reis, Brazil, 7 May 2008
McIDAS-V: An open source data analysis and visualization tool for multiand hyperspectral satellite data ITSC-XVI, Angra do Reis, Brazil, 7 May 2008 Tom Achtor, Tom Rink, Tom Whittaker, Dave Parker, Gail
Canadian National Research Data Repository Service. CC and CARL Partnership for a national platform for Research Data Management
Research Data Management Canadian National Research Data Repository Service Progress Report, June 2016 As their digital datasets grow, researchers across all fields of inquiry are struggling to manage
GOAPP Data Management Policy
Global Ocean-Atmosphere Prediction and Predictability Network GOAPP Data Management Policy Prepared by: GOAPP Data Management Committee (DMC) Jason Chaffey (DFO), Bill Merryfield (GOAPP Investigator),
JASMIN Cloud ESGF and UV- CDAT Conference 09-11 December 2014 STFC / Stephen Kill
JASMIN Cloud ESGF and UV- CDAT Conference 09-11 December 2014 STFC / Stephen Kill Philip Kershaw (1, 2), Jonathan Churchill (5), Bryan Lawrence (1, 3, 4), Stephen Pascoe (1, 4) and MaE Pritchard (1) Centre
Integration strategy
C3-INAD and ESGF: Integration strategy C3-INAD Middleware Team: Stephan Kindermann, Carsten Ehbrecht [DKRZ] Bernadette Fritzsch [AWI] Maik Jorra, Florian Schintke, Stefan Plantikov [ZUSE Institute] Markus
Data Requirements from NERSC Requirements Reviews
Data Requirements from NERSC Requirements Reviews Richard Gerber and Katherine Yelick Lawrence Berkeley National Laboratory Summary Department of Energy Scientists represented by the NERSC user community
Data Products via TRMM Online Visualization and Analysis System
Accessing Global Precipitation Data Products via TRMM Online Visualization and Analysis System (TOVAS) Zhong Liu Center for Spatial Information Science and Systems (CSISS), George Mason University and
Flexible and modular visualisation and data discovery tools for environmental information
21st International Congress on Modelling and Simulation, Gold Coast, Australia, 29 Nov to 4 Dec 2015 www.mssanz.org.au/modsim2015 Flexible and modular visualisation and data discovery tools for environmental
Baudouin Raoult, Iryna Rozum, Dick Dee
ECMWF contribution to the EU funded CHARME Project: A Significant Event Viewer tool Matthew Manoussakis Baudouin Raoult, Iryna Rozum, Dick Dee 5th Workshop on the use of GIS/OGC standards in meteorology
Progress in Creating a Global Polar Metadata Interoperability Network
Progress in Creating a Global Polar Metadata Interoperability Network Julie E. Friddell Canadian Cryospheric Information Network/Polar Data Catalogue Department of Geography, University of Waterloo Canada
McIDAS-V - A powerful data analysis and visualization tool for multi and hyperspectral environmental satellite data
McIDAS-V - A powerful data analysis and visualization tool for multi and hyperspectral environmental satellite data Thomas Achtor, Thomas Rink, Thomas Whittaker, David Parker and David Santek Space Science
PIT adoption for Climate Data Management
PIT adoption for Climate Management The German Climate Computing Center (DKRZ) 5th RDA Plenary Stephan Kindermann Overview DKRZ: A climate science service provider services for the international climate
Using Databases to Manage State Information for. Globally Distributed Data
Storage Resource Broker Using Databases to Manage State Information for Globally Distributed Data Reagan W. Moore San Diego Supercomputer Center moore@sdsc.edu http://www.sdsc sdsc.edu/srb Abstract The
Best Practices for Data Management. RMACC HPC Symposium, 8/13/2014
Best Practices for Data Management RMACC HPC Symposium, 8/13/2014 Presenters Andrew Johnson Research Data Librarian CU-Boulder Libraries Shelley Knuth Research Data Specialist CU-Boulder Research Computing
Research Data Storage, Sharing, and Transfer Options
Research Data Storage, Sharing, and Transfer Options Principal investigators should establish a research data management system for their projects including procedures for storing working data collected
Globus Research Data Management: Introduction and Service Overview. Steve Tuecke Vas Vasiliadis
Globus Research Data Management: Introduction and Service Overview Steve Tuecke Vas Vasiliadis Presentations and other useful information available at globus.org/events/xsede15/tutorial 2 Thank you to
Local Loading. The OCUL, Scholars Portal, and Publisher Relationship
Local Loading Scholars)Portal)has)successfully)maintained)relationships)with)publishers)for)over)a)decade)and)continues) to)attract)new)publishers)that)recognize)both)the)competitive)advantage)of)perpetual)access)through)
and Deployment Roadmap for Satellite Ground Systems
A Cloud-Based Reference Model and Deployment Roadmap for Satellite Ground Systems 2012 Ground System Architectures Workshop February 29, 2012 Dr. Craig A. Lee The Aerospace Corporation The Aerospace Corporation
CMIP5 Data Management CAS2K13
CMIP5 Data Management CAS2K13 08. 12. September 2013, Annecy Michael Lautenschlager (DKRZ) With Contributions from ESGF CMIP5 Core Data Centres PCMDI, BADC and DKRZ Status DKRZ Data Archive HLRE-2 archive
Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS
Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Workshop on the Future of Big Data Management 27-28 June 2013 Philip Kershaw Centre for Environmental Data Archival
OSG PUBLIC STORAGE. Tanya Levshina
PUBLIC STORAGE Tanya Levshina Motivations for Public Storage 2 data to use sites more easily LHC VOs have solved this problem (FTS, Phedex, LFC) Smaller VOs are still struggling with large data in a distributed
Homework: Visual Search and Interaction with NSF and NASA Polar Datasets Due: May 2nd, 2015, 12pm PT
Homework: Visual Search and Interaction with NSF and NASA Polar Datasets Due: May 2nd, 2015, 12pm PT 1. Overview In this assignment you will take your Apache Solr index constructed from Polar data that