Earth Science Data and Information System (ESDIS) Project Update October 15-16, 2008 National Snow and Ice Data Center & Physical Oceanography DAAC User Working Group Meeting Pasadena, CA Jeanne.Behnke@nasa.gov ESDIS Project Code 423 NASA GSFC
Topics ESDIS Organization System Context Key Metrics Data Architecture Key Activities Data Discovery Customer Satisfaction & Metrics Operations Management 2
Earth Science Data and Information System (ESDIS) Project The ESDIS Project is responsible for the Earth Observing System Data and Information System (EOSDIS), one of the largest civilian Science Information System in the world The EOSDIS: Ingests, archives, processes, and distributes an unprecedented volume of science data for NASA s flagship Earth science missions (e.g., Terra, Aqua, Aura, ICESat) Supports unique requirements of a variety of Earth science disciplines (e.g., land, atmosphere, snow/ice, and ocean) as well as inter- disciplinary researchers, climate modelers, and application users (e.g., U.S. Forest Service) Employs state-of-the-art hardware and software technology to achieve 3 required data throughput This Jason sea-surface height image shows sea surface height anomalies with the seasonal cycle (the effects of summer, fall, winter, and spring) removed. Each image is a 10-day average of data, centered on the date indicated. Courtesy: NASA EOSDIS Physical Oceanography DAAC
EOSDIS Manages Data For All 24 EOS Measurements ACRIMSAT (12/99) Solar Output Energy Budget CERES Albedo, Aerosols, Vegetation MISR Terra (12/99) Lower Atmospheric Chemistry MOPITT Surface Imaging MODIS ASTER Energy Budget Aqua (5/02) Atmospheric Sounders Surface Imaging CERES AMSR-E MODIS AIRS/AMSU/HSB SORCE (1/03) Solar Irradiance TIM, SIM, XPS SOLSTICE Trace Gases TES Aura (7/04) Atmospheric Dynamics and Chemistry HIRDLS, MLS, OMI Mesosphere ACRIM Stratosphere TRMM (11/97) Energy Budget CERES, LIS QuikScat (6/99) Sea Surface Winds SeaWinds METEOR 3M (12/01) Aerosol Trace Gases SAGE III Jason (12/01) Ocean Altimetry Poseidon/ JMR/DORIS ICESat (1/03) Ice Topography and Altimetry GLAS CloudSat (4/06) Cloud Properties CPR O 3 ClO BrO OH trace gases aerosols CALIPSO (4/06) Cloud, Aerosol Properties Rain Rain CALIOP Troposphere Ocean Biology Land Biology Volcanology Sea Surface Winds Temperature Moisture Evaporation H 2 O Sea Ice Ecosystem Dynamics Land Ice and Snow Cover O 3 precursor gases, aerosols Fire Occurrence 4 Mission & Science 04102007.ppt
Earth Science Data Availability by Mission 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 10/1984-2 years planned mission life 10/1978-10 yrs m life Heritage Missions EOS Era Missions ERS-1 NIMBUS-7 (TOMS I) Meteor 3 TOMS (TOMS II) JERS-1 KEY Science Data Available Planned Mission Life Extended Mission Life 4 year Data Access Period requirement Post Mission Estimate No Planned EOL April 2007 NASA Earth Science Senior Review ERBS (SAGE II) UARS TOPEX/Poseidon OrbView-2 (SeaStar) ERS-2 Earth Probe TOMS (TOMS III) ADEOS I early failure RadarSat 1 Landsat 7 TRMM QuikSCAT ACRIMSAT Terra GRACE Transition to USGS Jason-1 Meteor 3M (SAGE III) 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Aqua ADEOS II early failure ICESat SORCE Aura CERES data only EOSDIS, operational since August 1994, preserves earth science data and products from heritage missions as well as EOS missions, ensuring long term data records 5
Earth Science Context Diagram Data Acquisition Spacecraft Tracking & Data Relay Satellite (TDRS) Flight Operations, Data Capture, Initial Processing & Backup Archive Data Transport to DAACs Science Data Processing, Data Mgmt., Data Archive & Distribution Distribution, Access, Interoperability & Reuse Research Education Ground Stations Data Processing & Mission Control NASA Integrated Services Network (NISN) Mission Services EOSDIS Science Data Systems (DAACs) Data Pools REASoNs ECHO ACCESS W W W Value-Added Providers Interagency Data Centers Earth System Models ACCESS International Partners Polar Ground Stations Science Teams (SIPS) Measurement Teams Benchmarking DSS Technology Infusion 6
EOSDIS Facilities ASF DAAC SAR Products Sea Ice, Polar Processes Data centers, collocated with centers of science discipline expertise, archive and distribute standard data products produced by Science Investigator-led Processing Systems (SIPSs) SIPSs Key Data Center JPL MLS, TES PO.DAAC Ocean Circulation Air-Sea Interactions NCAR, U of Col. HIRDLS, MOPITT, SORCE NSIDC DAAC Cryosphere, Polar Processes San Diego ACRIM LP DAAC Land Processes & Features ORNL DAAC Biogeochemical Dynamics, EOS Land Validation GHRC Hydrological Cycle & Severe Weather CDDIS Crustal Dynamics Solid Earth GES DISC Atmos Composition & Dynamics, Global Modeling, Hydrology, Radiance GHRC AMSR-E, LIS LaRC CERES, SAGE III SEDAC Human Interactions in Global Change GSFC GLAS, MODIS, OMI, OBPG OBPG Ocean Biology & Biogeochemistry LAADS/ MODAPS Atmosphere ASDC Radiation Budget, Clouds, Aerosols, Tropo Chemistry 7
DRAFT SMD E. Weiler GSFC R. Strain Earth Science & Data Systems Earth Science M. Frielich Planetary. Flight Projects G. Morrow Appl. Eng. & Technology Sciences & Exploration. 400 500 600 Applied Science T. Fryberger Research J. Kaye Flight Programs S. Volz. Earth Science G. Colon Planetary Science Astrophysics Heliophysics Astrophysics. 420 430 440 Application Scientists Program Executive for Mission Operations C. Yuhas Program Scientists Program Executive for Earth Science Data Systems M. Maiden DD Earth Science 423 Data and Information System (ESDIS) Project D. Lowe Science Systems Development Office D. Marinelli Science Operations Office J. Behnke Terrestrial Ecology D. Wickland B. Emmanual W. Turner Applications S. Ambrose Physical Oceanography E. Lindstrom Data Center Operations Cryoshpere Science S. Martin National Snow and Ice Data Center Atmospheric Radiation H. Maring ORNL DAAC Socioeconomic Data & Applications Center Ocean Biology and Biogeochemistry P. Bontempi Atmospheric Dynamics R. Kakar Earth Surface and Interior J. LaBrecque Land Processes DAAC Physical Oceanography DAAC SAR Systems C. Dobson Global Hydrology Resource Center Upper Atmosphere Research K. Jucks Atmospheric Sciences Data Center Ocean Biology Processing Group SAR Data Center GSFC Earth Sciences DISC Crustal Dynamics DIS
EOSDIS Products Delivered: 1996 thru Sept 2008 (in Millions) 180 160 140 17,263 K 120 100 80 60 40 20 0 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 to date Sep 9
10
11
12
13
FY2008 Data Center Web Activity Sep 2008 ASF GESDISC LARC LPDAAC MODAPS NSIDC ORNL PODAAC 80,000 70,000 Unique Visitors Previous Month Current Month Previous Year 60,000 by Month Total Visitors Aug 08 66,439 Sep 08 97,026 Sep 07 58,338 Number of Visitors 50,000 40,000 30,000 FY to date Total Visitors Repeat Visitors 690,359 117,519 774,619 132,315 647,010 108,372 20,000 10,000 0 2 3 4 5-6 7-9 10-14 15-24 25-49 50-99 100+ FY08 Repeat Web Visitors by end of September Number of Visits 14
NASA Earth Science Interoperable Data Architecture 15
Earth Science Data Discovery Global Change Master Directory (GCMD) (http://gcmd //gcmd.nasa.gov) Provides a catalog ad portal for data discovery of NASA and other publicly available environmental data and services. ECHO (EOS Clearing House) Provides a data granule-level metadata for all NASA EOS data. ECHO enables common searches of earth science relevant data across all NASA on-line data access systems. XML-based structure allows more easy mapping between differing metadata formats. Transition to the new Warehouse Inventory Search Tool (WIST) Individual DAAC Tools OGC Tools Collaboration Tools and Portals 16
- Over 17,000 metadata records - Over 1,000 worldwide data providers - Average 5 million visits per month http://gcmd.nasa.gov/ 17
ECHO Status Holdings and Partners Current Holdings Collections 2,674 Granules 85 million Browse 28 million Data Partners Operational Public: 10 Test: 1 (GHRC) Client Partners Operational 3 (WIST, SNOW-I, ESA) In evaluation or test2 (WECHO, AQUA) Active development1 (WISRD) 18
2008 ECHO Improvements Synchronize TT/NCR prioritization scheme with ECS Defect tracking & prioritization Severity 1 escalation: are worked 24x7 until fixed for ECHO Upgraded Support 8:00am 7:00pm EST Operations support 24 Hour response time to echo@echo messages 48 Hour notification prior to Operational planned downtime Data Integrity fixes include better ingest validation; spatial representations fixed and enforced data rules Performance & scalability enhancement Ingest performance improved from 9K events/hr to 40K events/hr Recovery & failover Deployed a new redundant WIST system (recovery in under 15 mins) Data corruption or system corruption recovery in under 6 hrs 19
EOSDIS ACSI Customer Satisfaction Survey EOSDIS sponsors an annual independent customer survey in conjunction with the American Customer Satisfaction Index (ACSI). EOSDIS consistently exceeds the Federal Government average Ratings in the mid to upper 70s are considered very good by the rating organization, the CFI Group Federal Government (Overall) 2008 67 NASA EOSDIS 77 2008 NASA EOSDIS 2007 75 E-Business News & Information (Public/Private Sectors) 2008 30 40 50 60 70 80 72 20
ACSI and EOSDIS Metrics System ESDIS will coordinate the actions to address the 2008 ACSI survey results and determine focus areas for the next year Work with DAACs to examine specific results and actions Get the message out to users that we are looking at the ACSI results and have a corresponding action plan to address the key areas identified for improvement ESDIS provides CFI Group a list of email addresses collected in the previous 12 months (37,229 in 2008; 23,827 in 2007) CFI received 2,763 surveys in 2008 (2291 in 2007) ESDIS will continue to develop the metrics system (EMS) Provide opportunity to define new metrics on utilization of data documentation, data search, and data formatting services Sponsor use of NetInsight Web tracking for finding common access patterns, guiding web page re-organization, and enabling more efficient access to data and documentation 21
Data Management Coordination Work to facilitate core (DAAC-SIPS) and community (MEaSURES( MEaSURES) ) involvement in NASA ES data Two meetings: system engineering & management Upcoming focus is Data Center-SIPS interaction scheduled for November 2008 Regular telecons WIKI forums Support the Earth Science Data Systems Working Group (ESDSWG) and ESIP Federation Standards involvement - ex: HDF, Metadata, FGDC, Process definition 22
23
Operating Budgets and Plans Every year DAACs and ESDIS prepare a work plan and operations budget The work plan addresses contract tasks ESDIS looks for activities that will improve data center operations while maintaining a high level of customer satisfaction Budgets 5 years for input to NASA s s Program Operations Plans FY010-16 submission prepared for NASA Science Mission Directorate Program review Apr 2009 Annual excel spreadsheet entitled the snowball Due January February time frame Details mission/cost-work function per fiscal year» (e.g., ops, engineering, equipment traceable to contract WBS) Heritage from 1990 s Current budget projection is stable (i.e., flat) New mission data costs added as approved by ESDIS 24
ESDIS Project Contacts Dawn Lowe, Project Manager dawn.lowe@nasa lowe@nasa.gov Jeanne Behnke, Science Operations Office Manager jeanne.behnke@nasa behnke@nasa.gov John Moses, Science Operations Office john.f.moses@nasa moses@nasa.gov Please feel free to contact any of us with suggestions or comments! 25