Ocean Colour experience SeaWiFS / MODIS / VIIRS Bryan Franz and the NASA Ocean Biology Processing Group presented by Ewa Kwiatkowska, EUMETSAT Sentinel-3 Cal/Val Meeting 20-22 March 2012
Goal Provide a continuous, accurate time-series of global ocean bio-optical properties to the user community in a timely and efficient manner to facilitate research and applications in global change, carbon cycle, ocean ecosystem dynamics, and coastal monitoring. Focus on the long-term climate data record of water-leaving radiances and derived properties.
Ocean Color (OC) SST for MODIS Salinity from Aquarius NASA Goddard Space Flight Center Ocean Biology Processing Group oceancolor.gsfc.nasa.gov End-to-End for Ocean Color Sensor calibration/characterization Processing software & algorithms Data archiving and distribution Product validation Tools for user processing, display, analyses User support Distributed processing system 2.7 Petabytes online storage 400x global reprocessing for MODIS 4000x for SeaWiFS OC Missions Supported VIIRS/NPP: 2011-present MODIS/Aqua: 2002-present MODIS/Terra: 1999-present SeaWiFS/Orbview-2: 1997-2010 CZCS/NIMBUS-7: 1978-1986 OCM-2/Oceansat-2: 2010-present (ISRO) MERIS/Envisat: 2002-present (ESA) OCTS/ADEOS: 1996-1997 (JAXA) MOS/IRS-P3: 1996-2004 (DLR) New Sensor Development!
Quality Approach Focus on instrument calibration establishing temporal and spatial stability within each mission Apply common algorithms ensuring consistency of processing across missions Apply common vicarious calibration approach ensuring spectral and absolute consistency of water-leaving radiance retrievals under idealized conditions Perform detailed trend analyses (frequent internal reprocessing) assessing temporal stability and mission-to-mission consistency testing proposed calibration and algorithm changes Reprocess and redistribute timeseries (~ every 2 years) incorporating new instrument knowledge and algorithm advancements
SeaWiFS Sensor Degradation 3%
MODIS Lunar and Solar Calibration Trends MODIS 412nm Responsivity Changes Since Launch Aqua 0% 10% Gain Terra 50% 20% 30% 40% 50%
MODIS-Terra Vicarious On-orbit Characterization change in RVS shape and polarization sensitivity +15% 412 443 488 Polarization RVS -10% Scan Pixel Scan Pixel Scan Pixel Time 2000 2011
Effect of MODIS-Terra On-Orbit Recharacterization Global Deep-Water Chlorophyll Trend 100%
Sensor-Independent Approach ancillary data SeaWiFS L1A MODISA L1B MODIST L1B OCTS L1A MOS L1B OSMI L1A CZCS L1A MERIS L1B OCM-1 L1B OCM-2 L1B VIIRS-L1B observed radiances Multi-Sensor Level-1 to Level-2 (common algorithms) water-leaving radiances and derived prods Level-2 Scene predicted at-sensor radiances sensor-specific tables: Rayleigh, aerosol, etc. in situ water-leaving radiances (MOBY) vicarious calibration gain factors Level-3 Global Product Level-2 to Level-3
Vicarious Calibration Vicarious calibration is critical to achieve accurate ocean color retrievals from SeaWiFS and MODIS A stable vicarious calibration may require 40-50 high-quality matchups (we require all satellite retrievals within a 5x5-km area to be valid) Once a sufficient number of match-ups is achieved, no additional measurements are required Vicarious calibration must be repeated after any instrument calibration or processing algorithm changes
SeaWiFS Vicarious Gains convergence with sample size
Quality Assessment Methods Level-2 satellite to in situ match-up analysis regional analysis (e.g., in situ frequency distribution vs satellite) Level-3 global, regional, zonal mean time-series time-series comparison between sensors anomaly trends relative to seasonal cycle Level-2 to Level-3 ratio coincident Level-2 pixels to multi-day Level-3 bins assessment of scan angle dependencies detector to detector variability
Chl a in Good Agreement with Global In situ SeaWiFS vs in situ MODIS-Aqua vs in situ
MODIS-Aqua Rrs in good agreement with SeaWiFS Deep-Water solid line = SeaWiFS R2010.0 dashed = MODISA R2010.0 412 Rrs (str -1 ) 443 488 & 490 within 1% on average 510 531 547 & 555 667 & 670
Mean spectral differences agree with expectations SeaWiFS MODISA 488 490 oligotrophic mesotrophic eutrophic 547 & 555
MODIS-Terra Rrs in good agreement with SeaWiFS Deep-Water solid line = SeaWiFS R2010.0 dashed = MODIST R2010.0 412 Rrs (str -1 ) 443 488 & 490 510 531 547 & 555 667 & 670
MERIS Rrs is biased relative to SeaWiFS Deep-Water solid line = SeaWiFS R2010.0 dashed = MERIS R2 (2006) 412 443
Chlorophyll spatial variation in good agreement SeaWiFS Fall 2002 MODIS-Aqua MODIS-Terra
Chlorophyll spatial variation in good agreement SeaWiFS Fall 2008 MODIS-Aqua MODIS-Terra
Global Chlorophyll Timeseries Oligotrophic Subset SeaWiFS Mesotrophic Subset SeaWiFS
Global Chlorophyll Timeseries Oligotrophic Subset SeaWiFS MODIS-Aqua Mesotrophic Subset SeaWiFS MODIS-Aqua
Reprocessing (Frequency & Versions) 1996 V1996.0 SeaWiFS (simulated), MOS (Germany), OCTS (Japan) 1997 V1997.0 SeaWiFS (SeaWIFS processing initiated) 1998 V1998.0 SeaWiFS (calibration update) V1998.1 SeaWiFS (calibration update) 2000 V2000.0 SeaWiFS (significant algorithm & software update) V2000.1 SeaWiFS (calibration update) 2001 V2001.0 SeaWiFS, OCTS (reprocessing of OCTS for NASDA) 2002 V2002.0 SeaWiFS (significant algorithm updates)
Reprocessing History 2004 V2004.0 MODISA, SeaWiFS (add MODIS/Aqua) 2005 V2005.0 MODISA, SeaWiFS (significant algorithm updates, *polarization) V2005.1 MODISA, SeaWiFS (minor algorithm updates) 2006 V2005.1 OCTS, CZCS (REASON) 2007 V2007.0 MODIST, SeaWiFS (add MODIS/Terra forward stream) 2009 V2009.0 SeaWiFS (major algorithm updates) V2009.1 MODISA, SeaWiFS, MERIS (minor algorithm updates, add MERIS FR) 2010 V2010.0 - MODISA, MODIST, SeaWiFS, OCTS, CZCS, MERIS
OBPG Computing Facility NASA Goddard Space Flight Center Greenbelt, Maryland Building 28 Room W220-900 square feet dedicated to the project - Primary and backup A/C units - 225 Kva Eaton Powerware UPS with dual PDUs, additional 6 Kva rack-mounted UPS units - Connections to GSFC high speed networks: SEN - 45 (85) processing nodes (linux PCs) - 76 (124) RAID storage nodes (linux PCs) Typically we replace 50% of the system every 18 months, often just replacing drives. No significant down-time. No maintenance contracts.
Processing System Hardware Distribution Servers (ftp) 6 storage nodes 49.8 TB 1 server 250GB Ingest Servers 2 processing node 552 GB 1 storage node 5.1 TB Processing Cluster 85 processing nodes 189 TB Database Servers 7 database nodes 29.6 TB Backup Servers 2 storage nodes 19.7 TB Network Support Servers Testing Cluster Distribution and Storage Servers (web) Production: 3 web server nodes 72 TB 124 storage nodes 2.7 PB Extreme Networks Black Diamond 8810 Gigabit Ethernet switch 480 ports Aquarius Telemetry Viewer 2 Windows PC Aquarius Command Planning Tool Servers 2 processing nodes 586 GB AQOPS Servers (web) Production: 2 processing nodes Subversion CM Servers 2 development nodes 1 TB Development Servers 2 development nodes 10 TB Cal/Val & QC Systems User Desktops
OBPG Hardware (Latest Generation) Processing servers Intel Six-Core 2.4 Ghz CPU 16 GB DDR3-1333 MHz RAM SATA 750 GB data drives (5) SATA 250 GB system disk Storage RAID servers Intel Xeon Quad-core 2.13 GHz Low Voltage processor 8 GB DDR3-1333 MHz RAM SATA 2 TB data drives (16) SATA 250 GB system disk Analysis Servers Intel Six-Core 2.4 Ghz CPU 24 GB DDR3-1333 MHz RAM SATA 3 TB data drives (5) SATA 250 GB system disk
achieving research quality data requires an integrated team collocated with processing facilities to implement, test, evaluate Algorithms & Software Calibration Evaluation Reprocessing
Distribution Mechanisms Entire archive is directly accessible via http protocol: storage servers run internal webservers, access is managed by DB Data download through browsers (point & click), or ftp-like interface, or through user-scripted bulk download Ordering mechanism for regional data selection and geographic and parameter extraction supported Subscription services supported NRT image services supported Timely (interative) user support provided
MODIS Distribution GB/Day Files/Day Product Aqua Terra Aqua Terra ingested produced distributed Level-0 60 60 288 288 Level-1 110 110 288 288 Level-2 65 40 Level-3 10 5 Level-0 36 36 169 187 Level-1 159 37 1810 865 Level-2 309 47 10179 4120 Level-3 74 11 3838 652 + files produced by SeaDAS users
1150 registered users 17 thousand posts 12.5 million views since 2004
All Processing Source Code Distributed in SeaDAS 15 years in distribution, free, open-source, linux/os-x/windows(vm)
Lessons Learned multiple reprocessings are required to achieve and maintain quality global ocean color measurements reprocessing from uncalibrated radiances (L0 or L1A) is necessary, as instrument calibration knowledge evolves over time use of common algorithms can isolate instrument performance artifacts from algorithm performance evaluation of global and regional Level-3 trends provides a means to assess instrument temporal stability comparison of Level-3 trends between missions can corroborate geophysical changes consistency between missions can be achieved through application of common algorithms and calibration methods
Lessons Learned on-line direct access to cost-free data assures wide use and active participation of the user community http://oceancolor.gsfc.nasa.gov/cgi/browse.pl open source software for user-local data processing allows for community participation in new product development, enhances user understanding and confidence, facilitates community feedback and support http://oceancolor.gsfc.nasa.gov/seadas/ on-line forum allows knowledgeable staff to provide user support, which is vital to advancing the goals of the missions http://oceancolor.gsfc.nasa.gov/forum/oceancolor/forum_show.pl