GEOGG142 GMES Calibration & validation of EO products Dr. Mat Disney mdisney@geog.ucl.ac.uk Pearson Building room 113 020 7679 0592 www.geog.ucl.ac.uk/~mdisney
Outline Calibration & validation Example: MODIS LAI and NPP products Meaning of parameters?? Time, space, measurements? Scaling? Good place to start: CEOS Working Group on Cal/Val, Land Product Validation sub-group: http://lpvs.gsfc.nasa.gov/ and see eg Biophysical & references therein: http://lpvs.gsfc.nasa.gov/lai_references.html and good practice guidelines: http://lpvs.gsfc.nasa.gov/pdf/ CEOS_LAI_PROTOCOL_Aug2014_v2.0.1.pdf 2
Calibration & validation? Calibration: Process of adjusting empirical relationship between empirical estimates of biophysical parameter estimated from 2 (or more) sources e.g. ground-based and EO-derived LAI, or NDVI and LAI, or NDVI and fapar etc. etc. Local calibration to estimate fitting parameters (slope, intercept for a linear relationship) and uncertainty Limitations??? 3
Calibration & validation? Calibration: process of converting an instrument reading to a physically meaningful measurement Particularly radiometric calibration i.e. from DN to radiance measurement OR Process of adjusting empirical relationship between estimates of biophysical parameter estimated from 2 (or more) sources e.g. ground and EO-derived 4
Calibration & validation? Validation: experiments designed to verify instrument measurements using independent measurements Caveat: EO validation often means testing one model-derived estimate against another EO LAI, NPP etc. all require models Field estimates of LAI also require models i.e. NOT validation in true sense at all See later: when is LAI not LAI BUT: cal/val both essential to scientific remote sensing 5
Aside: focus on validation here but. Eg LAI CEOS WGCV: recommends CALIBRATION comparison between EO and ground-based NEEDS: reference estimates traceable to in situ measurements 3 sources available: 1. LAI measurements over individual Elementary Sampling Units (ESUs) 2. Spatially extensive LAI reference maps based on data driven relationships calibrated using ESU LAI 3. Spatially extensive LAI reference maps based on functional relationships calibrated using ESU LAI. And what is an ESU? See later 6
Validation example: MODIS NPP Productivity recap: Net Primary Productivity (NPP) annual net carbon exchange quantifies actual plant growth Conversion to biomass (woody, foliar, root) i.e. not just C0 2 fixation (GPP) NPP = GPP Ra (plant respiration) MODIS product example used here MOD17 GPP/NPP ATBD ntsg.umt.edu/mod17 http://neo.sci.gsfc.nasa.gov/search.html Turner et al (2005) 7
Productivity recap GPP/NPP from MODIS Requirements? MOD17 ATBD Running et al. (2004) Turner et al. (2005) Zhao et al. (2005) Heinsch et a. (2006) 8
MODIS GPP/NPP + QC?? http://secure.ntsg.umt.edu/projects/index.php/id/ca2901a0/fuseaction/projects.detail.htm 9
MOD17 validation approach Need to address time (days to years) and space (local to global) Permanent network of ground validation sites Quantify seasonal and interannual dynamics of ecosystem activity (cover time domain) EO to quantify heterogeneity of biosphere Quantify land cover, land cover change dynamics Models to: Quantify, understand unmeasured ecosystem Provide predictive capability (in time AND space) 10
How on earth..???? can we validate an EO-derived estimate of something that depends on soil, climate, land cover etc.? Given that it requires various models to go from a satellite observation (radiance), to reflectance, to LAI/FAPAR, to PSN, to GPP to NPP At 500m-1km pixels. Globally. And how do you even measure NPP on the ground?? 11
So, how might we validate? Need to consider scale Relate measurements at the small scale to 1km pixels?? Flux tower approach Eg BIGFOOT approach, FLUXNET etc. Measurements and validation at many scales Models to bridge time/space scales (but how good are models?) Fig from MOD17 ATBD 12
Ecosystem measurements: FLUXNET Fig from MOD17 ATBD http://daac.ornl.gov/fluxnet/ 13
Ecosystem measurements: FLUXNET 2009 http://daac.ornl.gov/fluxnet/ http://earthobservatory.nasa.gov/features/fluxnet/ 14
Ecosystem measurements: FLUXNET 2009-11 http://daac.ornl.gov/fluxnet/ http://www.fluxnet.ornl.gov/fluxnet/graphics.cfm http://earthobservatory.nasa.gov/features/fluxnet/ 15
Ecosystem measurements: FLUXNET http://daac.ornl.gov/fluxnet/ 16
Ecosystem measurements: FLUXNET by biome Some distribution of biome types, but clearly biased in location Even considering only limited biomes http://daac.ornl.gov/fluxnet/ 17
BigFoot approach to validating MODIS NPP E.g. Turner et al. (2005), 6 sites spanning range of vegetation and climate Crops, forest, tundra, grassland 5 x 5 km site at each plot (25 MODIS pixels) Flux tower & 100 (25x25m) sample plots within each area, seasonally measured for LAI and above-ground (A)NPP (from harvested leaf and wood material) Land cover from high res EO Use measured data at sample plots to calculate NPP, GPP Spatially distribute across site using (vegetation-calibrated) BiomeBGC model Requires daily met data, land cover, LAI Gives measured estimate from ground AND flux tower 18
BigFoot v flux tower GPP Turner et al. (2005) 19
BigFoot v MODIS GPP Not such good agreement as for flux tower (not surprisingly) Turner et al. (2005) 20
Comparison of MODIS NPP with flux data Differences due to Ra (autotrophic i.e. plant respiration)? PAR, VPD differences between those from DAO and actual? (VPD = deficit between the amount of moisture in the air and how much moisture the air can hold when it is saturated) Turner et al. (2005) 21
DAO PAR, VPD? Clearly some sites better agreement than others PAR generally good (relatively easy to measure) VPD less so e.g. SEVI (desert grassland site) VPD Other issues? Turner et al. (2005) 22
MODIS-estimated v BigFoot FPAR How do you measure FPAR even on the ground?? Requires models to interpret measurements of radiation Turner et al. (2005) 23
MODIS-estimated v BigFoot LUE (light use efficiency) LUE inferred from flux data Again, hard to even measure this on the ground.. Turner et al. (2005) 24
Zhao et al. (2005) Heinsch et al. (2006) 25
Process/SVAT (soil-veg-atm-transport) models Fig from MOD17 ATBD 26
Process models: how do we test/validate? From Running et al. (2004) MOD17 ATBD Biome-BGC model predicts the states and fluxes of water, carbon, and nitrogen in the system including vegetation, litter, soil, and the nearsurface atmosphere i.e. daily PSN 27
Process models: how do we test/validate? http://www.ntsg.umt.edu/models/bgc/ Fig from MOD17 ATBD 28
Data-Model Fusion [Using multiple streams of datasets with parameter optimization] C stock and flux measurements Inventory analyses Process-based information Climate data Remote sensing information CO 2 column from space Inverse modeling Process-based modeling Retrospective and forward analyses 29 Canadell et al. 2000
Multi-level model/data validation MOD17 ATBD: Synergy of various carbon measurement programs Fig from MOD17 ATBD 30
How do we decide on ground-based sampling strategy, scale? CEOS WGCV LPV: Elementary Sampling Units (ESUs). a contiguous spatial region over which the expected value of LAI can be estimated through in situ measurement corresponds to finest spatial scale of LAI estimates used for reference LAI maps. ESU size:.at least as large as one measurement footprint of the in situ instrument and typically includes a number of instrument measurements. ESU size: varies with surface condition, instrument field of view, illumination conditions (when transmission based measurements are used) and spatial sampling design. ESU size:.should be sufficient to allow repeat visit with minimum uncertainty due to changes in illumination or geolocation. 31
ESUs: CEOS WGCV LAI validation protocol 32
Summary Calibration Needed to allow comparison of data & products from multiple sensors & algorithms over time AND/OR to Can be done on-board, or via sensor intercomparison etc. Validation example: NPP Far removed from EO measurement & spatially, temporally variable Requires: observation networks over time and space and measurement of met. & biophysical data Models to interpolate spatially from ground-based, site-scale measurements Testing and intercomparison of models Ideally: optimal combinations of models + data across scales (e.g. via data assimilation) 33
References: NPP Running et al. (2004) A Continuous Satellite-Derived Measure of Global Terrestrial Primary Production, Bioscience 54(6), 547-560. Ganguly et al. (2008a, b) Generating vegetation leaf area index earth system data record from multiple Sensors, RSE, 112, 4318-4332 (Part II) and 4333-4343 (Part I) Turner et al. (2005) Site-level evaluation of satellite-based global terrestrial gross primary production and net primary production monitoring, Glob Change Biol, 11, 666-684. Zhao et al. (2005) Improvements of the MODIS terrestrial net and gross primary production data sets, RSE, 95, 164-176. Heinsch et al. (2006) Evaluation of Remote Sensing Based Terrestrial Productivity From MODIS Using Regional Tower Eddy Flux Network Observations, IEEE TGRS, 44(7), 1908-1925. General validation http://lpvs.gsfc.nasa.gov/lai_references.html Morisette et al. (2002) A framework for the validation of MODIS Land products, RSE, 83, 77-96. Disney et al. (2004) IJRS, 25(23), 5297-5317 34
Other cal/val links NPP: http://daac.ornl.gov/npp/npp_home.html Cal/val programs CEOS-WFGCV (Committee on EO Working Group on Cal/ Val) http://calvalportal.ceos.org/calvalportal/welcome.do http://lpvs.gsfc.nasa.gov/ http://landval.gsfc.nasa.gov/ SAFARI2000: http://daac.ornl.gov/s2k/safari.html VALERI: http://w3.avignon.inra.fr/valeri/ NCAVEO: http://www.ncaveo.ac.uk/ JAXA: http://www.eorc.jaxa.jp/alos/en/calval/calval_index.htm Etc etc etc 35
Example: MODIS core val sites http://landval.gsfc.nasa.gov/coresite_gen.html Justice et al. (1998) http://eospso.gsfc.nasa.gov/eos_observ/5_6_98/p55.html Privette et al. (2002) and RSE 83, 1-2, 1-359 36