Data Assimilation As A Verification Tool

Size: px

Start display at page:

Download "Data Assimilation As A Verification Tool"

Geoffrey Benson
7 years ago
Views:

1 Data Assimilation As A Verification Tool Dr. Dale M. Barker, MMM Division, NCAR

2 Data Assimilation Overview Assimilation system combines: Wide range of observations - y o Previous background forecast(s) - x b Observation/forecast error estimates. Laws of physics. Basic assimilation equation (3/4D-Var, EnKF): x a! x b = K[ y o! H(x b )] Data assimilation systems used in a number of ways: Provide initial conditions for numerical forecasts. Climate (reanalysis) studies. Observing system design (e.g. OSSEs). Observation monitoring/verification system.

3 Data Assimilation in NWP (Example) x lbc x b Observation Preprocessing (3DVAR_OBSPROC) y o WRF-Var x a Update Lateral BCs (WRF_BC) Forecast (WRF) Background Error (gen_be) B 0 x a! x b = K[ y o! H(x b )]

4 Data Assimilation for Monitoring/Verification x b Observation Preprocessing (3DVAR_OBSPROC) y o WRF-Var d Verification Toolkit Forecast (T+12, T+24, etc) Monitoring: Study observation performance Verification: Study forecast performance d = y o! H(x b )

5 The Global Observation Network (~2003)

6 High Resolution Data Assimilation Observations TMI brightness temperature

7 Example Sonde Observation Errors Used In DA Comparison of observation errors from different models: Pressure (hpa) km O-B AFWA HIRLAM RUC Pressure (hpa) km O-B AFWA HIRLAM RUC Error Standard Deviation (K) Error Standard Deviation (m/s) Temperature U-wind

8 Verification With Imperfect Observations Assume both observed and forecast temperature have errors ε: T o = T t +! o,t f = T t +! f d = T o! T f where T t is the true value. Need estimates of error variance: E(! o 2 ) = " o 2, E(! f 2 ) = " f 2 Assume observation/forecast errors uncorrelated and unbiased, then the best estimate of the truth (the analysis) is given by T a =! f 2! 2 o +! T +! o 2 o f! 2 o +! T 2 f f Analysis is only optimal if error estimates are accurate (use DA techniques). 2

9 Observation Monitoring: SSM/T1 - Sonde Temperature Observation/Forecast Scatter Plot Observed Temperature (K) SSMT1 Observations Sonde Observations MM5 Background Temperature (K)

10 Relative Frequency Monitoring/Verification Via O-F Differences Synop T O-B Bias = 0.198K, StDv=2.09K Metar T O-B Bias = 1.09K, StDv = 2.19K O-B Correlation O-B (K) Distance (km) METAR O-B u METAR O-B T METAR O-B q Domain Mean O-B (K) Assimilation Cycle (3hourly) METAR O-B T SYNOP O-B T Correlation 1 METAR O-B u 0.9 NMC E-W 0.8 NMC E-W Distance (km)

11 Observation Monitoring: AIRS WV channel 2005/08/26/00Z Observed Forecast Observed Minus Forecast

12 Observation Quality Control Observations have many sources of error: instrumental, representativeness, reporting, decoding, etc. Gross errors are usually either a) Human origin (e.g. wrong date, location reported, or b) Instrument failure. Bad obs. can have a large negative impact on assimilation: If in doubt, throw it out! Extensive quality control (QC) is essential to make the most of the data (examples: physical, range, buddy, complex, variational QC). QC algorithms can be applied to clean data for verification system.

13 Example DA-Based Verification: WRF In India RS/RW Stations Pilot balloon Stations

14 Example DA-Based Verification: WRF In India T+24 Forecast Error (RMS) T+24 Forecast Error (Bias)

15 Forecast Error (FE) Estimation in For DA Assume FEs estimated as perturbations x : Two ways of defining x : The NMC-method (Parrish and Derber 1992): where e.g. t2=24hr, t1=12hr forecasts or ensemble perturbations (Fisher 2003): B 0 = (x b! x t )(x b! x t ) T " x' x' T B 0 = x'x' T! A(x t 2 " x t1 )(x t 2 " x t1 ) T B 0 = x'x' T! C(x k " x )(x k " x ) T Analysis Increments: T (O-F=1K): 26 GFS-based NMC-method X GRID Tuning via innovation vector statistics and/or variational methods. VERTICAL SIGMA LEVEL CV5-ENS VERTICAL SIGMA LEVEL VERTICAL SIGMA LEVEL X GRID Mesoscale & Microscale 15 Meteorological Division / NCAR 10 5 WRF-Based ENS-method

16 Adjoint Sensitivity Example: Impact of Individual Observations on 24hr Forecast Skill From Gelaro (2006)

17 Data Assimilation As A Verification Tool Observation minus forecast differences are the main input to both DA and obs-based verification systems. DA permits verification against a wide range of QCed observations. Major efforts undertaken in DA to accurately estimate forecast and observation errors. Use info in verification? Adjoint/ensemble sensitivities can pinpoint origins of forecast error. Major DA research areas not covered: Biased DA, Model Error, DA parameter (e.g. forecast error) estimation.

18 Ensemble-Derived Forecast Error Covariances From Hamill (2006)

19 OI Quality Control Obs. Assimilated in OI OI analysis Withheld observation OI QC is based on the minus difference threshold

20 Verification With Imperfect Observations Consider simple example: Temperature at Boulder, Colorado (point X). X Observation T o (X) Forecast T f (X) What is the true temperature at point X? Old: Verification systems typically assume observation as truth. New: Assigns DA errors to both, with truth as weighted average.

21 Verification With Imperfect Observations The precision of the analysis is given optimally by Example 1: If analysis is perfect. Example 2: If observations) then! o 2 = 0! o 2 =! f 2 1! = 1 2 a! o! f! a 2 =! o 2 /2 then the observation is truth, and the (reasonable approximation for certain Note: The analysis is always more accurate than either the observation or the forecast (if the error estimates are accurate).

An Integrated Ensemble/Variational Hybrid Data Assimilation System

An Integrated Ensemble/Variational Hybrid Data Assimilation System DAOS Working Group Meeting September 2012 Tom Auligné, National Center for Atmospheric Research Acknowledgements: Contributors: Luke Peffers