Advantages of a precipitation analysis to verify the forecast for Europe Anna Ghelli, Thank you to Tiziana Cherubini Verification Workshop Boulder July 2002 Slide 1
Up-scaling technique General Circulation Models predict precipitation on spatial scales that are different from the observed scales Verification against irregular distributed data, as SYNOP available from the GTS might be, is liable to misinterpretation Precipitation forecast Areal quantity Grid-point value Up-scaling simple averaging procedure of all the observations contained in a model gridbox Verification Workshop Boulder July 2002 Slide 2
FC 1999111012 t+66 vt: 1999111206-1999111306 The forecast is shaded (as per legend) and the observations are the small numbers Verification Workshop Boulder July 2002 Slide 3
Up-scaled observations Verification Workshop Boulder July 2002 Slide 4
Observed yes Observed no Verification measures Forecast yes a b FREQUENCY BIAS INDEX FBI = a a + + b c Forecast no c d EQUITABLE THREAT SCORE ETS = a a R( a) + b + c R( a) R( a) = ( a + b)( a a + b + c + + c) d Verification Workshop Boulder July 2002 Slide 5
High density observations from January 1997 to February 2000 from Météo-France Deterministic model Spectral horizontal resolution T213 and T319 from April 1998, both resolutions have the same Gaussian grid (approximately 80 Km) vertical resolution 31 levels in the vertical (L31), 50 levels from March 1999 and 60 levels since October 1999. Verification Measures Average over France Monthly averages Openthresholds:0.1,1,2,4,8and16mm/24h Verification datasets: up-scaled observations and SYNOP from GTS Verification Workshop Boulder July 2002 Slide 6
DETERMINISTIC MODEL FBI a = a + + b c t+42 Fc yes Fc no Obs y a c Obs no b d ETS = a R( a) a + b + c R( a) Verification Workshop Boulder July 2002 Slide 7
The FBI is inflated when SYNOP on GTS are used for the verification Noticeable bias decrease since spring 1998 (change in spectral resolution) ETS shows a more skilful system when verified against up-scaled observation Verification Workshop Boulder July 2002 Slide 8
-- ETS -- time-series of monthly values t+42 The forecasting system is more skilful in winter (well defined synoptic systems) than in summer months (the precipitation is more convective) Higher values of ETS for smaller precipitation thresholds (0.1 and 1 mm/24h) Larger thresholds (8 and 16mm/24h) show lower values of ETS and a more marked seasonal trend. Verification Workshop Boulder July 2002 Slide 9
started a project in early to collect precipitation data from high-density network in Europe Favourable response Precipitation data from the various European countries participating in the project is not synchronised, therefore the analyses lag by about three months behind the forecasts. Verification Workshop Boulder July 2002 Slide 10
Forecast (shaded) and analysis (numbers) for November Verification Workshop Boulder July 2002 Slide 11
Forecast (shaded) and analysis (numbers) for December Verification Workshop Boulder July 2002 Slide 12
During the European rainy season MAE shows its minimum values, while in the summer the values are larger MAE -- 24h accumulated precipitation 3 2.5 2 mm/24h 1.5 1 0.5 0 Jan. Feb. Mar. Apr. May. Jun. Jul. Aug. Sep. Oct. Nov. Dec. Jan. 2002 Feb. 2002 Mar. 2002 Verification Workshop Boulder July 2002 Slide 13
Preliminary results The sample size varies considerably from month to month, because of the way the high-resolution observations reach. The period considered is from January to March 2002. Verification measures (FBI, ETS and MAE) Averages over the European area Monthly averages and three monthly averages (DJF, MAM, JJA and SON) Verification Workshop Boulder July 2002 Slide 14
0.4 0.3 ETS Forecast range t+66 0.2 0.1 0.4 0.3 ETS 0 MAM JJA SON DJF 0.2 1.4 FBI 0.1 1.2 0 0.25 1 2 5 10 15 20 40 1 0.8 MAM JJA SON DJF 0.25mm/24h 1mm/24h 2mm/24h MAM JJA SON DJF -2002 Verification Workshop Boulder July 2002 Slide 15
ETS shows slightly higher values in the spring months. ETS decreases rapidly as the threshold increases. This could be partly due to sample size and also the analysis may not be yet detailed enough to characterise European precipitation characteristics. The model seems to forecast a higher number of rainy events than observed for very small precipitation thresholds. FBI is closer to 1 when higher thresholds are considered. Verification Workshop Boulder July 2002 Slide 16
Conclusion Verification against irregular distributed data is liable to misinterpretation Preliminary results suggest that the FBI is inflated when forecasts are verified against SYNOP on the GTS An European precipitation analysis is a way forward to make verification more meaningful: the model perspective of verification has been presented in the talk. The precipitation analysis adds value to our forecasts through diverse verification methods that can be devised. Considering other perspectives of verification (forecaster, river management authorities, insurance company, etc) certainly represents a challenge for the future. Verification Workshop Boulder July 2002 Slide 17