The Sochi 2014 Winter Olympics

Size: px

Start display at page:

Download "The Sochi 2014 Winter Olympics"

Antony Doyle
7 years ago
Views:

1 The Sochi 2014 Winter Olympics Forecast Verification Framework Some Early Results ( FROST-2014 ~ Forecast and Research in the Olympic Sochi Testbed ) Pertti Nurmi Matias Brockmann & Sigbritt Näsman Acknowledgements : WMO FROST-2014 Expert Team & FMI verification system development team

2 XXII Olympic Winter Games Sochi 2

3 3

4 Aibga ridge : Alpine skiing top ~2300 m Rosa Khutor Sanki RusSki Gorki Laura ski stadium ~ 1400 m Olympics Mountain Cluster sports venues 4

5 Dmitry Kiktev Nanette Lomarda (WMO) 5

6 FROST-2014 : Models - Contributors Model Analysis / FC hours Contributor FMI Verification Notes Deterministic Forecasts COSMO-RU - 7 km 00, 06, 12, 18 UTC / + 78 hr HMC,Russia *** COSMO-RU - 2 km 00, 06, 12, 18 UTC / + 42 hr *** COSMO-RU - 1 km - to be considered as RDP GEM km 23 UTC / + 27 hr Env. Canada *** GEM - 1 km 21 UTC / + 25 hr *** GEM km 00 UTC / + 24 hr *** Processing difficulties HARMONIE-Sochi - 1 km 00, 06, 12, 18 UTC / + 36 hr FMI, Finland *** NMMB - 1 km 00, 12 UTC / + 24 hr NOAA,USA *** KMA - KMA, S-Korea to be considered as RDP ARPA (Sochi-mini) 00, 12 UTC / + 72 hr ARPA SIMC, Italy *** INCA - ZAMG, Austria Joint Hourly / + 48 hr consensus *** Ensemble Forecasts Aladin-LAEF-EPS - 11 km - ZAMG,Austria GLAMEPS - 11 km 06, 18 UTC / + 54 hr HIRLAM, Norway *** GLAMEPS - updated T Hourly / + 48 hr *** HarmonEPS km 06, 18 UTC / + 30 hr *** to be considered as RDP COSMO-RU-EPS - 2 km - HMC,Russia to be considered as RDP COSMO-S14-EPS - 7 km - ARPA SIMC, Italy NMMB-EPS - 7 km - NOAA, USA Nowcasting MeteoExpert - single station 10 minutes / + 4 hr IRAM,Russia *** Joint Hourly / + 48 hr consensus *** INCA ZAMG,Austria INTW Env.Canada to be considered as RDP CARDS Env.Canada to be considered as RDP ABOM Env.Canada to be considered as RDP RW Model ( Harmonie-driven ) FMI, Finland Verification within external CoMoSeF project 6

7 Verification Issues #1 All FROST-2014 contributors verify their own fc products General practice! BUT can be biased NOT sufficient / adequate Centralized (HMC, Russia) AND External verification by FMI also to validate verification results Two complementary approaches (like Vancouver Olympics SNOW-V10) a) High temporal resolution Point Verification User needs : FDP* Users require thresholds for their decisions Sports activities and events take place at (or along) points b) Research-oriented verification, Detailed model comparisons : RDP* * FDP = Forecast Demonstration Project / RDP = Research Demonstration Project / WMO terminology 7

8 Verification Issues #2 Hi-fidelity site-specific (point) observations highly important Enhanced observation network : 31 selected stations used in verification On-line production, storage + dissemination from all forecast contributors Common data base structure : ftp site Freeze the FDP system at certain stage No changes allowed in the last 3-4 months before the Olympics Standardized XML formats for fc & obs dissemination + storage! Only some did Comprehensive post-event verification possible only after all data archived and QC d Case studies Hard to generalize results fall under impact assessment and/or RDP Formal verification period : 15 January 15 March = 2 months Olympics : 7-23 February + Paralympics : 7-16 March 8

9 Verification Issues #3 Pointwise verification of all forecasts, tuned at all forecast = decision points a) Assumption : Reference baseline quality from a coarse, large-scale model b) Various hi-res models from contributors with various time/space resolutions c) Consensus products Joint model d) Inter-comparison between, and added value of (a) to (c) Impacts End-Products by the Olympics forecaster team not disseminated for verification Definition of each sports site/event relevant obs stations + fc variables and thresholds Not trivial! Use of verification software, e.g. MET by HMC Russia FMI online verification package tuned for Sochi Input (fc + obs) provided at common ftp site Only some did Formal verification period : 15 January 15 March = 2 months Olympics : 7-23 February + Paralympics : 7-16 March 9

10 FROST-2014 : Weather parameters - Thresholds Temperature ( O C ) T < T < -5-5 T < -2-2 T < 0 0 T < 2 2 T < 5 T 5 Wind speed ( m/s ) WS 3 WS 4 WS 5 WS 7 WS 11 WS 15 WS 19 Horizontal visibility ( m ) V < 100 V < 300 V < 1000 V < Precipitation amount RR < 0.3 RR 0.3 RR 1.0 RR 5.0 RR 10.0 RR hr and 24-hr (mm) High-impact What is hi-impact? Thresholds Photo johnny9s on Flickr Photo Prentice Korea on Flickr 10

11 FROST-2014 : Critical thresholds for sports events 11

12 FMI Verification User Interface Adapted for Sochi 2014 Olympics 12

13 FMI Verification User Interface Adapted for Sochi 2014 Olympics 13

14 Ski Biathlon (Laura, top 1400m) Ski Jump Valley (c. 600m) Alpine Ski (Rosa Khutor, top 2300m) FMI Verification User Interface Adapted for Sochi 2014 Olympics 14

15 T MAE All stations Determ. models HARMONIE best during daytime Sports events HARMONIE worst during nighttime who cares? Lo-res models, COSMO 7km + ARPA, worst 00UTC forecast runs (available for Olympics forecasters in the morning) used in verification statistics 15

16 T MAE Coastal Mountain stations cluster Determ. models Hi-res HARMONIE model, GEM best 0.25km, during day, strangely worst worst during at night coast is the feature in the mountains COSMO worst 00UTC forecast runs (available for Olympics forecasters in the morning) used in verification statistics 16

17 WS SEDI Mountain cluster Determ. models GEM 0.25km and JOINT best HARMONIE in the middle Lo-res model, COSMO 7km, worst 00UTC forecast runs (available for Olympics forecasters in the morning) used in verification statistics 17

18 PoT ROC Mountain Ski stadium cluster EPS models NB: Actual Olympics 2-wk period Calibrated + + hourly T-updated GLAMEPS best Raw More GLAMEPS noisy statistics worstdue to sample size (3 stations) All forecast runs used in the EPS verification statistics 18

19 Subjective Evaluation Forecast quality assessment : 10 HMC Russia forecasters / team leaders Scale : 0 = Not useful / 1 = Partly useful / 2 = Useful / 3 = Excellent Acknowledgement : Michael Tsyrulnikov, HMC Russia Forecast accuracy Model Overall usefulness Temp Precip Wind Gusts Visib Visualization (appearance) Timeliness and reliability COSMO-RU 7 km x COSMO-RU 2 km x 2.9 COSMO-RU 1 km x NMMB 1 km GEM 250 m Harmonie 1 km ( * ) ( * ) : Harmonie DID produce (good) visibility forecasts which were not used by Sochi forecasters (not disseminated as XML) 19

20 VIS SEDI 3 Alpine stations HARMONIE beats GEM 0.25km during first 15 hrs 00UTC forecast runs (available for Olympics forecasters in the morning) used in verification statistics 20

21 The Impact Issue End-user = Finnish skiing service team, comments : HARMONIE was superior HARMONIE visibility forecasts were excellent HARMONIE precipitation and cloud height forecasts were highly useful HARMONIE five-panel user interface was really good Sochi 2014 Road Weather Pilot: End-user Feedback Scale : 1 = Fully disagree / 2 = Disagree / 3 = Neutral / 4 = Agree / 5 = Fully agree Date Friction forecast gave a good estimate of slipperiness Storage terms (water, snow, ice) are realistic Road weather forecast was useful Notes (free text) Precipitation, however road surface dried towards Adler Dry road surface, i.e. No real need for RW forecast Variable road surface humidity Dry road surface, i.e. No real need for RW forecast FMI speciality : Testing of road weather forecasts for Sochi roads under externally funded CoMoSeF project 21

22 Future actions and activities Re-run of statistics after full final data sets available Comprehensive diagnostic verification Comparison with others verification results Detailed look at the most interesting cases vs. forecast performance Societal aspects The Impact Issue WMO SERA interests? Joint reporting-publishing-presentations, with WMO FROST-2014 contributors WWOSC i.e. here; FROST-2014 October meeting; EMS; AMS etc Do same at 2018 Winter Olympics in S-Korea (?) 22

23 Thank You

Nowcasting: analysis and up to 6 hours forecast

Nowcasting: analysis and up to 6 hours forecast Very high resoultion in time and space Better than NWP Rapid update Application oriented NWP problems for 0 6 forecast: Incomplete physics Resolution space