Verification of Experimental and Operational Weather Prediction Models March to May 2015

Transcription

1 MET info / Meteorology Verification of Experimental and Operational Weather Prediction Models March to May Bjørg Jenny Kokkvoll Engdahl and Mariken Homleid With contributions from: Anne-Mette Olsen, Vibeke Wauters Thyness, and Magnus Ovhed Photo: Ingrid Våset

2 Contents Models Post processed models HARMONIE, AROME-Norway and AROME-MetCoOp. ALARO- physics AROME physics SURFEX as surface model Data assimilation Surface analysis Upper air analysis Boundaries and initialization of upper air fields Verification measures. Forecasts of continuous variables Forecasts of categorical variables Observations Norway 7. Comments to verification results Pressure and variables at pressure levels Wind Speed m Max Mean Wind Speed m Wind gust Temperature m Post processed temperature m Daily precipitation Eastern Norway 6. Comments to the verification results Pressure Wind Speed m Wind gust Temperature m Post processed temperature m Daily precipitation Western Norway 8 7. Comments to the verification results Pressure Wind Speed m Max Mean Wind Speed m

3 7. Wind gust Temperature m Post processed temperature m Daily precipitation Northern Norway 6 8. Comments to the verification results Pressure Wind Speed m Max Mean Wind Speed m Wind gust Temperature m Post processed temperature m Daily precipitation Long term forecast 9. Comments to the verification results Temperature m Wind Speed m h Precipitation h Precipitation Appendix 69. m Wind speed Temperature m Daily precipitation ii

4 POST PROCESSED MODELS Models The following models are verified in this report. All except EC are or have been running at MET. EC Hirlam (H) Hirlam8 (H8) Global model (IFS) at the. From 6 January resolution T 79 or approximately 6 6 km horizontally. Available resolution for verification at MET is. latitude and longitude. Number of vertical levels increased from L9 to L7 June. Version 7., horizontal resolution defined by a km grid since February 8. Version 7., horizontal resolution defined by a 8 8 km grid since February 8. Harmonie. HARMONIE cycle 6h. with ALARO physics run on a.. km grid from May to January. Harmonie. HARMONIE cycle 6h. with AROME physics run on a.. km grid from May to 6 February. AROME-MetCoOp (AM) HARMONIE cycle 8h. with AROME physics run on a.. km grid on same domain as AROME-Norway; experimental since 9 December. Analysis and lead times of forecasts are denoted by e.g. + UTC which indicates forecast generated at UTC and valid hours later. Post processed models Most of the raw model data are post processed before being published on Yr. The models AM and H8 are statisticly post processed to represent the m max mean wind speed. After post processing, these models are called AM_PP and H8_PP. For m temperature, the raw data is first interpolated to a m grid and then Kalman filtered. These model fields are called AM.KF and H8.KF. The precipitation data is post processed by using a neighbourhood method. Through the method, a median value is obtained. This field, called AM.median is used to determine the precipitation symbol at Yr. For long term forecast, an ensemble of members run at is used to obtain a probabilistic forecast. These fields are called EC_UKAL_KONSENSUS and EC_UKAL_MEDIAN for the consensus and median forecast, respectively. If the probablistic forecast is calibrated before it is published on Yr, it is called EC_KAL_...

5 HARMONIE, AROME-NORWAY AND AROME-METCOOP The deterministic model just called, is also shown in the long term forecast graphs. HARMONIE, AROME-Norway and AROME-MetCoOp Experimental HARMONIE models have been run at MET Norway since August 8, leading to AROME-Norway which on October was introduced on yr.no, and AROME-MetCoOp which is run in cooperation between Swedish Meteorological and Hydrological Institue and MET Norway and replaced AROME-Norway on yr.no 7 May. HARMONIE is the acronym for HIRLAM s meso-scale forecast system (Hirlam Aladin Regional/Meso-scale Operational NWP In Europe). The HARMONIE system includes several configuration options. This section presents some of the main components and setups that are or has been used at MET. More documentation is available on ALARO- physics ALARO- has physical parameterizations targeted for grey scale resolutions ( - km). It is a spin-off of the Météo-France physical parameterizations used in the globale ARPEGE, but with a separate radiation scheme, MT micro-physical frame work, and the Toucans turbulence scheme. Much of the development has been done by the RC LACE (Regional Cooperation for Limited Area modeling in Central Europe) community.. AROME physics AROME (Applications of Research to Operations at MEsoscale) is targeted for horizontal resolution. km or finer. It uses physical parameterizations based on the French academia model Meso-NH and the external surface model SURFEX. AROME has been operational at Météo- France since 8 December 8, with a horizontal resolution of. km.. SURFEX as surface model SURFEX (Surface externalisée) is developed at Météo-France and academia for offline experiments and introduced in NWP models to ensure consistent treatment of processes related to surface. Météo-France is already using SURFEX for some of their configurations and is planning to use it for all their configurations. Surface modelling and assimilation benefits from the possibility to run offline experiments. SURFEX is also used for offline applications in e.g. hydrology, vegetation monitoring and snow avalanche forecasts.

6 HARMONIE, AROME-NORWAY AND AROME-METCOOP. Data assimilation SURFEX includes routines to simulate the exchange of energy and water between the atmosphere and surface types (tiles); land, sea (ocean), lake (inland water) and town. The land or nature tile can be divided further into vegetation types (patches). ISBA (Interaction between Soil Biosphere and Atmosphere) is used for modelling the land surface processes. There are ISBA options; - and -layer force restore and a diffusive approach, where the first one is used in HIRLAM. Towns may be treated by a separate TEB (Town Energy Balance) module. Seas and lakes are also treated separately. The lake model, FLAKE (Freshwater LAKE), has recently been introduced in SURFEX. A global ECOCLIMAP database which combines land cover maps and satellite information gives information about surface properties on km resolution. The orography is taken from gtopo. SURFEX Scientific Documentation and User s Guide are available on Data assimilation NWP models are updated regularly using observations received in real-time from the global observing system. With one exception the models run at MET are updated at, 6, and 8 UTC. AROME-MetCoOp is updated each third hour; at,, 6, 9,,, 8 and UTC... Surface analysis Surface analysis is performed by CANARI (Code d Analyse Nécessaire à ARPEGE pour ses Rejets et son Initialisation) (Taillefer, ). The analysis method is Optimal Interpolation and only conventional synoptic observations are used. meter temperature and relative humidity observations are used to update the surface and soil temperature and moisture. The snow analysis is also performed with CANARI in analogy with the HIRLAM snow analysis. Snow depth observations are used to update Snow Water Equivalent. The snow fields are analysed only at 6 UTC as there are very few snow depth observations at, and 8. The Sea Surface Temperature is not analysed, but taken from the boundaries. uses the OSTIA (Operational Sea Surface Temperature and Sea Ice Analysis) product, including SST from UK Met Office and SIC from MET. The surface temperature over sea ice is taken from the boundary model and remains unchanged through the forecast... Upper air analysis AROME-MetCoOp runs three dimensional variational (D VAR) data assimilation using conventional observations from synop stations, ships, radiosondes and aircrafts. AMSU-A and AMSU-B/MHS data from the polar orbiting NOAA and METOP satellites is also used.

7 VERIFICATION MEASURES. Boundaries and initialization of upper air fields. Boundaries and initialization of upper air fields Harmonie. and Harmonie. got their boundary values (-hourly) from the model at approximately 6 km resolution. The upper air fields were initialized from forecasts each cycle. Harmonie. had 6 vertical levels (6 using the definition). Harmonie. had also 6 vertical levels (HIRLAM6 using the HIRLAM definition). AROME-Norway and AROME-MetCoOp get their boundary values (-hourly) from the model at approximately 6 km resolution. They have currently 6 vertical levels. AROME- Norway do no upper air assimilation, the upper air fields are initialized from forecasts each cycle. None of the HARMONIE configurations at MET have applied digital filter initialization (DFI). Verification measures All model forecasts in this report are verified against observations by interpolating (bilinear) the grid based forecasts to the observational sites. As a consequence, it should be noted that it is the models abilities to forecast the observations that is being quantifed and assessed. Thus, there is no attempt in this report to verify area averaged precipitation for example. Verification is carried out both for raw and categorized forecasts. In the following, let f,..., f n denote the forecasts and o,..., o n the corresponding observations.. Forecasts of continuous variables The verification statistics applied to continuous variables are defined in the table below

8 VERIFICATION MEASURES. Forecasts of categorical variables Statistic Acronym Formula Range Optimal score Mean Error ME n n i= (f i o i ) to Mean Absolute Error MAE n n i= f i o i to Standard Deviation of Error SDE ( n n i= (f i o i ME) ) / to Root Mean Square Error RMSE ( n n i= (f i o i ) ) / to Correlation COR n n i= (fi f)(o i ō) SD(f)SD(o) to In the formula for COR the following definitions are used f = n n f i, i= ō = n n i= o i SD(f) = ( n n (f i f) ) / ( n, SD(o) = (o i ō) ) / n i= for the means and standard deviations of the forecasts and observations. i=. Forecasts of categorical variables All variables in this report are continuous in raw form, but it is possible to categorize them and verify these. For example, wind speed above a given threshold could be of interest which would result in two possible outcomes (yes and no). The verification is then completely summarized by a contingency table as the one shown below event observed yes no event forecasted yes a b no c d Verification statistics for such forecasts are listed in the following table

9 VERIFICATION MEASURES. Observations Statistic Acronym Formula Range Optimal score Hit rate HR a a+c to False alarm rate F b b+d to False alarm ratio FAR b a+b to Equitable threat score ETS a ar a+b+c ar / to ( = no skill) Hanssen-Kuipers skill score KSS HR - F to ( = no skill) In the formula for ETS ar = (a + b)(a + c)/n.. Observations All observations come from Klimadatavarehuset at MET. Only synop stations are used, except for precipitation where all availiable stations are used for better spatial coverage. The model wind speed is verified against the mean wind FF observations. For post processed wind speed, the maxium min mean wind speed last hour, FX, is used. 6

10 NORWAY Norway. Comments to verification results Mean Sea Level Pressure: Slight increase in bias for AM, compared with the winter season, which still has the highest bias amongst the models. still the best model for MSLP, while AM has the second lowest MAE. Wind speed: Mean wind speed: More diurnal variations in the bias during spring, than in the winter. Not so much diurnal variation for AM. Negative bias for. The Hirlam models have negative biases during daytime and positive biases in the night. AM has a postive bias, but only of about.ms. AM scores better than the rest of the models in ETS. AM also has considerably more events above ms compared with the rest of the models, still too few compared with the observations. Max mean wind speed: The post processed version of AM is generally underestimating the max mean wind speed. H8_PP has a smaller bias, but AM scores highest for ETS. Wind gust: The wind gust from AM is generally too low, while the wind gust from H8 is similarly too high. The 9hPa wind is quite similar in both models. MAE is considerably lower for the wind gust than the 9hPa wind. The wind gust scores better for all thresholds except the highest, were the 9hPa wind is slightly better. Temperature m: All models have a cold bias during the spring of. H has the smallest bias. and AM also have a cold trend, and end up at around - C after 6h. For this is actually an improvement from the winter season. While for AM it decrease in score. SDE is reduced for all models, compared with the winter season. MAE is also in general reduced, but the reduction is largest for. In total H is the model with the lowest MAE for the spring. Post processed temperature: Hardly any bias in temperature after post processing. MAE is a bit reduced, few differences between H8.KF and AM.KF. Daily precipitation: Positive biases for all models, except AM.median. has the highest bias, while AM has the smallest bias and the lowest MAE. HR is highest for for thresholds up to mm/day. For thresholds above this, AM has the highest HR. AM.median has the lowest FAR up to mm/day, and the highest ETS for thresholds below 8mm/day. has the highest ETS between 8-mm/day. Above mm/day AM has the highest score. This is seen again in the multiple contingency table, where AM.median has the highest number correct in 7

11 NORWAY. Comments to verification results the lowest category, rules the middle categories, while AM wins in the >mm/day category. 8

12

13 NORWAY. Pressure and variables at pressure levels. Pressure and variables at pressure levels.. AM H8 H ME 9 stations. [hpa] Lead Time [h] SDE 9 stations AM H8 H [hpa] Lead Time [h] MAE 9 stations.. AM H8 H [hpa] Lead Time [h]

14 NORWAY. Pressure and variables at pressure levels AM H8 H ME Lead time [h]: +,+6,..,+8 9 stations AM H8 H.... SDE Lead time [h]: +,+6,..,+8 9 stations

15 NORWAY. Pressure and variables at pressure levels ME AM UTC ME Hirlam UTC ME UTC SDE AM UTC SDE Hirlam UTC SDE UTC Geopotential height at Norwegian stations ME AM UTC ME Hirlam UTC ME UTC SDE AM UTC SDE Hirlam UTC SDE UTC Wind speed at Norwegian stations

16 NORWAY. Pressure and variables at pressure levels Mean Absolute Error Norwegian stations +,+,+6,+ UTC. Hirlam8 Harmonie. AROME_Norway AM..... Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Standard Deviation of Error Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar. Mean Error Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

17

18 NORWAY. Wind Speed m. Wind Speed m ME stations.. AM H8 H [m/s] Lead time [h].. AM H8 H SDE stations [m/s] Lead time [h] MAE stations. AM H8 H. [m/s] Lead time [h]

19 NORWAY. Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate stations AM H8 H Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM H8 H False Alarm Ratio stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score stations AM H8 H Exceedance threshold [m/s] 6

20 NORWAY. Wind Speed m ME Lead time [h]: +,+6,..,+8 stations 6 AM H8 H SDE 6 Lead time [h]: +,+6,..,+8 stations AM H8 H Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS stations [,] (,] (,7] (7,] (,Inf] Sum (,] (,] AM (,7] (7,] H8 (,7] (7,] (,Inf] (,Inf] 77 Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] (,] (,7] (7,] H (,] (,7] (7,] (,Inf] 9 8 (,Inf] 6 77 Sum Sum

21 Mean Error 9 Norwegian stations Hirlam8 Harmonie. AROME_Norway AM +,+,+6,+ UTC Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian coastal stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian mountainous stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

22 NORWAY. Wind Speed m Standard Deviation of Error 9 Norwegian stations +,+,+6,+ UTC Hirlam8 Harmonie. AROME_Norway AM Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian coastal stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian mountainous stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 9

23 NORWAY. Wind Speed m Mean Absolute Error 9 Norwegian stations +,+,+6,+ UTC Hirlam8 Harmonie. AROME_Norway AM Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian coastal stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian mountainous stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

24

25 NORWAY. Max Mean Wind Speed m. Max Mean Wind Speed m ME stations. AM AM_PP H8 H8_PP. [m/s] Lead time [h].. AM AM_PP H8 H8_PP SDE stations [m/s] Lead time [h] MAE stations. AM AM_PP H8 H8_PP [m/s] Lead time [h]

26 NORWAY. Max Mean Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate stations AM AM_PP H8 H8_PP Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM AM_PP H8 H8_PP False Alarm Ratio stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score stations AM AM_PP H8 H8_PP Exceedance threshold [m/s]

27 NORWAY. Max Mean Wind Speed m ME Lead time [h]: +,+6,..,+8 stations 6 AM AM_PP H8 H8_PP SDE 6 Lead time [h]: +,+6,..,+8 stations AM AM_PP H8 H8_PP Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS stations [,] (,] (,7] (7,] (,Inf] Sum AM (,] (,7] (7,] (,Inf] AM_PP (,] (,7] (7,] (,Inf] Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] H8 (,] (,7] (7,] H8_PP (,] (,7] (7,] (,Inf] (,Inf] Sum Sum

28

29 NORWAY. Wind gust. Wind gust ME stations AM_FG H8_FG AM_9 H8_ Lead time SDE stations.. AM_FG H8_FG AM_9 H8_ Lead time MAE stations.. AM_FG H8_FG AM_9 H8_ Lead time 6

30 NORWAY. Wind gust..8 Lead time [h]: +,+6,..,+ Hit Rate stations AM_FG H8_FG AM_9 H8_ Exceedance threshold..8 False Alarm Ratio Lead time [h]: +,+6,..,+ AM_FG H8_FG AM_9 H8_9 stations Exceedance threshold..8 Equitable Threat Score Lead time [h]: +,+6,..,+ stations AM_FG H8_FG AM_9 H8_ Exceedance threshold 7

31

32 NORWAY.6 Temperature m.6 Temperature m. ME 79 stations. ME [C]... AM H8 H Lead Time AM H8 H SDE 79 stations SDE [C] Lead Time MAE 79 stations.6. AM H8 H MAE [C] Lead Time 9

33 NORWAY.6 Temperature m AM H8 H ME Lead time [h]: +,+6,..,+8 79 stations AM H8 H..... SDE Lead time [h]: +,+6,..,+8 79 stations

34 Mean Error Norwegian stations +,+,+6,+ UTC Hirlam8 Harmonie. AROME_Norway AM Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian coastal stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian inland stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

35 NORWAY.6 Temperature m Standard Deviation of Error Norwegian stations Hirlam8 Harmonie. AROME_Norway AM +,+,+6,+ UTC Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian coastal stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian inland stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

36 NORWAY.6 Temperature m Mean Absolute Error Norwegian stations +,+,+6,+ UTC Hirlam8 Harmonie. AROME_Norway AM Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian coastal stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar Norwegian inland stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

37

38 NORWAY.7 Post processed temperature m.7 Post processed temperature m ME 79 stations.. ME [C].. AM AM.KF H8 H8.KF Lead Time SDE [C] AM AM.KF H8 H8.KF SDE 79 stations Lead Time MAE 79 stations.6. AM AM.KF H8 H8.KF MAE [C] Lead Time

39 NORWAY.7 Post processed temperature m AM AM.KF H8 H8.KF ME Lead time [h]: +,+6,..,+8 79 stations AM AM.KF H8 H8.KF..... SDE Lead time [h]: +,+6,..,+8 79 stations 6

40

41 NORWAY.8 Daily precipitation.8 Daily precipitation ME stations.. AM AM.med H8 H [mm].... Lead Time [h].. AM AM.med H8 H SDE stations [mm]... Lead Time [h] MAE stations.. AM AM.med H8 H [mm].. Lead Time [h] 8

42 NORWAY.8 Daily precipitation. Lead time [h]: + HR stations AM AM.med H8 H Exceedance threshold [mm]. Lead time [h]: + FAR stations Exceedance threshold [mm] AM AM.med H8 H..8 Lead time [h]: + ETS stations AM AM.med H8 H Exceedance threshold [mm] 9

43 NORWAY.8 Daily precipitation AM AM.med H8 H ME Lead time [h]: +,+ stations AM AM.med H8 H SDE Lead time [h]: +,+ stations OBS AM [,.] (.,] (,] (,] (,Inf] Sum [,.] (.,] (,] (,] (,Inf] Sum OBS AM.med [,.] (.,] (,] (,] (,Inf] Sum [,.] (.,] (,] (,] (,Inf] Sum OBS [,.] (.,] (,] (,] (,Inf] Sum [,.] (.,] (,] (,] (,Inf] Sum OBS H8 [,.] (.,] (,] (,] (,Inf] Sum [,.] (.,] (,] (,] (,Inf] Sum Lead time [h]: +,+ stations

44 NORWAY.8 Daily precipitation Mean Error 96 Norwegian stations Hirlam8 Harmonie. AROME_Norway AM + UTC Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar coastal stations Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 6 stations more than m above sea level Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 9 stations with daily mean precipitation > mm Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

45 NORWAY.8 Daily precipitation Standard Deviation of Error 8 96 Norwegian stations Hirlam8 Harmonie. AROME_Norway AM + UTC 6 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar coastal stations 8 6 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 6 stations more than m above sea level 8 6 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 9 stations with daily mean precipitation > mm 8 6 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

46 NORWAY.8 Daily precipitation Mean Absolute Error Norwegian stations Hirlam8 Harmonie. AROME_Norway AM + UTC Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 7 coastal stations 6 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 7 6 stations more than m above sea level 6 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar 7 9 stations with daily mean precipitation > mm 6 Mar Jun Sep Dec Mar Jun Sep Dec Mar Jun Sep Dec Mar

47 6 Eastern Norway 6. Comments to the verification results Snow-day in South-East Norway March 6 The public forecast on Yr.no was misleading. A warm front approached from the south, with warm moist air overlaying the established cold air in the region. The cold layer below ft is well represented in the profiles from Gardermoen and Rygge, with the warm air sliding on top. AROME-MetCoOp seems to be slightly warmer near the surface than the Hirlam models. The two temperature plots show values AROME-MetCoOp before and after post-processing of m-temperature, compared to SYNOP observations in the area near Oslo. Before postprocessing the m-tempertature is - C higher than the observations. The post-processing raises the temperature by some - C, and is thereby pulling the forecast further away from what is observed. Combined with a symbol algorithm that is based heavily on Tm, this produced a forecast of some - mm rain, while in reality we saw - cm snow. This error in forecasted/postprocessed m-temperature lasted for - days prior to the event, and this short period is not visible in the overall statistics for temperature. However, for the forecasts issued for this particular day, it was of great importance.

48 6 EASTERN NORWAY 6. Comments to the verification results Figure : Figures from the March 6 case. Topleft: Forecasters analysis of the synoptic situation. Topright: Vertical profile from Gardermoen. Bottom: Vertical profile from Rygge.

49 6 EASTERN NORWAY 6. Comments to the verification results Figure : Figures from the March 6 case. Topleft: Forecasted values from AROME (raw) and observations. Topright: Forecasted values from AROME (post processed) and observations. Bottom: Forecast from Yr. 6

50

51 6 EASTERN NORWAY 6. Pressure 6. Pressure ME 7 stations.. AM H8 H [hpa] Lead Time [h] [hpa] AM H8 H SDE 7 stations Lead Time [h] MAE 7 stations.. AM H8 H [hpa] Lead Time [h] 8

52 6 EASTERN NORWAY 6. Pressure ME Lead time [h]: +,+6,..,+8 7 stations AM H8 H SDE Lead time [h]: +,+6,..,+8 7 stations... AM H8 H 9

53

54 6 EASTERN NORWAY 6. Wind Speed m 6. Wind Speed m.. AM H8 H ME 76 stations. [m/s] Lead time [h] SDE 76 stations.. AM H8 H [m/s] Lead time [h] MAE 76 stations. AM H8 H [m/s] Lead time [h]

55 6 EASTERN NORWAY 6. Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate 76 stations AM H8 H Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM H8 H False Alarm Ratio 76 stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score 76 stations AM H8 H Exceedance threshold [m/s]

56 6 EASTERN NORWAY 6. Wind Speed m ME Lead time [h]: +,+6,..,+8 76 stations 6 AM H8 H 6 SDE Lead time [h]: +,+6,..,+8 76 stations AM H8 H Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS 76 stations [,] (,] (,7] (7,] (,Inf] Sum (,] (,] AM (,7] (7,] H8 (,7] (7,] 7 99 (,Inf] 6 (,Inf] Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] (,] (,7] (7,] H (,] (,7] (7,] (,Inf] (,Inf] Sum Sum

57

58 6 EASTERN NORWAY 6. Wind Speed m AM + ME at observing sites forecast means

59 6 EASTERN NORWAY 6. Wind Speed m ME 76 stations.. AM AM_PP H8 H8_PP. [m/s] Lead time [h] SDE 76 stations. AM AM_PP H8 H8_PP. [m/s] Lead time [h].. AM AM_PP H8 H8_PP MAE 76 stations [m/s] Lead time [h] 6

60 6 EASTERN NORWAY 6. Wind Speed m AM + SDE at observing sites forecast means

61 6 EASTERN NORWAY 6. Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate 76 stations AM AM_PP H8 H8_PP Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM AM_PP H8 H8_PP False Alarm Ratio 76 stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score 76 stations AM AM_PP H8 H8_PP Exceedance threshold [m/s] 8

62 6 EASTERN NORWAY 6. Wind Speed m ME Lead time [h]: +,+6,..,+8 76 stations 6 AM AM_PP H8 H8_PP SDE Lead time [h]: +,+6,..,+8 76 stations AM AM_PP H8 H8_PP Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS 76 stations [,] (,] (,7] (7,] (,Inf] Sum AM (,] (,7] (7,] (,Inf] AM_PP (,] (,7] (7,] (,Inf] Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] H8 (,] (,7] (7,] H8_PP (,] (,7] (7,] (,Inf] (,Inf] Sum Sum

63

64 6 EASTERN NORWAY 6. Wind gust 6. Wind gust ME 68 stations AM_FG H8_FG AM_9 H8_ Lead time 6 AM_FG H8_FG AM_9 H8_9 SDE 68 stations Lead time... AM_FG H8_FG AM_9 H8_9 MAE 68 stations Lead time 6

65 6 EASTERN NORWAY 6. Wind gust..8 Lead time [h]: +,+6,..,+ Hit Rate 68 stations AM_FG H8_FG AM_9 H8_ Exceedance threshold..8 False Alarm Ratio Lead time [h]: +,+6,..,+ AM_FG H8_FG AM_9 H8_9 68 stations Exceedance threshold..8 Equitable Threat Score Lead time [h]: +,+6,..,+ 68 stations AM_FG H8_FG AM_9 H8_ Exceedance threshold 6

66

67 6 EASTERN NORWAY 6. Temperature m 6. Temperature m. ME 9 stations. ME [C]... AM H8 H Lead Time SDE 9 stations.. AM H8 H SDE [C] Lead Time MAE 9 stations..8 AM H8 H MAE [C] Lead Time 6

68 6 EASTERN NORWAY 6. Temperature m ME.... Lead time [h]: +,+6,..,+8 9 stations... AM H8 H SDE. Lead time [h]: +,+6,..,+8 9 stations.... AM H8 H 6

69 6 EASTERN NORWAY 6. Temperature m AM + ME at observing sites forecast means

70 6 EASTERN NORWAY 6. Temperature m AM + ME at observing sites forecast means

71 6 EASTERN NORWAY 6. Temperature m AM + SDE at observing sites forecast means

72 6 EASTERN NORWAY 6. Temperature m AM + SDE at observing sites forecast means

73

74 6 EASTERN NORWAY 6.6 Post processed temperature m 6.6 Post processed temperature m ME 9 stations.. ME [C]... AM AM.KF H8 H8.KF Lead Time SDE 9 stations.. AM AM.KF H8 H8.KF SDE [C] Lead Time MAE 9 stations..8 AM AM.KF H8 H8.KF MAE [C] Lead Time 7

75 6 EASTERN NORWAY 6.6 Post processed temperature m AM AM.KF H8 H8.KF ME Lead time [h]: +,+6,..,+8 9 stations AM AM.KF H8 H8.KF..... SDE Lead time [h]: +,+6,..,+8 9 stations 7

76

77 6 EASTERN NORWAY 6.7 Daily precipitation 6.7 Daily precipitation ME stations.. AM AM.med H8 H [mm]... Lead Time [h] SDE stations.. AM AM.med H8 H [mm]... Lead Time [h].. AM AM.med H8 H MAE stations [mm].. Lead Time [h] 7

78 6 EASTERN NORWAY 6.7 Daily precipitation. Lead time [h]: + HR stations AM AM.med H8 H Exceedance threshold [mm]. Lead time [h]: + FAR stations AM AM.med H8 H Exceedance threshold [mm]..8 Lead time [h]: + ETS stations AM AM.med H8 H Exceedance threshold [mm] 7

79 6 EASTERN NORWAY 6.7 Daily precipitation Lead time [h]: +,+ ME stations AM AM.med H8 H SDE Lead time [h]: +,+ stations 8 6 AM AM.med H8 H Lead time [h]: +,+ OBS [,.] (.,] (,] (,] (,Inf] Sum [,.] [,.] OBS stations [,.] (.,] (,] (,] (,Inf] Sum AM (.,] (,] (,] (,Inf] AM.med (.,] (,] (,] (,Inf] Sum Sum OBS OBS [,.] (.,] (,] (,] (,Inf] Sum [,.] (.,] (,] (,] (,Inf] Sum [,.] [,.] (.,] (,] (,] H8 (.,] (,] (,] (,Inf] (,Inf] Sum Sum

80

81 6 EASTERN NORWAY 6.7 Daily precipitation AM + ME at observing sites forecast means

82 6 EASTERN NORWAY 6.7 Daily precipitation AM + SDE at observing sites forecast means

83 7 Western Norway 7. Comments to the verification results Wind speed m: For the period st of March to th of May Arome, Hirlam and Hirlam8 have a small positive bias in wind speed during night time and a small negative bias in the afternoon. has a negative bias at both day time and night times. Arome scores best for wind speeds below ms and Hirlam 8 scores best for wind speeds above ms. Max mean wind speed m: For Max Mean Wind Speed, both Arome and Hirlam8 have a negative bias. After postprocessing both Arome and Hirlam8 have small negative biases. For max mean wind speed Arome scores better for wind speeds below ms, while Arome and Hirlam are equal above ms. Wind gust: For wind gust Arome has a negative bias around ms, while Hirlam8 has a positive bias between. and ms. If we look at wind speed at 9 hpa (which is often used as an estimate of wind gust), there are only minor differences in bias between the Arome and Hirlam8. Both have a positive bias around ms at midnight and a negative bias around ms around noon. Wind gust from Arome scores best for wind speeds below ms, while Hirlam8 scores better for stronger wind gust. Wind gust from Arome and Hirlam8 score better than wind speed at 9 hpa for wind gust below ms, while wind speed at 9 hpa score better for wind gust above ms. Temperature m: For temperature Hirlam, Hirlam8, Arome and all have a negative bias. Both Arome and have an increasing negative bias with lead time. After post-processing both Arome and Hirlam8 have almost no bias. Precipitation: For precipitation Arome, Hirlam8 and Hirlam have almost no bias, while have a positive bias. Arome scores best for both light and heavy precipitation, while scores better for precipitation between and mm pr hours. Case VV: Fog over the North Sea: On June -, AROME has a large area with fog over the North Sea, while satellite images show no fog.

84 7 WESTERN NORWAY 7. Comments to the verification results Figure : Figures from the June at 6 UTC. Left: Fog from AROME. Right: Satellite image. 8

85

86 7 WESTERN NORWAY 7. Pressure 7. Pressure.. AM H8 H ME stations. [hpa] Lead Time [h] AM H8 H SDE stations [hpa] Lead Time [h] MAE stations.. AM H8 H [hpa] Lead Time [h] 8

87 7 WESTERN NORWAY 7. Pressure ME.8 Lead time [h]: +,+6,..,+8 stations AM H8 H SDE Lead time [h]: +,+6,..,+8 stations... AM H8 H 8

88

89 7 WESTERN NORWAY 7. Wind Speed m 7. Wind Speed m.. AM H8 H ME 7 stations [m/s] Lead time [h].. AM H8 H SDE 7 stations [m/s] Lead time [h] MAE 7 stations. AM H8 H. [m/s] Lead time [h] 86

90 7 WESTERN NORWAY 7. Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate 7 stations AM H8 H Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM H8 H False Alarm Ratio 7 stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score 7 stations AM H8 H Exceedance threshold [m/s] 87

91 7 WESTERN NORWAY 7. Wind Speed m ME Lead time [h]: +,+6,..,+8 7 stations AM H8 H SDE. Lead time [h]: +,+6,..,+8 7 stations AM H8 H Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS 7 stations [,] (,] (,7] (7,] (,Inf] Sum (,] (,] AM (,7] (7,] H8 (,7] (7,] (,Inf] (,Inf] 8 Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] (,] (,7] (7,] H (,] (,7] (7,] (,Inf] 9 (,Inf] 6 Sum Sum

92

93 7 WESTERN NORWAY 7. Wind Speed m AM + ME at observing sites forecast means

94 7 WESTERN NORWAY 7. Wind Speed m AM + SDE at observing sites forecast means

95

96 7 WESTERN NORWAY 7. Max Mean Wind Speed m 7. Max Mean Wind Speed m ME 7 stations.. AM AM_PP H8 H8_PP [m/s] Lead time [h] SDE 7 stations.. AM AM_PP H8 H8_PP [m/s] Lead time [h] MAE 7 stations. AM AM_PP H8 H8_PP [m/s] Lead time [h] 9

97 7 WESTERN NORWAY 7. Max Mean Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate 7 stations AM AM_PP H8 H8_PP Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM AM_PP H8 H8_PP False Alarm Ratio 7 stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score 7 stations AM AM_PP H8 H8_PP Exceedance threshold [m/s] 9

98 7 WESTERN NORWAY 7. Max Mean Wind Speed m ME Lead time [h]: +,+6,..,+8 7 stations AM AM_PP H8 H8_PP SDE Lead time [h]: +,+6,..,+8 7 stations..... AM AM_PP H8 H8_PP Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS 7 stations [,] (,] (,7] (7,] (,Inf] Sum AM (,] (,7] (7,] (,Inf] AM_PP (,] (,7] (7,] (,Inf] Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] H8 (,] (,7] (7,] H8_PP (,] (,7] (7,] (,Inf] 8 (,Inf] 6 7 Sum Sum

99

100 7 WESTERN NORWAY 7. Wind gust 7. Wind gust AM_FG H8_FG AM_9 H8_9 ME 67 stations Lead time SDE 67 stations.. AM_FG H8_FG AM_9 H8_ Lead time MAE 67 stations.. AM_FG H8_FG AM_9 H8_ Lead time 97

101 7 WESTERN NORWAY 7. Wind gust..8 Lead time [h]: +,+6,..,+ Hit Rate 67 stations AM_FG H8_FG AM_9 H8_ Exceedance threshold..8 False Alarm Ratio Lead time [h]: +,+6,..,+ AM_FG H8_FG AM_9 H8_9 67 stations Exceedance threshold..8 Equitable Threat Score Lead time [h]: +,+6,..,+ 67 stations AM_FG H8_FG AM_9 H8_ Exceedance threshold 98

102

103 7 WESTERN NORWAY 7.6 Temperature m 7.6 Temperature m. ME 9 stations. ME [C]... AM H8 H Lead Time SDE 9 stations.7.6 AM H8 H SDE [C] Lead Time.7.6. AM H8 H MAE 9 stations MAE [C] Lead Time

104 7 WESTERN NORWAY 7.6 Temperature m AM H8 H ME Lead time [h]: +,+6,..,+8 9 stations AM H8 H.... SDE Lead time [h]: +,+6,..,+8 9 stations

105 7 WESTERN NORWAY 7.6 Temperature m AM + ME at observing sites forecast means

106 7 WESTERN NORWAY 7.6 Temperature m AM + ME at observing sites forecast means

107 7 WESTERN NORWAY 7.6 Temperature m AM + SDE at observing sites forecast means

108 7 WESTERN NORWAY 7.6 Temperature m AM + SDE at observing sites forecast means

109

110 7 WESTERN NORWAY 7.7 Post processed temperature m 7.7 Post processed temperature m ME 9 stations. ME [C]... AM AM.KF H8 H8.KF Lead Time SDE [C] AM AM.KF H8 H8.KF SDE 9 stations Lead Time MAE 9 stations.. AM AM.KF H8 H8.KF MAE [C] Lead Time 7

111 7 WESTERN NORWAY 7.7 Post processed temperature m ME Lead time [h]: +,+6,..,+8 9 stations AM AM.KF H8 H8.KF SDE Lead time [h]: +,+6,..,+8 9 stations.... AM AM.KF H8 H8.KF 8

112

113 7 WESTERN NORWAY 7.8 Daily precipitation 7.8 Daily precipitation [mm]..... AM AM.med H8 H ME 6 stations.. Lead Time [h] AM AM.med H8 H SDE 6 stations [mm]... Lead Time [h].. AM AM.med H8 H MAE 6 stations [mm].. Lead Time [h]

114 7 WESTERN NORWAY 7.8 Daily precipitation. Lead time [h]: + HR 6 stations AM AM.med H8 H Exceedance threshold [mm]. Lead time [h]: + FAR 6 stations Exceedance threshold [mm] AM AM.med H8 H..8 Lead time [h]: + ETS 6 stations AM AM.med H8 H Exceedance threshold [mm]

115 7 WESTERN NORWAY 7.8 Daily precipitation ME Lead time [h]: +,+ 6 stations AM AM.med H8 H SDE Lead time [h]: +,+ 6 stations AM AM.med H8 H Lead time [h]: +,+ OBS [,.] (.,] (,] (,] (,Inf] Sum [,.] [,.] OBS 6 stations [,.] (.,] (,] (,] (,Inf] Sum AM (.,] (,] (,] (,Inf] AM.med (.,] (,] (,] (,Inf] Sum Sum OBS OBS [,.] (.,] (,] (,] (,Inf] Sum [,.] (.,] (,] (,] (,Inf] Sum [,.] [,.] (.,] (,] (,] H8 (.,] (,] (,] (,Inf] 6 9 (,Inf] 6 7 Sum Sum

116

117 7 WESTERN NORWAY 7.8 Daily precipitation AM + ME at observing sites forecast means

118 7 WESTERN NORWAY 7.8 Daily precipitation AM + SDE at observing sites forecast means

119 8 Northern Norway 8. Comments to the verification results Case VNN: Complex topography, Arome.km and preciptiation distribution Episode with heavy precipitation in Vest-Finnmark May - Synoptic situation: Slow moving system with semi-stationary occlusion front. Feature: Heavy precipitation, 9 observations with -mm/h. Several "seasonal" records. Model AROME 8Z.. Max values: mm/h (raw) 7mm/h (pp-max) mm/h (pp-median) Max values lowland (<m): mm/h (estimated with topography mask.) Comparing model with observations (mainly in lowland) shows that the model overall underpredicted the amounts. However, the model have much more precipitation at higher elevations, where we have no observations. It is important to compare the precipitation fields with elevation masks in complex terrain, it is otherwise easy to overpredict the amounts in lower, habitated areas. Current postprossessing that smooth precipitation fields (pp-median, pp-max) may sometimes give a better estimates compared to observations. Unfortunately there are a lot a variations between cases, due to topography and precipitation distribution, so it is hard to tell if one of the precip-fields is generally better than the others. Comparing them all, including elevation mask and/or habitated areas is a common practise among experienced forecasters.

120 8 NORTHERN NORWAY 8. Comments to the verification results Figure : Figures from the heavy precipitation event May -. Topleft: Satellite image and forecasters analysis showing the synoptic situation. Topright: Accumulated h-precipitation field from AROME. Bottomleft: Max values lowland estimated with topography mask. Bottomright: Scatterplot of observed and forecasted h-precipitation. 7

121

122 8 NORTHERN NORWAY 8. Pressure 8. Pressure ME stations. AM H8 H. [hpa] Lead Time [h] AM H8 H SDE stations [hpa] Lead Time [h] MAE stations.. AM H8 H. [hpa] Lead Time [h] 9

123 8 NORTHERN NORWAY 8. Pressure ME Lead time [h]: +,+6,..,+8 stations... AM H8 H SDE.. Lead time [h]: +,+6,..,+8 stations AM H8 H

124

125 8 NORTHERN NORWAY 8. Wind Speed m 8. Wind Speed m ME 7 stations.. AM H8 H [m/s] Lead time [h] SDE 7 stations. AM H8 H. [m/s] Lead time [h] MAE 7 stations. AM H8 H [m/s] Lead time [h]

126 8 NORTHERN NORWAY 8. Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate 7 stations AM H8 H Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM H8 H False Alarm Ratio 7 stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score 7 stations AM H8 H Exceedance threshold [m/s]

127 8 NORTHERN NORWAY 8. Wind Speed m ME Lead time [h]: +,+6,..,+8 7 stations AM H8 H SDE Lead time [h]: +,+6,..,+8 7 stations AM H8 H Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS 7 stations [,] (,] (,7] (7,] (,Inf] Sum (,] (,] AM (,7] (7,] H8 (,7] (7,] (,Inf] (,Inf] 9 6 Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] (,] (,7] (7,] H (,] (,7] (7,] (,Inf] (,Inf] 6 Sum Sum

128

129 8 NORTHERN NORWAY 8. Wind Speed m AM + ME at observing sites forecast means

130 8 NORTHERN NORWAY 8. Wind Speed m AM + SDE at observing sites forecast means

131

132 8 NORTHERN NORWAY 8. Max Mean Wind Speed m 8. Max Mean Wind Speed m ME 7 stations. AM AM_PP H8 H8_PP. [m/s] Lead time [h] SDE 7 stations. AM AM_PP H8 H8_PP. [m/s] Lead time [h].. AM AM_PP H8 H8_PP MAE 7 stations [m/s] Lead time [h] 9

133 8 NORTHERN NORWAY 8. Max Mean Wind Speed m..8 Lead time [h]: +,+6,..,+ Hit Rate 7 stations AM AM_PP H8 H8_PP Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ AM AM_PP H8 H8_PP False Alarm Ratio 7 stations Exceedance threshold [m/s]..8 Lead time [h]: +,+6,..,+ Equitable Threat Score 7 stations AM AM_PP H8 H8_PP Exceedance threshold [m/s]

134 8 NORTHERN NORWAY 8. Max Mean Wind Speed m ME Lead time [h]: +,+6,..,+8 7 stations AM AM_PP H8 H8_PP SDE 6 Lead time [h]: +,+6,..,+8 7 stations AM AM_PP H8 H8_PP Lead time [h]: +,+6,..,+8 UTC OBS [,] (,] (,7] (7,] (,Inf] Sum [,] [,] OBS 7 stations [,] (,] (,7] (7,] (,Inf] Sum AM (,] (,7] (7,] (,Inf] AM_PP (,] (,7] (7,] (,Inf] Sum Sum OBS OBS [,] (,] (,7] (7,] (,Inf] Sum [,] (,] (,7] (7,] (,Inf] Sum [,] [,] H8 (,] (,7] (7,] H8_PP (,] (,7] (7,] (,Inf] 6 6 (,Inf] Sum Sum

135

136 8 NORTHERN NORWAY 8. Wind gust 8. Wind gust AM_FG H8_FG AM_9 H8_9 ME 67 stations Lead time SDE 67 stations.. AM_FG H8_FG AM_9 H8_ Lead time MAE 67 stations.. AM_FG H8_FG AM_9 H8_ Lead time

137 8 NORTHERN NORWAY 8. Wind gust..8 Lead time [h]: +,+6,..,+ Hit Rate 67 stations AM_FG H8_FG AM_9 H8_ Exceedance threshold..8 False Alarm Ratio Lead time [h]: +,+6,..,+ AM_FG H8_FG AM_9 H8_9 67 stations Exceedance threshold..8 Equitable Threat Score Lead time [h]: +,+6,..,+ 67 stations AM_FG H8_FG AM_9 H8_ Exceedance threshold

138

139 8 NORTHERN NORWAY 8.6 Temperature m 8.6 Temperature m ME 8 stations. ME [C].... AM H8 H Lead Time.8.7 AM H8 H SDE 8 stations.6 SDE [C] Lead Time MAE 8 stations. AM H8 H. MAE [C] Lead Time 6