Quality Assurance in Atmospheric Modeling

Size: px

Start display at page:

Download "Quality Assurance in Atmospheric Modeling"

Theodore Campbell
8 years ago
Views:

1 Quality Assurance in Atmospheric Modeling To Appear in the Bulletin of the American Meteorological Society Thomas T. Warner

2 An Explosion in Mesoscale Model Usage

3 Rapid Increase in Model Users Due to: Easy access to and use of models Availability of compucng resources Value of model- generated atmospheric informacon (observing capabilices develop slowly, model resolucon scales with computer development) Expected use of modeling in most research New applicacons (regional climate)

4 So, you want to run a mesoscale model? 14 Steps toward improving the outcome

5 1. Clearly Define the ScienCfic or PracCcal ObjecCve Hypothesis to be tested? How will you know whether you are successful? Is this a fishing trip? Do you really need a mesoscale model? uclafacultyassociation.blogspot.com

6 2. IdenCfy and Develop a Physical Understanding of the Atmospheric Processes that must be Accurately Simulated. Necessary to design simulacon(s) what processes are important? (conveccon, radiacon, cloud physics, snowcover, etc.) Necessary to judge efficacy of simulacon Not complete understanding a[er all this may be the reason for the study but enough to guide choices in methodology

7 3. Perform a Thorough Analysis of all Available ObservaCons Increase familiarity with phenomena or processes under invescgacon Know how and where model results may be helpful Be be_er able to judge adequacy of simulacons sriks6711.wordpress.com

8 4. Prepare an Experimental Design Case study? Which case? Is a real- case simulacon the best strategy or should it be idealized? Do we need a model at all? Ensemble of cases? Process study? How will verificacon be done?

9 5. Define the model configuracon Domain ResoluCon (horizontal and verccal)? eo.ucar.edu

10 What do Mesoscale Models Resolve? 7 x

11 6. Don t run the model just yet. Running the model before steps 1-5 are complete could simply waste Cme. (the sooner the model is run, the longer the study will take) Re- evaluate goals and design of study as necessary.

12 7. Choose Cme and method of inicalizacon InterpolaCon from other model? VariaConal data assimilacon? Ensemble data assimilacon? What spinup is allowed? Knowledge of techniques? abarnyard.com

13 8. Evaluate sensicvity to lateral boundary locacons and specificacon (if limited area) Tradeoff between large domain (computaconal cost) and small domain (determinacon of solucon by boundaries Open or periodic (idealized) Quality of model/data on lateral boundaries sonatinassong.blogspot.com

14 Influence of Lateral Boundaries Figure. Twelve-hour simulations of 250-hPa winds (m s-1) from the 40-km grid increment Eta Model initialized at 1200 UTC 3 August 1992, based on experiments that used a large (a) and small (b) computational domain. The isotach interval is 5 m s-1. From Treadon and Peterson (1993).

15 9. Define the most appropriate physical parameterizacons. Braham, 1974, BAMS

16 Different Schemes, Different Results Figure 4. Average rainfall rate, for a spring-season convective event (a), based on observations (OBS) and for five simulations that used different treatments for the convection - four different parameterizations, and no parameterization (EX). Also depicted is the rainfallrate bias score averaged for three warm-season convective events (b), again for each of the four parameterizations and for the use of no parameterization. The four convective parameterizations were the Grell (GR), Kain-Fritsch (KF), Betts-Miller (BM), and Anthes- Kuo (AK) schemes. Adapted from Wang and Seaman (1997).

17 10. Understand the limits to predictability of the phenomenon being studied It's tough to make predictions, especially about the future. Yogi Berra Errors affect progressively larger scales with time Zhang, Fuqing, Naifang Bei, Richard Rotunno, Chris Snyder, Craig C. Epifanio, 2007: Mesoscale Predictability of Moist Baroclinic Waves: Convection-Permitting Experiments and Multistage Error Growth Dynamics. J. Atmos. Sci., 64, nndb.com

18 11. Establish a verificacon plan before the model is run and perform a thorough verificacon, using appropriate metrics, of the model solucon using all available observacons. Why before? Because model output may vary with verificacon method? Importance? Confidence in model use (process study or forecast) QuanCfy strengths and weaknesses Assure right answer for right reason. Choose appropriate verificacon metrics

19 Beware of VerificaCon Metrics

20 12. Be organized Maintain an experimental log Be systemacc about what is archived Note: Tom s office NEVER looked like this

21 13. Use good coding praccces and well- documented so[ware Will new code be developed? Black- box vs. white- box tescng Be able to repeat your results (because you will probably have to) Upgrades to community codes use newest version? Latest version could have major bugs Does upgrade affect reproducibility?

22 14. Use open- source so[ware tools to improve the efficiency of the modeling process Use version- control so[ware to keep track of model versions Use well- documented packages Open- source packages are available to facilitate interpretacon of model output Calc.f Calc.f_old Calc.f_new Calc.f_new_a Calc.f_new_b Calc.f_older Calc.f_somewhat_newer Calc.f_i-cant-remember Calc.f_err Calc.f_err1

23 Conclusions Research models are tools, and like any tool, must be used properly Modeler should be as familiar with observacons as the model More planning and preparacon of simulacons leads to shorter (and be_er) duracon of studies It is encrely possible to set out to do a modeling study and never run the model.

24 Tom s Legacy Understand how nature works, using whatever tools are needed Know the observacons first Run models last, not first Confront model results with observacons and meteorological insight Model to teach and teach to model

Aformal definition of quality assurance that is

Aformal definition of quality assurance that is QUALITY ASSURANCE IN ATMOSPHERIC MODELING by Thomas T. Warner The rapid growth in the number of atmospheric model users is motivation for reviewing best practices in atmospheric modeling and emphasizing