met.no AeroCom tools for HTAP Michael Schulz, Jan Griesfeller MetNo cf,wcs: Snehal Waychal, Martin Schultz, Michael Decker FZJulich Geneve 20.03.13 acknowledgment: MetNo, EU service contract
Aerocom Tools for HTAP - Overview Goal for provision of AeroCom services Facilitate analysis of multi-model output Document model progress Support EMEP/UNECE assessments Underpin policy implementation with model calculations Specific goal of presentation Give instructions and web addresses Collect open questions OpQu PLEASE INTERRUPT
What is AeroCom (aerocom.met.no) Evaluate global aerosol models, fair and transparent > 20 models and versions of the them documented 4 recent Phase II publications on forcing just published ACP 2013, in support of 5 th IPCC (Myhre et al., Stier et al., Samset et al., Randles et al.) Annual workshops since 2002 10 years AeroCom 23-26 September 2013 MPI/Hamburg
HTAP model data protocol Intput definition Output Protocols have been combined HTAP-AeroCom-ACCMIP-CMIP5 Excel file available (MS, FD) CF convention for of standard names units, definiton, netcdf attributes http://cf-pcmdi.llnl.gov/documents OpQu : Table for synonymes of variables? Missing variables? Proposition for Expansion of CF? (with consulting Martin Schultz)
http://cf-pcmdi.llnl.gov/documents
Input and Output for participating models http://aerocom. met.no/download/ 1 TB is available for providing emissions, boundary conditions for models Output harmonization Recommendation for tools: cmor, cdo, nco, fortran http://aerocom.met.no/formattools.html OpQu : Should we provide nco solutions for cf checker error messages? Which boundary conditions to store? Size?
If you want to check your netcdf format Use the online cf checker Developed by Julich Running at met.no
http://aerocom-test.met.no/upload
If you want to check the format Instructions: Cut out first time steps with ncks d time,0,10 in.nc out.nc (saves time for upload and testing) Go to and upload file out.nc http://aerocom-test.met.no/upload OpQu : Range check to be refined Quick enough for large files, multiple users? Can we link to massive upload? Non regular grids? Force all modellers to be cf compliant? nco help library to be build?
After files are cf checked == ftp upload User registration is needed, send email to jan.griesfeller@met.no Upload to ftp.aerocom.met.no Send email to Jan once ftp is completed Give us a short Experiment Name (Model-Version-ExpName, max 25 char) Files are transfered manually to database And are removed! from ftp server by Jan OpQu : Naming convention for HTAP II files? Eventually automatization with cf checker?
Data flow / AeroCom database http://aerocom.met.no/data.html Modeller Scientist Student End-user User server aerocom-users.met.no /metno/aerocom/users/ aerocom1/htap Database read only wcs clients WCS server Webdab server Upload tool OR ftp server rsync every night Original AeroCom / HTAP database behind MetNo Firewall
How to document model experiments? Minimum requested from modeller: Ascii readme file and proper netcdf attributes with model references, authors, institution Done on metno side: small renamings, to make an order xml catalogue for wcs server AeroCom catalogues & ascii tables OpQu : htap wiki questionaries? What can we use from CMIP5?
Planned HTAP model output processing Overall Model Performance Visualization Dokumented via AeroCom web interface Where compared? Correlated? Regional Normalized Bias? Local month to month? Histogram? Zonal Absolute Bias?
Remarks on database security and dimension Database is behind met.no firewall => Copy to user server by rsync Disks are scaleable, but what do we need? OpQu : How large? Who pays? Do we need to protect htap output from other aerocom database users (aerocom, cciaerosol, EU projects, EMEP) Data policy https://wiki.met.no/aerocom/data_policy?
Multiple pathways to accces the model data!! User server linux ubuntu precise w. emacs, scp, cdo, nco, ncview, +? Requires user registration (jan.griesfeller.met.no) Please send motivation paragraph wcs client at met.no (being prepared by Julich) wget using wcs client OpQu : How large do we need to dimension wcs? webdab in addition? dods in additon?
Aerocom Tools for HTAP - Overview Model ouput for HTAP will be grouped Access can be restricted to group OpQu : Do we want to install further subprojects? For analysis? Could be reflected in disk organisation, access, analysis, docu
Aerocom idl tools, what it is Evolving idl library Shared with co-developers Combines reading obs and model Standard catalogues and scores Development in funded projects Image and ascii output Fast batch processing Comparable across model generations
Statistics: AATSR4Land Recent example: cci-aerosol Scoring different ATSR retrievals Model name NumObs #R CORR RMS NMB % RMSbc AATSR_ADV.v1.42 1394 0,822 0,102 29,7 0,105 AATSR_ORAC.v2.02 1394 0,823 0,091 9,4 0,091 AATSR_SU_v3.1 1370 0,799 0,102 27,4 0,101 AATSR_SU_v4.0 1394 0,863 0,081 7,7 0,083 MISR_V31_1x1 276 0,856 0,085 11,2 0,081 MODIS5.1aqua 1185 0,749 0,114 7,1 0,108 MODIS5.1terra 1285 0,744 0,114 1,5 0,113
Recent example: MACC-II Scoring Reanalysis versus Aeronet Correlation AOD MACC Aeronet Bias MACC reanalysis 2003-2011 Modis terra and aqua AOD Assimilated in Aerosol model Evaluated at Aeronet sites MACC-REA Validation report Dec 2012, Aerosol chapter
Web interface to AeroCom idl tool images http://aerocom.met.no/cgi-bin/aerocom/ surfobs_annualrs.pl Free access Image catalogue search New: Last choices rememberd Observational Network Difference maps to «reference» OpQu : Link to google map for time series plots? of scores Publication quality of images?
New feature of aerocom interface remembering choices Mylist feature (model selection stored in cookie..) =>
New feature Comparison to Reference Difference MACC IFS SON 2012 against SON MACC climatology 2003-2010
Idl tools specific link to EBAS/NILU database Idl tools contain specific reading routines per parameter from NASA Ames Use massive download from NILU Feedback on data quality into EBAS Usage of EBAS info on data source, instrument, Infp copied into time series plots
Example EBAS info into time series plot EMEP vs PM10, 2009 Instrument matrix, component Network
Cataloging benchmark datasets For aerosol model evaluation http://aerocom.met.no/databenchmarks.htm l Listing on aerocom website Advertise consolidated important datasets Development eg with ACTRIS, AeroCom NILU complements website, but for EBAS OpQu : Any datasets you would like to be listed? Aerocom median? HTAP medians? MACC-IFS climatology? Latest EMEP runs?
Thanks for your attention Any OpQu?