Content. Benchmark database. inhomogeneous data, surrogate data and synthetic data. Benchmark dataset. Creation benchmark Outline talk

Size: px

Start display at page:

Download "Content. Benchmark database. inhomogeneous data, surrogate data and synthetic data. Benchmark dataset. Creation benchmark Outline talk"

Cathleen Ross
7 years ago
Views:

1 Benchmark database inhomogeneous data, surrogate data and synthetic data Victor Venema Content Introduction to benchmark dataset Some results Some questions about exercise Question about future work Analyse and publish the results Meteorological Institute Bonn Benchmark dataset ) Real (inhomogeneous) climate records Most realistic case Investigate if various HA find the same breaks ) Synthetic data For example, Gaussian white noise Insert know inhomogeneities Test performance 3) Surrogate data Empirical distribution and correlations Insert know inhomogeneities Compare to synthetic data: test of assumptions Creation benchmark Outline talk ) Start with homogeneous data ) Multiple surrogate and synthetic realisations 3) Mask surrogate records ) Add global trend ) Insert inhomogeneities in station time series ) Published on the web 7) Homogenize by COST participants and third parties 8) Analyse the results and publish ) Start with homogeneous data Monthly mean temperature and precipitation Later also daily data (WG), maybe other variables (pressure, wind) Homogeneous, no missing data Longer surrogates are based on multiple copies Generated networks are a ) Multiple surrogate realisations Multiple surrogate realisations Temporal correlations Station cross-correlations Empirical distribution function Annual cycle removed before, added at the end Number of stations,, 9 or Cross correlation varies as much as possible

2 ) Insert inhomogeneities in stations Independent breaks Determined at random for every station and time Breaks per a Monthly slightly different perturbations Temperature Additive Size: Gaussian distribution, σ=.8 C Rain Multiplicative Size: Gaussian distribution, <x>=, σ=% Example break perturbations station Example break perturbations network Temperature perturbations ) Insert inhomogeneities in stations Correlated break in network One break in % of networks In 3 % of the station simultaneously Position random At least % of data points on either side Example correlated break Correlated break ) Insert inhomogeneities in stations Outliers Size Temperature: < or > 99 percentile Rain: <. or > 99.9 percentile Frequency % of networks: % % of networks: 3 %

3 Example outlier perturbations station Example outliers network Outliers Outliers ) Insert inhomogeneities in stations Example local trends Local trends (only temperature) Linear increase or decrease in one station Duration: between 3 and a Maximum size: Gaussian distribution, σ=.8 C Frequency: once in % of the stations 8 Local trends ) Published on the web Inhomogeneous data are published on the COST- HOME homepage Everyone is welcome to download and homogenize the data venema/themes/homogenisation 7) Homogenize by participants Return homogenised data Should be in COST-HOME file format (next slide) For real data including quality flags Return break detection file BREAK OUTLI BEGTR ENDTR Multiple breaks at one data possible 3

4 Typical errors COST-HOME file format network file The file format needs to be perfect! Forgetting the station-file that describes which stations belong to the homogenised network Changing the file names in this station file to homogeneous data files (Forgetting to return the files with the quality flags) The sizes of the breaks are not in the break file Please, keep directory structure of the benchmark like it is, also for partial contributions The only difference is the main directory All files are tab-delimited ASCII files Typical errors Detected breaks file The file format needs to be perfect! Forgetting the station-file that describes which stations belong to the homogenised network Changing the file names in this station file to homogeneous data files (Forgetting to return the files with the quality flags) The sizes of the breaks are not in the break file Please, keep directory structure of the benchmark like it is, also for partial contributions The only difference is the main directory All files are tab-delimited ASCII files Typical errors see discussion COST-HOME file format monthly data The file format needs to be perfect! Forgetting the station-file that describes which stations belong to the homogenised network Changing the file names in this station file to homogeneous data files (Forgetting to return the files with the quality flags) The sizes of the breaks are not in the break file Please, keep directory structure of the benchmark like it is, also for partial contributions The only difference is the main directory All files are tab-delimited ASCII files

5 Contributions No. homogenised networks - algorithm Participant Algorithm Remarks Table. Number of homogenised networks per algorithm. José Guijarro. Péter Domonkos 3. Michele. Dubravka Rasol & Olivier Mestre. Matthew Menne & Claude Williams. Christine Gruber & Ingeborg Auer 7. Gregor Vertacnik 8. Petr Stepanek 9. Lucie. Enric Aguilar Climatol CM-D, -D, NSHT-D Automated pairwise hom. NSHT Versions with different settings 3 Versions / detection algorithms Detection Craddock based; surrogate temp. networks All surrogate temp.; 3 surrogate precip. Networks Versions; all temp. Networks (part of real #3 is missing) Surrogate temp. & surrogate precip. All surrogate temp. Surrogate temp. & surrogate precip. Surrogate temp. Not in the right format yet Homogenisation alg. Climatol A Climatol C Climatol D Climatol E Climatol F ClimatolG APHa APHa CM-D -D SNHT-D All networks Real netw. Surrogate netw Synthetic netw. 9 9 No. homogenised networks input data Mean no. outliers per station Table. Mean number of outliers per station for every algorithm Table 3. Summary data: Number of homogenised networks per network Network No. networks Temp. netw. Precip. netw. All 37 3 Homogenisation alg. All networks. 3.. Real netw. Surrogate netw Synthetic netw. Real Surrogate Surrogate # Climatol A Climatol C Surrogate ~# 87 7 Climatol D Climatol E Synthetic Synthetic # Synthetic ~# Climatol F ClimatolG APHa APHa CM-D.7.7 -D.. SNHT-D.3.3 Mean no. breaks per station Homogenising the exercise Table. Mean number of breaks per station for every algorithm Homogenisation alg. All networks Real netw. Surrogate netw Climatol A.3.8. Climatol C..8. Climatol D... Climatol E..9.8 Climatol F...7 ClimatolG.. APHa.8.. APHa.7..9 CM-D.. -D SNHT-D Synthetic netw Tab-delimited files: also space-delimited? Mixture of strings and numbers Data quality files only for real data section Do we want to use the Diurnal Temperature Range (DTR)? Not useful for surrogate and synthetic data! If we do, everyone should do it End or begin uncorrected? Compute statistics independent of absolute level? Filling missing values part exercise? Human quality control or raw algorithm output? Homogenise all or homogenisable networks, times

6 Contributions who is missing? Analysing the results Participant. José Guijarro. Péter Domonkos 3. Michele. Dubravka Rasol & Olivier Mestre. Matthew Menne & Claude Williams. Christine Gruber & Ingeborg Auer 7. Gregor Vertacnik 8. Petr Stepanek 9. Lucie. Enric Aguilar Algorithm Climatol CM-D, -D, NSHT-D Automated pairwise hom. NSHT Remarks Versions with different settings 3 Versions / detection algorithms Detection Craddock based; surrogate temp. networks All surrogate temp.; 3 surrogate precip. Networks Versions; all temp. Networks (part of real #3 is missing) Surrogate temp. & surrogate precip. All surrogate temp. Surrogate temp. & surrogate precip. Surrogate temp. Not in the right format yet What measures define a well homogenised dataset? Real data vs. data with known truth Ensemble mean for real data? Breaks Position, hit rate size distribution detection probability as function of size Data itself Root mean square error (RMSE) RMSE (without outliers) RMSE (bias corrected) Uncertainty in the network mean trend How to study which components are best? Deadline(s) Agreed on 9/9, September this year Multiple deadlines For example: synthetic data, real data, surrogate data After deadline the truth can be revealed After deadline the other contributions can be revealed(?) Start earlier analysing the results For example: May, July, September Bologna, May, EGU, 9 April Articles Articles Overview COST Action & benchmark with very basic analysis results Performance difference between synthetic (Gaussian, white noise) and surrogate data How to deal multiple contributions per algorithm? Do we have references to all algorithms? What should the others be about Analysing results, which components are best Who will organise, coordinate it? Not everyone should do the same analysis How to subdivide the work? After deadline: sensitivity analysis

Uncertainties in the Surface Temperature Record

Uncertainties in the Surface Temperature Record Matthew Menne & Claude Williams NOAA/National Climatic Data Center Asheville, North Carolina 1 NOAA/National Climatic Data Center The datasets Sources of