Total Ozone Product: Validation Report Doc.No. Issue : : EUM/TSS/DOC/13/706444 v1 EUMETSAT Eumetsat-Allee 1, D-64295 Darmstadt, Germany Tel: +49 6151 807-7 Fax: +49 6151 807 555 Date : 24 May 2013 http://www.eumetsat.int WBS : EUMETSAT The copyright of this document is the property of EUMETSAT.
Document Change Record Issue / Revision Date DCN. No Summary of Changes Version 1 24 May 2013 Document created Table of Contents 1 Introduction... 3 1.1 Applicable Documents... 3 1.2 Document Structure... 3 2 Short summary of the TOZ Retrieval Method... 4 3 Description of the Encountered Problem... 5 4 Updates to the Operational Version... 7 5 Comparisons for the Product Under Validation... 8 6 Conclusions... 15 Table of Figures Figure 1: Snapshot of the TOZ product over Europe for 25 January 2013, 0500 UTC.... 5 Figure 2: Same as Figure 1, but for the scientific prototype.... 5 Figure 3: Same as Figure 1, but now with the updates as outlined in this section.... 7 Figure 4: TOZ results for the product under validation (upper left), the operational product (upper right), and for the scientific prototype (bottom), for 15 May 2013, 0000 UTC.... 8 Figure 5: As Figure 4, but for 0300 UTC.... 9 Figure 6 : As Figure 4, but for 1200 UTC.... 10 Figure 7: As Figure 4, but for 1500 UTC.... 11 Figure 8: TOZ results for the product under validation (top), the operational product (centre), and for the scientific prototype (bottom), for 15 May 2013, 0000 UTC.... 13 Table of Tables Table 1: Number of adjacent processing elements showing a TOZ difference of 50 DU and higher, for the product under validation, the operational product, and the scientific prototype for the four analysed repeat cycles on 15 May 2013.... 12 Table 2: Number of adjacent processing elements showing a TOZ difference of 50 DU and higher, for the product under validation, and the distribution according the retrieval type (both adjacent elements cloudy, both clear, or one cloudy one clear) for the four analysed repeat cycles on 15 May 2013.... 14 Table 3: Number of adjacent processing elements showing a TOZ difference of 50 DU and higher, for the operational product, and the distribution according the retrieval type (both adjacent elements cloudy, both clear, or one cloudy one clear) for the four analysed repeat cycles on 15 May 2013.... 14 Table 4: Overall difference (in DU) between the product under validation resp. the operational product and the scientific prototype... 14 Page 2 of 15
1 INTRODUCTION This document describes the validation of the latest changes of the MSG Total Ozone (TOZ) product, as it is produced by the Meteorological Products Extraction Facility (MPEF) at EUMETSAT Headquarters. These changes were implemented to resolve a problem observed after the initial product implementation in 2012. The document describes the problem and why it was observed in the operational system but not in the scientific prototype. The analysis of the problem came to a conclusion, largely resolving the problem, which is also described in this document. 1.1 Applicable Documents AD 1 ATBD for the MSG GII / TOZ Product EUM/MET/DOC/11/0247 AD 2 Implementation Document update of the MSG EUM/MET/DOC/09/0538 products GII and TOZ AD 3 Product Validation Report No. 8 (TOZ) EUM/OPS/REP/06/1764 1.2 Document Structure Section 2 recalls the principles of the TOZ retrieval scheme. The problem observed concerning the TOZ results is described in Section 3. Section 4 explains the chosen strategy to mitigate the problem together with some initial results obtained from a test environment, which were then implemented in the operational MPEF. Section 4 shows the results for this MPEF implementation. Page 3 of 15
2 SHORT SUMMARY OF THE TOZ RETRIEVAL METHOD The TOZ product is retrieved together with the MSG global instability (GII) product. The main idea of the retrieval scheme is to retrieve a full profile of atmospheric temperature, humidity and ozone, from which the instability and TOZ parameters are derived. The chosen algorithm approach is a socalled statistical-physical retrieval in that prior information and measurements are combined in a statistically optimal way, with a physical radiative transfer model used to relate atmospheric properties to measurements. Prior information is required because the information contained in the observed radiances is not sufficient to completely define the atmospheric profile. The prior profile (also known as background profile ) is obtained from short range NWP forecasts and also serves as the first guess profile. The physical retrieval is an optimal estimation using an inversion technique, i.e. tries to find an atmospheric profile which best reproduces the observations. In general, this is a multi-solution problem, and a background profile is used as a constraint. The algorithm works differently for clear sky and cloudy situations: In clear sky conditions, the full set of relevant MSG-SEVIRI channel information is explored, and the first guess temperature, humidity, and ozone profile may be changed within the retrieval. The relevant lower boundary condition is the surface skin temperature (which may also be changed by the retrieval). The clear sky results are the full set of GII and TOZ parameters. For cloudy situations, however, no instability information can be derived from the satellite observations (as clouds obstruct the signal from the lowest atmospheric level, which is of high importance for the GII product). Theoretical considerations concerning the information content of the MSG IR 9.7 channel (the ozone channel), together with the fact that most of the atmospheric ozone is in the stratosphere, shows that a meaningful retrieval of total column ozone is possible over low clouds. For this special case, the lower boundary condition within the retrieval needs to be changed to the cloud top pressure. With respect to the atmospheric profile, only the ozone profile is subject to changes within the retrieval, i.e. the cloudy sky result is only the TOZ value. The retrieval method is described in detail in the corresponding Algorithm Theoretical baseline Document [AD 1]. In February 2012, the TOZ product described in this document was made operationally available to users. Page 4 of 15
3 DESCRIPTION OF THE PROBLEM ENCOUNTERED After the product was made available operationally, a problem was identified in the operational version, regarding high variability within the TOZ product between adjacent processing elements. Figure 1 illustrates this. Figure 1: Snapshot of the TOZ product over Europe for 25 January 2013, 0500 UTC. White areas show high cloud situations where no TOZ retrieval is possible, or where the retrieval failed for other reasons. TOZ values are in DU = Dobson Units. Figure 2: Snapshot of the Scientific Prototype over Europe for 25 January 2013, 0500 UTC. White areas show high cloud situations where no TOZ retrieval is possible, or where the retrieval failed for other reasons. TOZ values are in DU = Dobson Units.. Page 5 of 15
In this particular case, high variability is e.g. observed over the Mediterranean region around Italy, or also over the North Atlantic off the west coasts of France and Spain. Comparison to the scientific prototype (Figure 2) shows that the prototype does not show this variability however, at the expense of showing less product coverage, i.e. more white areas, which denote high cloud situations and/or failed retrievals. Initial investigations showed that the high variability occurs mostly in areas where the cloudy retrieval method is used within the processing. The main difference to the prototype here is that the acceptance criteria in the cloudy ozone retrieval used within the operational algorithm is more relaxed compared to the prototype, which was initially done for timeliness reasons. The acceptance criterion for a successful retrieval is the RMS between the observed and the simulated brightness temperatures (using the retrieved profiles in the simulations): RMS T T 2 B B,m n where T B is the observed brightness temperature and T B,m is the simulated brightness temperature. The summation is done over the n channels used in the retrieval, which are 7 MSG IR channels for the clear sky retrieval. In the original prototype code, this acceptance criterion was identical for the clear and cloudy conditions, i.e. the results were always tested against all 7 channels, while the operational version only tested against two channels, IR9.7 and IR10.8. The test against only two channels was the main cause for the observed blocky structure of the results. Page 6 of 15
4 UPDATES TO THE OPERATIONAL VERSION A number of changes were done to the operational TOZ version, in case of cloudy situations. The aim of the changes was to mitigate the influence of the acceptance criterion without a non-negligible increase of the processing time. Offline tests showed that the following settings gave rather good results when compared to the prototype: (1) The cloud top pressure threshold for high cloud situations, where no meaningful TOZ retrieval is possible, was set from 300 hpa to 400 hpa, resulting in a number of processing fields where no retrieval was even attempted. (2) The maximum number of allowed iterations was changed from 6 to 2. (3) The RMS test is done against 3 channels: WV6.2, IR9.7 and IR10.8, as these have the most impact within the retrieval. Figure 3 shows an example from the offline testing for the same case as shown in Figure 1 and Figure 2: The results now look much more in line with the prototype version, and especially the observed high spatial variability has largely disappeared. Figure 3: Same as Figure 1, but now with the updates as outlined in this section. When compared to the operational version, the number of neighbouring processing areas with a TOZ difference of more than 50 DU has decreased from 8919 to 2096. Note: This number is still much smaller in the scientific prototype in this case 51 but it should be noted that the prototype uses forecast data with a much higher vertical resolution, together with a higher spatial and temporal resolution. Page 7 of 15
5 COMPARISONS FOR THE PRODUCT UNDER VALIDATION This section will show the impact of the three changes (1), (2), (3) specified in Section 4, as implemented in the operational MPEF, compared to the operational product and the scientific prototype. The comparisons were made for four repeat cycles on 15 May 2013 at 0000, 0300, 1200 and. 1500 UTC. Figure 4 through Figure 7 that follow show snapshots of the full disk results for the product under validation, the operational product, and the scientific prototype. Figure 4: TOZ results for the product under validation (upper left), the operational product (upper right), and for the scientific prototype (bottom), for 15 May 2013, 0000 UTC. Page 8 of 15
Figure 5: TOZ results for the product under validation (upper left), the operational product (upper right), and for the scientific prototype (bottom), for 15 May 2013, 0300 UTC. Page 9 of 15
Figure 6 : TOZ results for the product under validation (upper left), the operational product (upper right), and for the scientific prototype (bottom), for 15 May 2013, 1200 UTC. Page 10 of 15
Figure 7: TOZ results for the product under validation (upper left), the operational product (upper right), and for the scientific prototype (bottom), for 15 May 2013, 1500 UTC. Page 11 of 15
An obvious feature is that the product under validation has less retrievals than the operational product, i.e. it shows more white areas in the plots. This result was already observed with the offline tests described in the previous section, and is also observed in the scientific prototype. The main reasons are a stricter rejection of high cloud situations, by change (1) and more failed retrievals due to the 3-channel RMS check change (3). A failure here is likely due to a mismatch between the forecasted upper level humidity field and the measurements in channel WV6.2, implying that also the ozone retrieval is not very reliable as the humidity fields also have an effect on channel IR9.7. A future enhancement of the product could handle this by also allowing the humidity profile to be changed. This will be tested in the scientific prototype first. Figure 8 shows a close-up view over Europe for one of the repeat cycles (0000 UTC), allowing a better assessment of the high spatial variability problem. Comparing the top and the centre panels in Figure 8, it is obvious that the high variability is much reduced (e.g. regarding Eastern Europe) in the product under validation, which is well in line with the prototype results. In this particular case, a small mismatch is observed north of Scotland, where the product under validation (top panel) still shows a higher variability level than the prototype, likely due to the use of different forecast fields in the two versions (which can be more quantitatively assessed once the Reprocessing MPEF is set up for such tests). Table 1 provides some statistical analysis regarding the spatial variability. Time Product under Validation Operational Product Scientific Prototype 0000 UTC 3312 6549 196 0300 UTC 3318 6086 34 1200 UTC 3663 6638 73 1500 UTC 3755 7394 134 Table 1: Number of adjacent processing elements showing a TOZ difference of 50 DU and higher, for the product under validation, the operational product, and the scientific prototype for the four analysed repeat cycles on 15 May 2013. As mentioned above, the discrepancy to the scientific prototype is still large, likely due to the use of different background forecast data. Nevertheless, compared to the operational version, the variability is reduced by about 50% with the latest product changes. Table 2 and Table 3 provide some more detailed analysis, showing that the high spatial variability almost exclusively occurs for the cloudy retrievals. For completeness, Table 4 lists the overall differences between the product under validation resp. the operational product and the scientific prototype (in absolute Dobson Units). All differences are well within the uncertainty of the method, and are even slightly reduced for the product under validation. Page 12 of 15
Figure 8: TOZ results for the product under validation (top), the operational product (centre), and for the scientific prototype (bottom), for 15 May 2013, 0000 UTC. Page 13 of 15
Time Product under Validation Retrieval Type Cloudy Clear Mixed 0000 UTC 3312 3018 69 225 0300 UTC 3118 2803 120 195 1200 UTC 3663 3473 5 185 1500 UTC 3755 3456 19 280 Table 2: Number of adjacent processing elements showing a TOZ difference of 50 DU and higher, for the product under validation, and the distribution according the retrieval type (both adjacent elements cloudy, both clear, or one cloudy one clear) for the four analysed repeat cycles on 15 May 2013. Time Operational Product Retrieval Type Cloudy Clear Mixed 0000 UTC 6549 5848 274 427 0300 UTC 6086 5316 300 470 1200 UTC 6638 6096 75 467 1500 UTC 7394 6526 86 782 Table 3: Number of adjacent processing elements showing a TOZ difference of 50 DU and higher, for the operational product, and the distribution according the retrieval type (both adjacent elements cloudy, both clear, or one cloudy one clear) for the four analysed repeat cycles on 15 May 2013. Time Product under Validation minus prototype Operational product minus prototype 0000 UTC 0300 UTC 1200 UTC 1500 UTC 6.0 5.7 4.5 5.3 8.1 7.7 7.0 7.5 Table 4: Overall difference (in DU) between the product under validation resp. the operational product and the scientific prototype. Page 14 of 15
6 CONCLUSIONS The encountered problem on the operational TOZ product the observed high spatial variability of the TOZ values was found to be a feature of the operational code and not of the scientific prototype. It was also found that this feature occurred mostly for retrievals using the cloudy retrieval method. A comparison to the prototype revealed that the only major difference between the two versions was how the retrieval was terminated and classified as successful: The prototype uses the root mean square (RMS) differences between the simulated and observed brightness temperatures of 7 IR channels, while the operational code only checks the RMS of two channels: IR9.7 and IR10.8. A small RMS here results in a TOZ retrieval which is declared successful while the specific processing area could have other differences to the forecast fields which are disregarded in this case and may lead to wrong TOZ values (e.g. if the mid and upper level moisture fields are different than forecasted, that would have an effect on the IR9.7 signal which is not properly accounted for). In the prototype system, the retrieval would ultimately fail in this case. For runtime reasons, a full replication of the prototype RMS over seven channels is not possible in the operational code (as many processing areas would then undergo several iterations in the retrieval before they are declared as failed), and a good compromise was found to use three channels in the RMS computation: WV6.2, IR9.7 and IR10.8. The number of neighbouring processing elements with a TOZ difference exceeding 50 DU was roughly halved, but still considerably higher than in the prototype version. A reason for this remaining difference is likely the different use of forecast data in the prototype (using 91 atmospheric levels) and the operational code (using only 30 levels) an issue which will be addressed in one of the future updates of the operational code, as this affects the entire processing chain, starting with the forecast acceptance, the radiative transfer calculations and the use of the forecasts in a number of other meteorological products. A further planned product update is the use of other channels also within the cloudy retrieval method, specifically channels WV6.2 and WV7.3, which would allow a change of the moisture fields above the clouds and may ultimately lead to more successful retrievals without re-introducing the high spatial variability. Page 15 of 15