Virtual Met Mast verification report:

Transcription

1 Virtual Met Mast verification report: June

2 Authors: Alasdair Skea Karen Walter Dr Clive Wilson Leo Hume-Wright 2

3 Table of contents Executive summary Introduction Verification process Define Data Sources Define Verification Statistics Grouping sites to calculate bias statistics Verification results summary Other Verification Diagnostics Comparison of VMM with NOABL and NCIC Diurnal variation Height variation Virtual Met Mast Turbulence Intensity model Verification of Virtual Met Mast Turbulence Intensity model Performance of Virtual Met Mast Turbulence Intensity model Annex A VMM s uncertainty: contributing factors Appendix B Complexity index

4 Executive summary Virtual Met Mast is based on the UK Met Office s operational Numerical Weather Prediction (NWP) Model and has been designed to produce long term, site and height specific wind climatologies and associated time series datasets for the onshore and offshore wind energy industry. Virtual Met Mast is typically used to support site search and selection activities prior to the installation of an on-site monitoring mast and often supports the positioning of these masts on larger sites. Virtual Met Mast is also of benefit during site design development and construction, where the output can be compared with traditional methods for assessing long term wind resources, thereby acting as an effective due diligence process and solution. For lower value feed in tariff (FiT) and other smaller projects, where the costs and timeframes associated with the installation and collection of data using real masts are prohibitive, it can be used as the primary source for delivering accurate, long term wind assessments. It will also be of real value in countries, or specific areas of the world, where the existence of local wind data is either poor or totally unavailable. Similarly, with offshore developments being undertaken further away from the coastline, the ability of Virtual Met Mast to generate a site specific virtual reference dataset is critical for establishing the long term wind climatology for these zones. The inclusion of additional site climatological parameters such as modelled average turbulence intensity at 15m/s average wind speed, average wind shear exponent, and 50-year return 3-second gust wind speed, deliver an early indication of turbine class suitability according to standards IEC A full description of Virtual Met Mast solution can found on the Met Office web site at For the industry to use Virtual Met Mast as a practical decision support solution it is necessary to define its performance over a range of sites, heights and complexity challenges presented to the model. This report details the results of comparing and verifying Virtual Met Mast science at 89 sites, using up to a total of ~160 site-years of monitored data at a range of heights. Using this extensive source data we are able to define the bias (model deviation from measurements) in the mean wind speeds and also, critically, the confidence levels relating to the estimated long-term mean speeds. The degree of the downscaling challenge for Virtual Met Mast in going from the 4 km grid down to the particular site in question is closely related to the complexity of the local topography but not entirely so: larger topographic features some distance away can have an impact, and individual nearby obstacles (such as large buildings, cliffs, trees or operating wind turbines) are not modelled. 4

5 Confidence in the estimated long-term mean wind speed at a site is expressed as the 90 th percentile (W90) of the distribution of long term mean bias values from sites of similar complexity. Each on-shore location is assigned a complexity index derived from Virtual Met Mast diagnostics. These indices have been grouped into three categories (Low, Medium and High Complexity) and the overall bias statistics (mean and W90) calculated are shown in Table 1. All offshore sites, except those that are near shore and influenced by coastal effects, present the same level of complexity. Note that these figures are not comparable with statistics derived from the variability in 10-minute or hourly mean winds, which are a separate and additional consideration. Model Complexity Mean Bias (VMM - Obs) (m/s) 90 th Percentile (W90) (m/s) ALL SITES Offshore Near-Shore Low Complexity Medium Complexity High Complexity Table 1: Bias statistics for the complexity categories defined. These confidence statistics have been calculated from the Virtual Met Mast Verification Database of 189 anemometers at 89 sites. The criteria for the categorisation of sites model complexities and bias statistics calculated for each are subject to refinement as further verification data become available. This most recent update to Virtual Met Mast Verification report has not updated the results from the retuning and verification of the modelling of Average Turbulence Intensity at 15m/s, TI av (15) provided in the previous Verification report released in January This is due to the small additional number of usable anemometers available since the previous Verification. The inclusion of 89 anemometers at various heights at various sites where standard deviation of wind speed data were available has delivered the removal of overall mean bias. The standard error of site specific TI av (15) biases is Based on the quality of results from this comprehensive verification process it may therefore be concluded that Virtual Met Mast may be used with confidence to support larger and smaller projects, both onshore and offshore. For further information please contact [email protected] 5

6 1. Introduction This report describes the Virtual Met Mast verification process and presents the results of comparing 162 site-years of data at 89 sites, covering a range of model complexity types and hub heights. The process used provides objective and quantified statistics which have been selected as appropriate descriptors within the Met Office and externally. The report defines the data sources utilised, the analytical process followed, and a number of key derived statistics. A description and the results of the most recent (January 2013) verification of Virtual Met Mast Turbulence Intensity model are included in Section Verification process The verification process consists of three major steps as follows:- 1. Define data sources. 2. Define verification statistics to be produced by the analysis. 3. Group verification sites and calculate bias statistics. 2.1 Define Data Sources Virtual Met Mast produces an hourly wind time series covering the period January 2001 to date. It is the accuracy of the mean wind speed over this period that is the primary focus of this report. Two sources of monitored data are used for comparison with Virtual Met Mast calculations: 1. Met Mast data collected on prospective sites at heights between 20m and 100m (often there are anemometers at several heights on a single mast). The data are usually provided as ten minute means, in which case the mean speed and direction in the ten minute period leading up to the hour is compared with the wind calculations made on the hour by Virtual Met Mast. Careful consideration has been given regarding the inclusion and weighting of data from observing periods of widely varying length. Various tests are applied to assess the quality of the observed data, with dubious data removed. 2. Nacelle data collected from turbines in operation. Data are recorded at one height, but several sources are often available in close proximity from turbines on the same site. The data are sampled in the same way as Met Mast data and the same weighting and filtering criteria are applied to data deemed to be of sufficient quality. However, data tend to be of poorer quality than mast data, and when the number of sample sites has grown sufficiently these cases will be filtered from the verification. 6

7 DATA SOURCE Height (m) Advantages Disadvantages Length of data Data freq. means Epoch Prospective sites monitoring masts Quality measurements, Multiple sources at one location Shorter monitoring periods Varying widely 10 min 2001 onwards Turbine nacelle anemometers Close locations Poor Data Quality Less than 10 years 10 min 2005 onwards Table 2: Summary of features for different data sources. 2.2 Define Verification Statistics Where observations at more than one height on the same mast are available, the wind speed biases are averaged for all heights above 20m. When there are several nacelle observations on the same site, a subjective assessment on their inclusion is made based on spacing, topography and land use. The following statistics have been selected to verify the performance of Virtual Met Mast when compared with monitored data, and are based on long term mean wind speeds: Biases - defined as the difference in wind speed between Virtual Met Mast and monitored data, over the monitoring period. A bias value is produced for each hour of concomitant data and the time average bias then calculated for the location, across all measuring heights over 20m. One overall bias figure is produced for each location. Standard deviation of biases gives the uncertainty of the bias values around the mean. In this report the standard deviation of the biases in long term mean wind speed is calculated as: σ = Root Mean Squared error (RMS) The RMS gives a measure of the combined systematic error (bias) and random error (standard deviation). RMS = W90 defined as the 90 th percentile of the normally distributed long term mean bias values, calculated as 1.28 times the standard deviation. Subtracting this figure from the Virtual Met Mast long-term mean wind speed gives the speed that 90% of the sites of the same population will exceed. See Annex A for more details about the various factors contributing to VMM uncertainty. 7

8 When calculating the bias statistics, several weighting functions have been considered to account for the uncertainty in using various lengths of monitoring periods. Note that the ultimate objective is to assess the overall error in Virtual Met Mast hourly time series from January 2001 to date and not on each individual hourly observation. The various weighting alternatives considered were: 1. Only use records over six months in length and apply the same weight to them. 2. Linear weighting as a function of record length. 3. Linear weighting as a function of record length up to one year, and same weight when longer. 4. A weighting function derived from the deviation analysis of different length records using the verification datasets available. At this time, Option 4 has been chosen (see Figure 1). This will be kept under review. Figure 1: The weights applied to the bias values depend on the observational time length. 8

9 2.3 Grouping sites to calculate bias statistics Bias statistics can be presented for all verification sites taken as a single population. This grouping produces the most reliable statistics but could mask significant differences in the way Virtual Met Mast is performing in different geographical scenarios. However there may be various embedded bias populations in the full set, with the most logical split being to separate offshore sites from those onshore. Offshore, where the land use is constant and with no topography influences, Virtual Met Mast should perform well. On-shore sites are assigned a complexity index which depends on the downscaling challenge for Virtual Met Mast in going from the 4 km grid down to the particular site in question. It is closely related to the complexity of the local topography but not entirely so; larger topographic features some distance away can have an impact. Sites are placed into Low, Medium and High complexity groups according to their Complexity Index (A/S) and bias statistics calculated for each group. A map of the regions of differing complexity categorisation is presented in Appendix B. For each verification site the following information is gathered: The bias The Complexity Index The record length of site monitoring for weighting the bias values when calculating the bias statistics of each group. Linear weighted mean and standard deviation are calculated for each group of sites. 9

10 3. Verification results summary Table 3 summarises the overall results. The standard error in the last column indicates the uncertainty in the mean bias as a result of working with small sample sizes. Model Complexity Number of 20m+ sites N Bias statistics (from 2001), for heights over 20m (m/s) Mean W90 St. Error = σ N ALL sites Offshore Near-shore Low Complexity Medium Complexity High Complexity Table 3: Derivation of bias mean and W90 for heights above 20m. It should be noted that, as more samples become available or alternative model complexity related sub-divisions are identified, some changes to these figures are to be expected. The complexity categorisation will be kept under review. Note: The W90 statistics published here are the 90 th percentiles of each complexity category s standard deviation of biases in long term mean wind speeds. These bias statistics are not comparable with statistics derived from the variability in 10-minute or hourly mean winds, which are a separate and additional consideration. Statistics based on a number of sites long term mean biases or speed-ups do not belong to the same population as those based on spot, 10-minute, hourly or monthly biases or speed-ups. 10

11 4. Other Verification Diagnostics 4.1 Comparison of VMM with NOABL NOABL (National Objective Analysis of Boundary Layer) is a commonly used site screening tool, comprising 1km square mean wind speeds at 10, 25 and 45m of height. The dataset is the result of an air flow model that estimates the effect of topography on wind speed 1. VMM performance has been assessed against NOABL by comparing their long-term wind speed estimates against observations at 68 UK sites, where NOABL mean wind speeds were extrapolated to observational heights using wind profile laws. For several of the stations observations at more than one height were available. Data have been grouped by complexity, and by height into 4 15m wide bins. As shown in Figures 2 and 3, and Tables 4 and 5, across all complexity and height categories, VMM mean biases are lower than for NOABL. Uncertainties on the mean bias are similar. Figure 2: VMM performance comparison against NOABL by complexity Error bars represent the standard deviation of the biases

12 Figure 3: VMM performance comparison against NOABL by height. Error bars represent the standard deviation of the biases. Model Complexity Number of Mean Bias Std Dev Bias RMS 20m + (m/s) (m/s) (m/s) heights VMM NOABL VMM NOABL VMM NOABL ALL HC MC LC NS Table 4: Bias statistics by complexity for NOABL and VMM screening tools. Model Complexity Number of Mean Bias Std Dev Bias RMS 20m + (m/s) (m/s) (m/s) heights VMM NOABL VMM NOABL VMM NOABL ALL m m m m Table 5: Bias statistics by height for NOABL and VMM screening tools. 12

13 4.2 Diurnal variation There is a tendency for the wind speed to increase during the afternoon and fall away again during the evening for onshore sites. This arises from the increased turbulent energy in the atmosphere in the afternoon due to solar heating. Often, on individual days, this effect is masked by synoptic weather features but comes through as a feature in hourly wind climatology. Figure 3 gives a typical example, for a low complexity site, of how Virtual Met Mast is able to capture the daily cycle. The correlation between the calculated and observed wind speed values is good with a standard deviation of the hourly biases less than 0.25 m/s. Figure 3: Example of VMM modelling the diurnal cycle. 13

14 4.3 Height variation Site data are available at a variety of heights above ground/sea level, enabling an assessment of whether VMM has a propensity to perform differently at different heights. Reasonably, the expectation might be that it performs better as height increases, as the impacts of local surface features are reduced. Figure 4 shows the mean bias and the standard deviation around the mean for the sites-heights available. Bias values have been binned in 10m steps (left plot) and also all together (right plot). The variability for each height bin is shown with error bars reflecting the standard deviation on the bias. The coloured histograms within each height bin show the distribution of sites by the model complexity for each height range considered to be compared with the overall distribution (right plot). It can be seen that above 20m there is no significant bias or trend. With only the 50m bin approximating to the overall proportions of site types, these figures must be treated with caution as indicative of there not being a height bias. More verification data at locations with multiple observing heights are required. Figure 4: Monthly biases and their standard deviations about the mean and categorised by mast height. 14

15 5 Virtual Met Mast Turbulence Intensity model Full Virtual Met Mast reports deliver an estimate for omni-directional average Turbulence Intensity in the 1 m/s wide, 15 m/s wind speed bin, for indicative assessments of wind turbine class suitability in accordance with standards IEC These estimates are based on a parameterised model, with input variables taken from site-specific location and height, orographic and surface roughness parameters, and Virtual Met Mast wind speeds. While the parameterisation delivers an estimate of average Turbulence Intensity, TI av (15), the nature of hourly VMM spot wind speeds does not allow for the modelling of its variability; represented by the standard deviation of Turbulence Intensity readings in the 15 m/s wind speed bin, TI sd (15). Therefore calculations of Characteristic Turbulence Intensity, CTI(15) [IEC Edition 2] and Representative Turbulence Intensity, RTI(15) [IEC Edition 3] are not available. However, using the approximation that TI sd (15) = 20% TI av (15), the Edition 2 turbulence classification categories are presented in terms of TI av (15) only: Turbulence Class TI av (15) limit S > 0.15 A 0.15 B < Verification of Virtual Met Mast Turbulence Intensity model The parameterisation of Turbulence Intensity within Virtual Met Mast has been adjusted to give the best possible estimate of on-site average turbulence. These estimates have been verified against observations by 89 individually calibrated cup anemometers where standard deviation of wind speed data were available, from 20m to 81m height above ground, at 35 sites of varying orographic ruggedness and land use roughness. Six of the 89 anemometers were NRG Max#40 instruments and the remainder were either Vector A100LK/LM or WindSensor P2546A instruments. The averaging period was 10 minutes in all cases and, where reported, the logger sampling rate was 1 Hz, except one site where it sampled at 0.5 Hz, and one where it sampled at 3 Hz. These datasets have been included in the verification without adjustment. 15

16 Figure 5: Number of anemometers having a given number of 10 minute Turbulence Intensity samples at 15 m/s In all cases where an anemometer had recorded any samples in the 15 m/s average wind speed bin, the anemometer was included in the verification. There was a small but significant number of anemometers (particularly at low heights above ground, at low wind speed sites, from short monitoring period datasets) where there were few records from which to generate statistically reliable values for observed TI av (15). While some smoothing according to results in adjacent wind speed bins could have been performed, these datasets have been included in the verification without adjustment. It should be noted also that Virtual Met Mast model does not attempt to model the turbulence generating effects of individual site-specific obstacles in close proximity to the measurement positions. 5.2 Performance of Virtual Met Mast Turbulence Intensity model Figures 6 and 7 present the results of the verification of Virtual Met Mast Turbulence Intensity model. The mean TI av bias from the 89 tests is The standard deviation of TI av biases is 0.024, giving a 90 th percentile confidence,t90, of Of the 20 cases where Observed data indicated suitability of a Class A wind turbine, the VMM estimated correctly in 18 cases and was very close with the other two. There was more variability in the performance of the model where Observations indicated suitability of a Class S (only) or a Class B turbine. The model will be refined in publicised VMM version updates as more observational data become available. 16

17 Figure 6: VMM vs Observed TI av(15) comparisons at various heights ( 20m), at 35 sites, with IEC Turbulence class bandings Figure 7: TI av(15) Bias (VMM Obs) distribution from 89 tests 17

18 Annex A VMM s uncertainty: contributing factors The uncertainty on the VMM s long-term wind speed has various contributors which are described here. The standard statistical approach for combining independent errors is to add the variances (σ 2 ) of the contributing components that are subject to error or variability. The components impacting on Virtual Virtual Met Mast are described here. σ 2 VMM = σ 2 4km + σ 2 Clim + σ 2 Ann Where: σ 2 4km is the variance of the bias over the period 2001 to date. This term reflects the uncertainty stemming from the downscaling process needed to interpolate data from the 4 km grid down to a particular site. σ 2 Clim is the variance between climatologies back to 1989 and even longer climatologies, some extending back over 50 years. Of course this component does not apply where only the climatology of the last 23 years (1989 onwards) is considered to be relevant. It is not currently included in the VMM uncertainty calculation. σ 2 Ann is an additional variance that needs to be included in order to present the result as an annual mean wind speed, which is as favoured by the wind engineering industry. This is normally represented by a 6% error but it is not currently included. These variances are transformed into the W90 statistic. W90 = 1.28 x σ VMM is the 90 th percentile of the bias distribution, assumed to be normal. It makes possible the calculation of the long term wind speed that there is 90% probability of exceeding at a site. σ VMM is defined separately for different site groupings, where there is evidence that the grouping samples a distinct population of bias errors. Note: The W90 is a measure of the confidence in the long term mean wind speed estimated by the VMM, on the basis of its accuracy in verifications at sites of similar complexity. This should not be confused with the uncertainty term for the inter-annual variability of winds accounted for in the production of energy yield estimates P90 figures; which is an entirely separate and subsequent assessment. 18

19 Appendix B Complexity Index Figure 5: A/S Complexity Index values within the regional model domain The silhouette area per unit horizontal area (A/S) is a parameter used to quantify the complexity of a site. It is a measure of the upslopes encountered divided by the length of the cross section of the orography. 19

20

21 Met Office FitzRoy Road, Exeter Devon EX1 3PB United Kingdom Tel (UK): (Int) : Fax (UK): (Int) : [email protected]