Using Weaher Ensemble Predicions in Elecriciy Demand Forecasing James W. Taylor Saïd Business School Universiy of Oxford 59 George Sree Oxford OX1 2BE, UK Tel: +44 (0)1865 288678 Fax: +44 (0)1865 288651 Email: james.aylor@sbs.ox.ac.uk and Robero Buizza European Cenre for Medium-range Weaher Forecass Shinfield Park Reading RG2 9AX, UK Tel: +44 (0)118 9499653 Fax: +44 (0)118 9869450 Email: r.buizza@ecmwf.in Inernaional Journal of Forecasing, 2003, Vol. 19, pp. 57-70.
Using Weaher Ensemble Predicions in Elecriciy Demand Forecasing
Using Weaher Ensemble Predicions in Elecriciy Demand Forecasing Summary Weaher forecass are an imporan inpu o many elecriciy demand forecasing models. This sudy invesigaes he use of weaher ensemble predicions in elecriciy demand forecasing for lead imes from one o 10 days ahead. A weaher ensemble predicion consiss of 51 scenarios for a weaher variable. We use hese scenarios o produce 51 scenarios for he weaher-relaed componen of elecriciy demand. The resuls show ha he average of he demand scenarios is a more accurae demand forecas han ha produced using radiional weaher forecass. We use he disribuion of he demand scenarios o esimae he demand forecas uncerainy. This compares favourably wih esimaes produced using univariae volailiy forecasing mehods. Keywords: Energy forecasing; Weaher ensemble predicions; Forecasing accuracy; Predicion inervals
1. Inroducion Weaher variables are used o model elecriciy demand. Demand forecass are produced by subsiuing a forecas for each weaher variable in he model. Tradiionally, single poin weaher forecass have been used. In his paper, we consider a new ype of forecas, called weaher ensemble predicions. An ensemble predicion consiss of 51 differen members. Each member is a differen scenario for he fuure value of he weaher variable. The ensemble, herefore, conveys he degree of uncerainy in he weaher variable. We use he 51 weaher ensemble members o produce 51 scenarios for elecriciy demand a lead imes from one o 10 days ahead. Meeorologiss someimes find ha he mean of he 51 ensemble members for a weaher variable is a more accurae forecas of he variable han a radiional single poin forecas (Leih, 1974; Moleni e al., 1996). In view of his, we consider he use of he average of he 51 demand scenarios as a poin forecas of demand. We use he disribuion of he elecriciy demand scenarios as an inpu o esimaing he uncerainy in demand forecass. I is imporan o assess he uncerainy in order o manage he sysem load efficienly (Adams e al., 1991). A measure of risk is also beneficial for hose rading elecriciy. In his paper, our analysis is based on daily elecriciy demand daa. We use he demand forecasing mehodology of he Naional Grid (NG) as a basis for our analysis. NG is responsible for he ransmission of elecriciy in England and Wales. The company s demand forecass have always been a crucial inpu o operaional planning, where he generaion oupu is scheduled o mee cusomer demand. Since he re-srucuring of he indusry in 1990, he NG demand forecass have also been an imporan influence on he price and dynamics of he elecriciy marke. Accurae demand forecass are required by uiliies who need o predic heir cusomers demand, and by hose wishing o rade elecriciy as a commodiy on financial markes. Weaher ensemble predicions are described in Secion 2. Secion 3 presens he mehod and variables currenly used by NG. Secion 4 considers how weaher ensemble 1
predicions can be used o improve he accuracy of demand forecass. Secions 5 and 6 invesigae he poenial for using weaher ensemble predicions o assess he uncerainy in demand forecass. The esimaion of demand forecas error variance is considered in Secion 5, and demand predicion inervals are he focus of Secion 6. The final secion provides a summary and conclusion. 2. Ensemble Weaher Predicions The weaher is a chaoic sysem. Small errors in he iniial condiions of a forecas grow rapidly, and affec predicabiliy. Furhermore, predicabiliy is limied by model errors due o he approximae simulaion of amospheric processes in a sae-of-he-ar numerical model. These wo sources of uncerainy limi he accuracy of single poin forecass, which are generaed by running one single model-inegraion wih bes esimaes for he iniial condiions (see Figure 1). ***** Figure 1 ***** Generally speaking, a complee descripion of he weaher predicion problem can be saed in erms of he ime evoluion of an appropriae probabiliy densiy funcion (pdf) in he amosphere s phase space. An esimae of he pdf provides forecasers wih an objecive way o undersand he uncerainy in single poin predicions. Ensemble predicion aims o derive a more sophisicaed esimae of he pdf han ha provided by a univariae exrapolaion of he disribuion of hisorical errors. Ensemble predicion sysems generae muliple realisaions of numerical predicions by using a range of differen iniial condiions in he numerical model of he amosphere. The frequency disribuion of he differen realisaions, which are known as ensemble members, provides an esimae of he pdf. The iniial condiions are no sampled as in a saisical simulaion because his is no pracical for he complex, high-dimensional weaher predicion model. Insead, hey are designed o 2
sample direcions of maximum possible growh (Moleni e al, 1996; Palmer e al. 1993; Buizza e al., 1998). Since December 1992, boh he US Naional Cener for Environmenal Predicions (NCEP, previously NMC) and he European Cenre for Medium-range Weaher Forecass (ECMWF) have inegraed heir deerminisic predicion wih medium-range ensemble predicion (Toh and Kalnay, 1993; Tracon and Kalnay, 1993; Palmer e al., 1993). The number of ensemble members is limied by he necessiy o produce weaher forecass in a reasonable amoun of ime wih he available compuer power. In December 1996, afer differen sysem configuraions had been considered, a 51-member sysem was insalled a ECMWF (Buizza e al., 1998). The 51 members consis of one forecas sared from he unperurbed, bes esimae of he amosphere iniial sae plus 50 ohers generaed by varying he iniial condiions. Sochasic physics was inroduced ino he sysem in Ocober 1998 (Buizza e al., 1999). This aims o simulae model uncerainies due o random model error in he parameerised physical processes. A he ime of his sudy, ensemble forecass were produced every day for lead imes from 12 hours ahead o 10 days ahead. The ensemble forecass were archived every 12 hours, and are hus available for midday and midnigh. The archived weaher variables include boh upper level weaher variables (ypically wind, emperaure, humidiy and verical velociy a differen heighs) and surface variables (e.g. emperaure, wind, precipiaion, cloud cover). ECMWF disseminaes ensemble forecass o he Naional Meeorological Ceners of is European member saes, as par of an operaional suie of weaher producs. In our work we used ensemble predicions generaed by ECMWF from 1 November 1998 unil 30 April 2000. We limied our sudy o his period because he inroducion of sochasic physics in Ocober 1998 subsanially improved he characerisics of he ensemble predicions of surface variables. We use ensemble predicions for he following hree variables: emperaure, wind speed and cloud cover. 3
3. Elecriciy Demand Forecasing 3.1. Modelling Elecriciy Demand in England and Wales There is no consensus as o he bes approach o elecriciy demand forecasing. The range of differen approaches includes ime-varying splines (Harvey and Koopman, 1993), muliple regression models (Ramanahan e al., 1997), judgemenal forecass and arificial neural neworks (see Hipper e al., 2001). In his paper, we implemen he forecasing process used a NG. We presen he modelling approach and he weaher variables in some deail, as hey form he basis of our analysis in he remainder of he paper. The approach aken by NG is firs o forecas he demand a he 10 or 11 daily urning poins and a several sraegically posiioned fixed poins, such as midday and midnigh. These urning poins and fixed poins are collecively known as cardinal poins. Forecass for periods beween cardinal poins are hen obained by a procedure known as profiling which involves fiing a curve o he cardinal poins (see Taylor and Majihia, 2000). Harvey and Koopman (1993) describe a similar approach, which involves fiing a ime-varying spline beween a number of cardinal poins. A NG, he cardinal poin forecass are produced by separae regression models, which are funcions of seasonal and weaher variables (Baker, 1985). This mehod has similariies wih he mehod of Ramanahan e al. (1997), who produced hourly forecass by using a separae regression model for each hour of he day. 3.2. Modelling Midday Elecriciy Demand In his paper, we focus on predicing demand (load) in England and Wales a midday. This is convenien because ensemble predicions are currenly available for midday, alhough in he fuure hey cerainly could be produced for any required period of he day. Midday is always chosen as a fixed cardinal poin by NG, and so here is no need o perform he NG profiling heurisic. Midday is a paricularly imporan period in many summer monhs because i is ofen when peak demand occurs. We follow he procedure of NG and 4
Ramanahan e al. (1997) and produce a model for midday based on demand for previous middays and weaher variables. Figure 2 shows a plo of elecriciy demand in England and Wales a midday for each day in 1999. One clear feaure of demand is he srong seasonaliy hroughou he year, which resuls in a difference of abou 5000 MW beween ypical winer and ypical summer demand. Anoher noiceable seasonal feaure occurs wihin each week where here is a consisen difference of abou 6000 MW beween weekday and weekend demand. There is unusual demand on a number of special days, including public holidays, such as 1 January. In pracice, NG forecass demand on hese days using judgemenal mehods. As in many oher sudies of elecriciy demand, we eleced o smooh ou hese special days, as heir inclusion is likely o be unhelpful in our analysis of he relaionship beween demand and weaher. An alernaive o his would o be rea he special days as missing observaions. ***** Figure 2 ***** Shor o medium-erm forecasing models mus accommodae he variaion in demand due o he seasonal paerns shown in Figure 2 and due o weaher. A NG, demand is modelled using hree weaher variables: effecive emperaure, cooling power of he wind and effecive illuminaion. These variables are consruced by ransforming sandard weaher variables in such a way as o enable efficien modelling of weaher-induced demand variaion (Baker, 1985). Effecive emperaure (TE ) for day is an exponenially smoohed form of TO, which is he mean of he spo emperaure recorded for each of he four previous hours. TE (1) 1 1 = TO + 2 TE 2 1 The influence of lagged emperaure aims o reflec he delay in response of heaing appliances wihin buildings o changes in exernal emperaure. Cooling power of he wind (CP ) is a nonlinear funcion of wind speed, W, and average emperaure, TO. I aims o describe he draughinduced load variaion. 5
CP 1 2 W (18.3 TO ) = 0 if if TO < 18.3 TO 18.3 o o C C (2) Effecive illuminaion is a complex funcion of visibiliy, number and ype of cloud and amoun and ype of precipiaion. Since NG needs o model he demand for he whole of England and Wales, weighed averages are used of weaher readings a Birmingham, Brisol, Leeds, Mancheser and London. The weighed averages aim o reflec populaion concenraions in a simple way by using he same weighing for all he locaions excep London, which is given a double weighing. Since he aim of his paper is o invesigae he poenial for he use of ensemble predicions in elecriciy demand forecasing, we use only weaher variables for which ensemble predicions were available. Ensemble predicions are available for emperaure, wind speed and cloud cover (CC ) a midday and midnigh. In view of his, we replaced effecive illuminaion by cloud cover, and we used spo emperaure, insead of average emperaure, TO, o consruc effecive emperaure and cooling power of he wind from NG s formulae in expressions (1) and (2). A common approach o elecriciy demand forecasing is o predic separaely he weaher-relaed demand and he non-weaher-relaed demand, he base load. For simpliciy, in his paper, we follow he wo-sage approach of NG. The firs sage aims o idenify he weaher-relaed componen by esimaing a regression model similar o he following: demand = a 0 + a 1 TE + a 2 TE 2 + a 3 CP + a 4 CC + a 5 + a 6 2 + a 7 3 + a 8 4 + a 9 FRI + a 10 SAT + a 11 SUN + a 12 W1 + a 13 W2 + a 14 W3 + ε (3) where FRI, SAT and SUN are 0/1 dummy variables for Fridays, Saurdays and Sundays; W1, W2 and W3 are 0/1 dummy variables represening he hree summer weeks when a large amoun of indusry closes; ε is an error erm; and he a i are consan parameers. The ime polynomial is used o model in a deerminisic way he yearly seasonal effec ha was 6
eviden in Figure 2. We followed NG in using daa from he previous wo years o esimae he model, and so a quaric ime polynomial was appropriae. The second sage of he NG approach involves summing forecass of he weaherrelaed demand and he base load. A forecas for he weaher-relaed demand is produced by subsiuing radiional weaher poin forecass in he following expression aken from he esimaed regression model in (3): 2 weaher relaed _ demand = aˆ 1 TE + aˆ 2 TE + aˆ 3 CP + aˆ 4 CC (4) Forecass for he base load are produced judgemenally by NG. Since we do no have he experise o produce judgemenal forecass, we used he simple alernaive of a univariae ARMA-regression model. Using he usual diagnosic ess, we derived he following model: base _ demand ε = φ ε 1 + φ ε = b 1 2 2 1 1 0 + b1 FRI + b2 SAT + b3 SUN + b4 W 2 + b5 W 3 + θ u + u + ε (5) where u is a whie noise error erm and he b i, φ i and θ i are consan parameers. 4. Using Weaher Ensembles for Demand Poin Forecasing 4.1. Creaing 51 Scenarios for Weaher-Relaed Elecriciy Demand A sandard resul in saisics is ha he expeced value of a non-linear funcion of random variables is no necessarily he same as he non-linear funcion of he expeced values of he random variables. This is an imporan issue when forecasing from non-linear models (Lin and Granger, 1994). Le us reconsider he forecas of he weaher-relaed demand, which was given in expression (4). In view of he definiion of cooling power of he wind, given in expression (2), and he presence of he TE 2 erm in (4), i is clear ha he weaher-relaed demand is a non-linear funcion of he fundamenal weaher variables: emperaure, wind speed and cloud cover. The usual approach o forecasing he weaher-relaed demand in all elecriciy demand models simply involves subsiuing a single poin forecas for each weaher variable. Bearing in mind he resul regarding he expecaion of a non-linear funcion of random 7
variables, i would be preferable o firs consruc he probabiliy densiy funcion for he weaher-relaed elecriciy demand, and hen o calculae he expecaion. Alhough esimaion of he densiy funcion of weaher-relaed demand is no sraighforward, weaher ensemble predicions do enable a reasonably sophisicaed esimae o be consruced. Since we have 51 ensemble members for emperaure, wind speed and cloud cover, we can subsiue hese 51 weaher scenarios ino expression (4) o deliver 51 scenarios for weaher-relaed demand. The hisogram of hese 51 demand scenarios is an esimae of he densiy funcion. The esimae of he mean is calculaed as he mean of he 51 demand scenarios. In Secions 5 and 6, we assess he accuracy of he variance and shape of his esimaed disribuion. This is less of an issue in his secion, as our aim is o esimae he mean of he densiy funcion. Meeorologiss ofen find ha he mean of he 51 ensemble members for a weaher variable is a more accurae forecas of he variable han he single poin forecas. The collecion of 51 ensemble members mus, herefore, conain informaion no capured by he single poin forecas. This provides furher moivaion for forecasing weaher-relaed demand using he mean of he 51 demand scenarios. 4.2. Comparison of Forecasing Mehods We used 22 monhs of daily daa from 1 January 1997 o 31 Ocober 1998 o esimae model parameers, and 18 monhs of daily daa from 1 November 1998 o 30 April 2000 o evaluae he differen forecasing mehods. Afer eliminaing special days, his 18 monh period gave 500 days for evaluaion. We produced forecass for each day in our evaluaion period for lead imes of one o 10 days ahead. We compared four differen ses of forecass using he mean absolue percenage error (MAPE) summary measure, which is used exensively in he elecriciy demand forecasing lieraure. 8
Mehod 1: radiional weaher poin forecass - Afer esimaing he models in expression (3) and (5) for he wo-sage approach described in Secion 3, we produced forecass by he usual procedure of subsiuing radiional single poin weaher forecass in expression (4) for he weaher-relaed demand. Mehod 2: mean of scenarios - Using he same models from he wo-sage approach, we produced forecass using he mean of he 51 scenarios for weaher-relaed demand. This approach is based on he weaher ensemble predicions since he 51 scenarios are consruced from he 51 ensemble members. Mehod 3: acual weaher used as forecass - In order o esablish he limi on demand forecas accuracy ha could be achieved wih improvemens in weaher forecas informaion, we produced demand forecass using he wo-sage approach wih acual observed weaher subsiued for he weaher variables in he weaher-relaed demand expression in (4). Clearly his level of forecas accuracy is unaainable, as perfec weaher forecass are no achievable. Mehod 4: pure ARMA - In order o invesigae he benefi of using weaher-based mehods a differen lead imes, we produced a furher se of benchmark forecass from he following wellspecified model ha does no include any of he weaher variables: demand ε = ϕ ε 1 = c 0 + c1 FRI + c2 SAT + c3 SUN + c4 W 2 + c5 W 3 + ϕ ε + ψ u 1 2 2 1 1 + u + ε where he c i, ϕ i and ψ 1 are consan parameers. Figure 3 shows MAPE resuls for he four differen mehods. I is widely acceped ha, for one day-ahead forecasing, a weaher-based mehod is preferable o a mehod ha does no use weaher informaion. Indeed, all of he mehods enered in a recen one day-ahead 9
forecasing compeiion used emperaure as an explanaory variable (Ramanahan e al., 1997). We are no aware of a consensus of opinion regarding lead imes up o 10 days ahead. Our resuls show ha he weaher-based mehods comforably dominae he mehod using no weaher variables a all 10 lead imes. ***** Figure 3 ***** I is ineresing o noe from he MAPE resuls ha, for one day-ahead demand forecasing, here is very lile difference beween he performance of he mehods using weaher forecass and ha of he benchmark mehod using acual observed weaher. The difference increases seadily wih he lead ime due o he worsening accuracy of he weaher forecass. The resuls show ha using weaher ensemble predicions, insead of he radiional approach of using single weaher poin forecass, led o improvemens in accuracy for almos all he 10 lead imes. These improvemens increased wih he lead ime, and brough he MAPE resuls noiceably closer o hose of he mehod using acual observed weaher, which is an unaainable benchmark. For lead imes of 4 days ahead or more, he accuracy of he new ensemble-based approach is as good as ha of he radiional approach a he previous lead ime. This could be described as a gain in accuracy of a day over he radiional approach. 5. Using Weaher Ensembles o Esimae he Demand Forecas Error Variance We now urn our aenion o esimaing he uncerainy in demand forecass. In Secion 6, we consider he esimaion of predicion inervals. In his secion, we aim o esimae he variance of he probabiliy disribuion of demand forecas error. This is no a rivial ask, as he forecas error variance is likely o vary over ime due o weaher and seasonal effecs. Since he mehod using weaher ensemble predicions as inpu produced he mos accurae pos-sample forecass in he previous secion, we focus on esimaion of he variance of he forecas errors from his mehod. The approach ha we ake is o model he variance in a series of hisorical pos-sample forecas errors. Our modelling of he uncerainy 10
focuses on he error series and does no affec he poin forecass. A similar approach is aken by Engle e al. (1993) who model he magniude of forecas errors. We consider lead imes of one o 10 days ahead, unlike Engle e al. who focus only on one day-ahead forecasing. We use he firs 9 monhs (1 November 1998 o 31 July 1999) of pos-sample errors from our earlier analysis of poin forecasing o esimae model parameers, and he remaining 9 monhs (1 Augus o 30 April 2000) of pos-sample errors o evaluae he resuling variance forecass. 5.1. Mehods for Esimaing Demand Forecas Error Variance In his secion, we presen seven mehods for esimaing he error variance. Mehods 2 and 3 are used for forecasing volailiy in financial daa, and Mehods 4 o 7 are designed o incorporae weaher ensemble informaion in he esimae of he error variance. Mehod 1: naïve - For each lead ime, k, we calculaed he variance of he k day-ahead errors in he esimaion period of 9 monhs. Mehod 2: ewma - An exponenially weighed moving average of recen squared errors allows he esimae o adap over ime. We implemened his mehod and opimised he smoohing parameer separaely for each lead ime. Mehod 3: garch - An alernaive o he ad hoc mehods described so far is he GARCH saisical modelling approach (see Engle, 1982; Bollerslev, 1986). In addiion o lagged squared error erms and lagged condiional variance erms, exogenous explanaory variables can be included in GARCH models. We experimened wih simple univariae explanaory variables. However, he only one ha was significan for any of he models was he dummy variable for Saurdays, SAT. The one day-ahead GARCH(1,1) variance forecas is given by 11
ˆ σ ˆ + 2 2 2 = α 0 + α1 e 1 + β1σ 1 γ 1 SAT An ineresing issue arises in fiing saisical models o k sep-ahead errors. The series of k sep-ahead errors from an opimal predicor is likely o possess auocorrelaion, which can be described by a moving average process of order k-1 (see Granger and Newbold, 1986, p. 130). This was eviden in our forecas errors. We conrolled for his by fiing he GARCH model o he residuals of an MA(k-1) model fied o he k day-ahead errors. In using he GARCH model for predicion, he MA(k-1) componens play no par as he predicion is for k days ahead. Mehod 4: scenario variance - The level of uncerainy in he demand forecass depends o an exen on he uncerainy in he weaher forecass. This moivaes he use of a measure of weaher forecas uncerainy in he modelling of demand forecas uncerainy. The variance of he 51 demand scenarios, discussed in Secion 4.1, conveys he uncerainy in he weaher componen of demand. For each day in our pos-sample period, we calculaed he variance, σ 2 ENS,, of he 51 scenarios for each of he 10 lead imes, and used his as an esimae of he demand forecas error variance. Mehod 5: recalibraed scenario variance - The variance of he 51 scenarios is likely o underesimae he demand forecas error variance because i does no accommodae he uncerainy due o he model error and he parameer esimaion error associaed wih expressions (3) and (4). In view of his, for each lead ime, we performed a linear bias correcion 2 by regressing he squared forecas error on σ ENS,. The recalibraed esimaor is of he form: ˆ σ + 2 ˆ ˆ = a b σ 2 ENS, 12
Mehod 6: mixed garch - Since here is likely o be useful informaion in he weaher ensemble predicions ha is no capured by he univariae ime series exrapolaion of he GARCH model, 2 we esimaed GARCH models wih σ ENS, as an addiional poenial explanaory variable. This new variable was significan only in he models for lead imes 2, 5, 8, 9 and 10. For he oher lead imes, he mixed garch model was idenical o he garch model of Mehod 3. Mehod 7: combinaion - Combining is an alernaive o he mixed garch model for synhesising informaion from he ensemble predicions and he pas forecas error variance. We calculaed he simple average of he recalibraed scenario variance esimaor and he garch esimaor. We chose he garch esimaor simply because i is he mos sophisicaed of he univariae mehods. 5.2. Comparison of Variance Esimaors Table 1 repors he coefficien of deerminaion, R 2, from he regression of he squared pos-sample forecas errors on he variance forecass for he 9-monh pos-sample evaluaion period. This measure is widely used in volailiy forecas evaluaion. The regression correcs for any bias and he R 2 measures he degree o which he esimaor varies wih he changing variance of he errors. I is, herefore, a measure of he informaional conen of he esimaor. Typically, he R 2 values are low, wih values less han 10% being he norm (Andersen and Bollerslev, 1998). The enries in bold in each column of Table 1 indicae he bes performing mehod for each lead ime. The R 2 for he naïve esimaor was zero for all lead imes since i does no vary during he 9-monh evaluaion period. The resuls for he scenario variance mehod and he recalibraed scenario variance mehod are idenical because he R 2 measures covariaion afer performing a bias correcion on he esimaor. Alhough he garch mehod performed well a he early lead imes and a he 10-day horizon, overall, he bes resuls were recorded wih he ensemble-based mehods and he combinaion. ***** Table 1 ***** 13
Table 2 shows he roo mean squared error (RMSE) pos-sample evaluaion resuls. 1 2 2 RMSE = ( e i ˆ σ i ) n where e i is he load forecas error and n is he number of observaions in he 9 monh possample evaluaion period. Unlike he R 2, he RMSE does no correc for bias, and so he resuls of Table 2 are a more sraighforward reflecion of forecasing performance. In Secion 5.1, we suggesed ha bias would be a major issue for he scenario variance mehod. Comparing Mehods 4 and 5 in Table 2, we can see ha he RMSE resuls for his esimaor noably improve wih he recalibraion. The bold enries in he able indicae ha he combinaion is generally he bes mehod up o 5 days ahead and ha he recalibraed scenario variance mehod is he bes beyond 5 days ahead. ***** Table 2 ***** In summary, Tables 1 and 2 show ha here is benefi in using weaher ensemble informaion in consrucing demand forecas error variance esimaes. In view of is srong performance using boh evaluaion measures and is relaive simpliciy, we would recommend he recalibraed scenario variance mehod. i 2 6. Using Weaher Ensembles o Esimae Demand Predicion Inervals Predicion inervals are widely used o convey he uncerainy in a forecas. In his secion, we consider a number of ways of esimaing predicion inervals for elecriciy demand forecass. Alhough 95% and 90% inervals are mos common in he research lieraure, Granger (1996) suggess ha 50% inervals are also widely used by praciioners. He poins ou ha 50% inervals are more robus o disribuional assumpions and are less affeced by ouliers. He criicises 95% limis for ofen being embarrassingly wide, and hus no very useful. In order o consider boh he ails and he body of he predicive disribuion, we focus on esimaion of 50% and 90% inervals. More specifically, we evaluae differen approaches o esimaing he bounds 14
of hese inervals: he 5%, 25%, 75% and 95% quaniles. The θ% quanile of he probabiliy disribuion of a variable y is he value, Q(θ), for which P(y<Q(θ))=θ. As in Secion 5, we use 9 monhs of pos-sample errors from our earlier analysis of demand poin forecasing o esimae mehod parameers, and he remaining 9 monhs of pos-sample errors o evaluae he esimaors. 6.1. Mehods for Esimaing Demand Forecas Error Quaniles The variance esimaors, invesigaed in Secion 5, can be used as he basis of quanile esimaors. We used eiher a Gaussian disribuion or he empirical disribuion of he corresponding sandardised forecas errors, e / σˆ (see Granger e al., 1989): Mehod 1: naïve variance esimaor wih Gaussian disribuion. Mehod 2: ewma variance esimaor wih Gaussian disribuion. Mehod 3: garch variance esimaor wih Gaussian disribuion. Mehod 4: recalibraed scenario variance variance esimaor wih Gaussian disribuion. Mehod 5: mixed garch variance esimaor wih Gaussian disribuion. Mehod 6: naïve variance esimaor wih empirical disribuion. Mehod 7: ewma variance esimaor wih empirical disribuion. Mehod 8: garch variance esimaor wih empirical disribuion. Mehod 9: recalibraed scenario variance variance esimaor wih empirical disribuion. Mehod 10: mixed garch variance esimaor wih empirical disribuion. Mehod 11: scenario quanile - We used he quaniles, Q ( ), of he disribuion of ENS, θ scenarios as esimaes of he quaniles of he forecas error disribuion. Mehod 12: recalibraed scenario quanile - The widh of he predicive disribuion is likely o be greaer han he widh of he disribuion of scenarios. We used quanile regression o 15
recalibrae he scenario quanile esimaor wih he forecas errors as dependen variable and Q ( ) as regressor (see Granger, 1989). The form of he resulan recalibraed esimaor is: ENS, θ Q ˆ ( θ ) = aˆ + bˆ Q ENS, ( θ ) Mehod 13: combinaion - We calculaed he average of he garch based esimaor wih empirical disribuion and he recalibraed scenario quanile. 6.2. Comparison of Quanile Esimaors Table 3 compares esimaion of he 5% quaniles a he 10 differen lead imes for he pos-sample period of 9 monhs. The able shows he percenage of pos-sample forecas errors falling below he quanile esimaors. For an unbiased esimaor of he 5% quanile, his will be 5% (see Taylor, 1999). The enries in bold in each column of Table 3 indicae he bes performing mehod for each lead ime. The aserisks indicae he enries ha are significanly differen from he ideal value a he 5% significance level. The accepance region for he hypohesis es is consruced using a Gaussian disribuion and he sandard error formula for a proporion. The resuls show ha he ewma variance esimaor wih Gaussian disribuion and he combinaion mehod perform well, and ha he scenario quanile mehod is vasly improved wih he quanile regression recalibraion. ***** Table 3 ***** To summarise he overall relaive performance of he mehods a he differen lead imes, we calculaed chi-squared goodness of fi saisics. For each mehod, a each lead ime, we calculaed he saisic for he oal number of pos-sample forecas errors falling wihin he following five caegories: below he 5% quanile esimaor, beween he 5% and 25% esimaors, beween he 25% and 75%, beween he 75% and 95%, and above he 95%. Table 4 shows he resuling chi-squared saisics. The aserisks indicae significance a he 5% level. Unforunaely, we canno sum he chi-squared saisics across lead imes o give a single 16
summary measure for each of he esimaors because he chi-squared saisics for he differen lead imes are no independen. The resuls indicae ha an empirical disribuion is preferable o a Gaussian assumpion. The combinaion and he recalibraed scenario quanile perform consisenly well across he 10 lead imes. ***** Table 4 ***** The percenage of errors falling below a quanile esimaor evaluaes only bias; we should also consider he variabiliy of he esimaion error. For example, he firs column of resuls in Table 3 shows ha 4.4% of he one day-ahead errors fell below he naïve variance esimaor wih Gaussian disribuion. Since he ideal is 5%, he esimaor is a lile low on average; i possesses a degree of bias. Alhough he level of bias in his esimaor is he second bes of he 13 esimaors, oher esimaors should vary in accordance wih he varying variance of he disribuion beer han he naïve esimaor, which by consrucion does no vary a all. I would be useful if we could evaluae his variabiliy characerisic. The R 2 measure used for evaluaing he variance esimaors in Secion 5 correcs for bias, so ha he R 2 hen reflecs variaion abou he bias. Similarly, a quanile regression R 2 measure can be used o evaluae quanile esimaor predicion variance (Taylor, 1999). The package STATA (Saa, 1993) provides a pseudo-r 2, analogous o he R 2 in LS regression. Table 5 shows his pseudo-r 2 for esimaion of he 5% quaniles; high values of he pseudo-r 2 are preferable. ***** Table 5 ***** The firs column of resuls in Table 5 shows ha he pseudo-r 2 for he naïve esimaor is zero, bu for many of he oher esimaors i is considerably more. These resuls reflec he fac ha hese esimaors vary more wih he unobservable quanile. The pseudo-r 2 reflecs covariaion beween esimaor and unobservable quanile. Consequenly, he quanile esimaors based on he same variance esimaor, which differ only by a linear ransformaion, have he same pseudo-r 2. Table 5 suggess ha he esimaors based on he recalibraed scenario variance esimaor and hose based on he mixed garch variance esimaor end o have he 17
highes pseudo-r 2. Many of he ohers perform well a he early lead imes bu disappoiningly for he longer horizons. Based on he chi-squared resuls in Table 4 and he pseudo-r 2 resuls for he four differen quaniles, we would enaively conclude ha, overall, he mehods ha perform he bes for quanile esimaion are he combinaion and he recalibraed scenario variance wih empirical disribuion. 7. Summary and Conclusions We have invesigaed he scope for using weaher ensemble predicions in elecriciy demand forecasing for lead imes from one o 10 days ahead. We used he 51 ensemble members for each weaher variable o produce 51 scenarios for he weaher-relaed componen of elecriciy demand. For almos all 10 lead imes, he mean of he demand scenarios was a more accurae demand forecas han ha produced by he radiional procedure of subsiuing a single poin forecas for each weaher variable in he demand model. Since demand is a nonlinear funcion of weaher variables, his radiional procedure amouns o approximaing he expecaion of a non-linear funcion of random variables by he same non-linear funcion of he expeced values of he random variables. The mean of he 51 scenarios is appealing because i is equivalen o aking he expecaion of an esimae of he demand probabiliy densiy funcion. The disribuion of he 51 demand scenarios provides informaion regarding he uncerainy in he demand forecas. However, since he disribuion does no accommodae demand model uncerainies, i will end o underesimae he demand forecas uncerainy. In view of his, we recalibraed measures of variance and quaniles aken from he scenario disribuion. The resuling variance esimaor compared favourably wih esimaors produced using univariae volailiy forecasing mehods. Using he same variance esimaor as a basis for esimaing predicion inervals also compared well wih univariae mehods. We, herefore, conclude ha here is srong poenial for he use of weaher ensemble predicions in improving he accuracy and uncerainy assessmen of elecriciy demand forecass. 18
Acknowledgemens We are very graeful o Shani Majihia and Chris Rogers of he Naional Grid Group for supplying daa and informaion regarding he company s approach o demand forecasing, and o Tony Hollingsworh of he European Cenre for Medium-range Weaher Forecass for helpful suggesions. We are also graeful o hree anonymous referees for heir helpful commens. References Adams, G., Allen, P.G. and Morzuch, B.J. (1991). Probabiliy disribuions of shor-erm elecriciy peak load forecass, Inernaional Journal of Forecasing 7, 283-297. Andersen, T.G. and Bollerslev, T. (1998). Answering he Criics: Yes, ARCH models do provide good volailiy forecass, Inernaional Economic Review 39, 885-906. Baker A.B. (1985). Load forecasing for scheduling generaion on a large inerconneced sysem. In Comparaive Models for Elecrical Load Forecasing, Bunn DW, Farmer ED (eds), Wiley: Chicheser, pp 57-67. Bollerslev, T. (1986). Generalized auoregressive condiional heeroskedasiciy, Journal of Economerics 31, 307-327. Buizza, R., Miller, M. & Palmer T.N. (1999). Sochasic simulaion of model uncerainies, Quarerly Journal of he Royal Meeorological Sociey 125, 2887-2908. Buizza, R., Peroliagis, T., Palmer, T.N., Barkmeijer, J., Hamrud, M., Hollingsworh, A., Simmons, A. & Wedi, N. (1998). Impac of model resoluion and ensemble size on he performance of an ensemble predicion sysem, Quarerly Journal of he Royal Meeorological Sociey 124, 1935-1960. Engle, R.F. (1982). Auoregressive condiional heeroscedasiciy wih esimaes of he variance of Unied Kingdom inflaion, Economerica 50, 987-1008. Engle, R., Granger, C.W.J., Ramanahan, R. & Vahid-Araghi, F. (1993). Probabilisic mehods in forecasing hourly loads and peaks, Elecric Power Research Insiue, EPRI TE- 101902, Palo Alo, California. Granger, C.W.J. (1989). Combining forecass - weny years laer, Journal of Forecasing 8, 167-173. Granger, C.W.J. (1996). Can we improve he perceived qualiy of economic forecass?, Journal of Applied Economerics 11, 455-473. Granger, C.W.J. & Newbold, P. (1986). Forecasing Economic Time Series, London: Academic Press (2nd ediion). 19
Granger, C.W.J., Whie, H. & Kamsra, M. (1989). Inerval forecasing: an analysis based upon ARCH-quanile esimaors, Journal of Economerics 40, 87-96. Harvey, A.C. & Koopman, S.J. (1993). Forecasing hourly elecriciy demand using imevarying splines, Journal of he American Saisical Associaion 88, 1228-1236. Hipper, H.S., Pedreira, C.E. & Souza, R.C. (2001). Neural Neworks for Shor-Term Load Forecasing: A Review and Evaluaion, IEEE Transacions on Power Sysems 16, 44-55. Leih, C.E. (1974). Theoreical skill of Mone Carlo forecass, Monhly Weaher Review 102, 409-418. Lin, J. & Granger, C.W.J. (1994). Forecasing from non-linear models in pracice, Journal of Forecasing 13, 1-9. Moleni, F., Buizza, R., Palmer, T.N. & Peroliagis, T. (1996). The new ECMWF ensemble predicion sysem: mehodology and validaion, Quarerly Journal of he Royal Meeorological Sociey 122, 73-119. Palmer, T.N., Moleni, F., Mureau, R., Buizza, R., Chapele, P. & Tribbia, J. (1993). Ensemble predicion. Proceedings of he ECMWF Seminar on Validaion of models over Europe: vol. I, ECMWF, Shinfield Park, Reading, RG2 9AX, UK. Ramanahan, R., Engle, R., Granger, C.W.J., Vahid-Araghi, F. & Brace C. (1997). Shor-run forecass of elecriciy loads and peaks, Inernaional Journal of Forecasing 13, 161-174. Saa. (1993). STATA 3.1, Saa Corporaion, College Saion: TX. Taylor, J.W. (1999). Evaluaing volailiy and inerval forecass, Journal of Forecasing 18, 111-128. Taylor, J.W. & Majihia, S. (2000). Using combined forecass wih changing weighs for elecriciy demand profiling, Journal of he Operaional Research Sociey 51, 72-82. Toh, Z. & Kalnay, E. (1993). Ensemble forecasing a NMC: he generaion of perurbaions, Bullein of he American Meeorological Sociey 74, 2317-2330. Tracon, M.S. & Kalnay, E. (1993). Operaional ensemble predicion a he Naional Meeorological Cener: pracical aspecs, Weaher and Forecasing 8, 379-398. 20
pdf pdf 0 forecas lead ime, Figure 1: Schemaic of ensemble predicion. The iniial probabiliy densiy funcion, pdf 0, represens he iniial uncerainies. From he bes esimae of he iniial sae, a single poin forecas (bold solid curve) is produced. This poin forecas fails o predic correcly he fuure sae (dash curve). An ensemble of perurbed forecass (hin solid curves) saring from perurbed iniial condiions, designed o sample he iniial uncerainies, can be used o esimae he probabiliy of fuure saes. In his example, he esimaed probabiliy densiy funcion, pdf is bimodal. The figure shows ha wo of he perurbed forecass almos correcly prediced he fuure sae. Therefore, a ime 0, he ensemble sysem would have given a non-zero probabiliy of he fuure sae. 21
MW 50000 45000 40000 35000 30000 25000 Jan-99 Feb-99 Mar-99 Apr-99 May-99 Jun-99 Jul-99 Aug-99 Sep-99 Oc-99 Nov-99 Dec-99 Figure 2: Demand for Elecriciy a Midday in England and Wales in 1999. 22
MAPE 3.0 2.5 2.0 1.5 1.0 0.5 0.0 1 2 3 4 5 6 7 8 9 10 Lead ime (days) 1. radiional weaher poin forecass 2. mean of scenarios (based on weaher ensembles) 3. acual weaher used as forecass (unaainable benchmark) 4. pure ARMA (using no weaher variables) Figure 3: MAPE for elecriciy demand poin forecass for pos-sample period, 1 November 1998 o 30 April 2000. 23
Lead ime (days) 1 2 3 4 5 6 7 8 9 10 Univariae 1. naïve 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2. ewma 2.1 1.4 2.1 0.0 0.9 0.0 0.0 5.3 0.0 1.8 3. garch 2.6 2.1 3.2 0.1 0.8 0.5 3.7 2.4 0.6 7.2 Ensemble based 4. scenario variance 1.2 3.5 0.6 1.9 3.0 5.4 6.0 8.5 6.7 4.1 5. recalibraed scenario variance 1.2 3.5 0.6 1.9 3.0 5.4 6.0 8.5 6.7 4.1 6. mixed garch 2.6 7.1 3.2 0.1 4.1 0.5 3.7 9.4 8.2 6.1 Combinaion 7. average of 3. and 5. 3.0 4.2 0.9 1.9 2.6 4.7 6.0 8.8 6.6 6.8 Table 1: R 2 percenage measure for forecas error variance esimaion mehods for pos-sample period, 1 Augus 1999 o 30 April 2000. 24
Lead ime (days) 1 2 3 4 5 6 7 8 9 10 Univariae 1. naïve 1032 958 1090 1198 1340 1315 1401 1624 1800 1880 2. ewma 1025 950 1079 1197 1350 1311 1412 1614 1801 1888 3. garch 1031 954 1088 1217 1358 1334 1397 1690 1891 1857 Ensemble based 4. scenario variance 1192 1140 1276 1376 1507 1477 1572 1820 2003 2084 5. recalibraed scenario variance 1028 947 1103 1216 1323 1281 1358 1556 1774 1874 6. mixed garch 1031 918 1088 1217 1359 1334 1397 1682 1902 1994 Combinaion 7. average of 3. and 5. 1017 933 1078 1185 1322 1286 1359 1588 1780 1838 Table 2: RMSE/1000 for forecas error variance esimaion mehods for pos-sample period, 1 Augus 1999 o 30 April 2000. 25
Lead ime (days) 1 2 3 4 5 6 7 8 9 10 Variance esimaors wih Gaussian 1. naïve 4.4 4.4 3.6 4.0 4.0 5.2 4.4 3.2 3.6 3.6 2. ewma 6.0 6.0 5.2 5.2 6.0 6.0 6.0 6.0 3.6 2.8 3. garch 4.0 5.6 7.1 8.3* 7.1 7.1 8.3* 9.9* 9.5* 6.0 4. recalibraed scenario variance 4.0 3.6 3.2 4.0 4.4 4.4 3.2 3.6 4.4 4.4 5. mixed garch 4.0 4.0 7.1 8.3* 8.7* 7.1 8.3* 12.3* 15.1* 14.3* Variance esimaors wih empirical 6. naïve 4.4 4.4 3.2 4.4 4.0 3.6 4.4 3.2 3.2 2.0 7. ewma 6.3 7.5 4.4 4.8 6.7 3.6 4.0 3.2 3.2 2.0* 8. garch 6.0 5.6 3.2 4.4 6.0 4.8 6.7 3.2 3.2 4.8 9. recalibraed scenario variance 3.6 4.4 2.4 4.0 4.4 2.0* 3.2 2.4 4.8 4.4 10. mixed garch 6.0 2.8 3.2 4.4 4.0 4.8 6.7 4.0 6.3 6.0 Demand scenario quanile 11. scenario quanile 43.7* 46.0* 27.0* 25.0* 20.6* 20.6* 16.7* 20.6* 17.9* 17.9* 12. recalibraed scenario quanile 4.4 3.6 3.2 4.0 4.4 2.8 3.6 2.8 3.6 4.4 Combinaion 13. average of 8. and 12. 4.8 4.4 2.8 4.0 4.8 3.2 4.8 3.2 2.4 4.4 Table 3: Percenage of errors falling below esimaes of 5% forecas error quanile for pos-sample period, 1 Augus 1999 o 30 April 2000. * indicaes significan a 5% level. 26
Lead ime (days) 1 2 3 4 5 6 7 8 9 10 Variance esimaors wih Gaussian 1. naïve 11.7 * 9.3 9.0 5.1 4.2 3.3 0.6 4.0 4.5 5.0 2. ewma 8.9 15.9* 20.3* 6.6 10.6* 2.2 3.7 8.2 4.7 4.8 3. garch 12.8* 17.7* 20.3* 14.6* 12.2* 10.7* 14.9* 30.6* 41.8* 25.7* 4. recalibraed scenario variance 12.2* 10.8* 14.7* 4.8 3.0 2.8 3.1 4.0 13.9* 9.7* 5. mixed garch 12.8* 19.1* 20.3* 14.6* 20.2* 10.7* 14.9* 91.6* 153.0* 172.1* Variance esimaors wih empirical 6. naïve 4.9 9.4 4.6 1.1 1.1 1.8 3.7 5.9 11.0* 18.1* 7. ewma 9.3 10.3* 9.7* 2.0 3.1 1.8 9.4 6.4 11.0* 19.0* 8. garch 3.8 1.5 3.5 1.1 1.0 4.4 9.3 7.6 6.3 28.6* 9. recalibraed scenario variance 6.5 7.0 5.0 1.7 0.3 5.0 3.9 8.2 19.0* 23.1* 10. mixed garch 3.8 10.6* 3.5 1.1 0.8 4.4 9.3 11.6* 31.1* 41.5* Demand scenario quanile 11. scenario quanile 1122.2* 1165.8* 1334.4* 639.2* 1536.0* 676.6* 1240.3* 831.7* 1167.4* 1465.5* 12. recalibraed scenario quanile 4.5 7.5 3.8 2.8 1.7 5.3 2.5 3.3 2.8 34.7* Combining 13. average of 8. and 12. 2.9 4.9 3.8 1.7 0.3 5.6 2.7 2.8 6.0 19.3* Table 4: Chi-squared saisics summarising overall esimaor bias for 5%, 25%, 75% and 95% forecas error quaniles for he pos-sample period, 1 Augus 1999 o 30 April 2000. * indicaes significan a 5% level. 27
Lead ime (days) 1 2 3 4 5 6 7 8 9 10 Variance esimaors 1.& 6. naïve 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.& 7. ewma 10.9 9.3 9.9 6.7 5.3 0.0 0.2 2.6 0.0 1.3 3.& 8. garch 11.0 6.3 3.2 4.1 4.3 0.4 5.4 0.3 0.2 5.2 4.& 9. recalibraed scenario variance 5.1 5.6 3.2 2.7 4.8 10.3 7.5 8.6 8.1 3.6 5.& 10. mixed garch 11.1 10.1 3.2 4.1 7.5 0.4 8.3 8.6 6.9 4.6 Demand scenario quanile 11. scenario quanile 0.2 3.3 0.5 0.2 0.8 0.5 1.1 0.7 0.6 0.7 12. recalibraed scenario quanile 0.2 3.3 0.5 0.2 0.8 0.5 1.1 0.7 0.6 0.7 Combinaion 13. average of 8. and 12. 11.2 7.6 1.0 0.6 2.0 1.1 3.9 0.5 0.4 3.2 Table 5: Pseudo R 2 percenage measure for esimaors of 5% forecas error quanile for pos-sample period, 1 Augus 1999 o 30 April 2000. 28