Forecasting Electricity Prices

Transcription

1 Forecasing Elecriciy Prices Derek W. Bunn 1 and Nekaria Karakasani London Business School 2003 v1 Absrac This is a review paper documening he main issues and recen research on modeling and forecasing elecriciy prices. The special marke microsrucure of elecriciy is described, as an explanaion of he exraordinary sochasic properies of elecriciy price ime series. The research lieraure deriving from he applicaion of models adaped from financial asses, for boh spo and forward prices, is reviewed and criicised. Final emphasis is placed upon he virues of compuaionally inensive srucural modeling. Keywords: Elecriciy, Volailiy, Regime-swiching, Srucural Models Inroducion Developing predicive models for elecriciy prices is a relaively new area of applicaion for he forecasing profession. Unil recenly, elecriciy was a monopoly in mos counries, ofen governmen owned, and if no, highly regulaed. As such, elecriciy prices refleced he governmen s social and indusrial policy, and any price forecasing which was underaken was really focussed on hinking abou underlying coss. In his respec, i ended o be over he longer erm, aking a view on fuel prices, echnological innovaion and generaion efficiency. This changed dramaically, however, during he 1990s. Following he examples of srucural reforms and marke liberalizaions in Chile, 1 Corresponding auhor: Professor Derek W. Bunn, London Business School, Sussex Place, Regens Park, London NW1 4SA, UK. dbunn@london.edu

2 Briain, Norway, Argenina, and Ausralasia in he early 1990s, oher European counries such as Spain and Germany followed sui a few years laer, as well as various regions in Norh and Souh America, and his rend has coninued so ha power secor reform has now become a maor issue worldwide. Ownership in he secor has generally become privae raher han public, compeiive markes (pools and power exchanges) have been inroduced for wholesale rading and reail markes gradually liberalised o erode local franchises. Typically he indusry has been spli up ino separae companies for generaion, ransmission, local disribuion and reail supply. Transmission and disribuion are nework services and, as naural monopolies, are regulaed. Generaion is progressively deregulaed as compeiion develops beween a sufficien number of companies o promoe an efficien wholesale marke. Reail suppliers buy from he wholesale marke and sell o cusomers. Indusrial and commercial cusomers have generally been he firs o receive full marke liberalisaion. The residenial secor has been opened in many counries, bu ofen quie slowly, and in some cases no a all. All of his srucural change has been moivaed by a faih in he abiliy of compeiive forces o creae a more efficien and enerprising indusry, han eiher he public secor or regulaed monopolies could deliver. Of course, i is he prospec of business risks ha should drive he efficiency gains, and he maor new elemen of risk is wholesale price uncerainy. In ha he maoriy of he new wholesale spo markes are imperfec and inefficien, and he emergen power exchanges incomplee, in he financial sense, and insufficienly liquid, he need for careful and deailed modeling of prices becomes an essenial aspec of risk managemen in he indusry. If compeiion were so efficien ha prices refleced marginal coss, even in peak periods of demand, hen here would be a complee price convergence of elecriciy wih he underlying fuel coss (eg gas), and spo price modeling would be relaively sraighforward o specify. If he fuures markes were liquid and complee, as well, hen forward prices would have a simple dynamic srucure similar o oher financial asses, and convenional risk managemen echniques could be applied using financial producs from he forward markes. Neiher of hese siuaions

3 prevails in mos elecriciy markes, however, wih he resul ha price models are considerably richer in srucure han is seen in mos oher commodiies, and he forecasing echniques which are being applied are sill in heir early sages of mauriy. This paper summarises he special characerisics of price formaion in elecriciy markes in he nex secion, followed in subsequen secions by a review of he various empirical models which he research lieraure is now manifesing. We are focusing upon he empirical ime-series class of models, as hese are now he basis of shor and medium erm forecass in pracice. This focus is disinc from he large research lieraure on ex ane economic modeling of elecriciy markes, using sylized game heoreic or simulaion mehods, o undersand he price implicaions of various marke designs, or equilibrium condiions, bu where i is generally recognised ha accurae price forecass will no generally be obained (Green and Newbery, 1992; Joskow and Frame, 1998; Green, 1999; Basone, 2000, Skanze e al., 2000; Bunn and Oliveira, 2001; Rouledge e al, 2001; Baldick, 2002, Day e al, 2002, Andersen and Xu, 2002; Bessembinder and Lemmon, 2002). Price Formaion in Elecriciy Markes The crucial feaure of price formaion in wholesale elecriciy spo markes, is he insananeous naure of he produc. The physical laws ha deermine he delivery of power across a ransmission grid require a synchonised energy balance beween he inecion of power a generaing poins and he offake a demand poins (plus some allowance for ransmission losses). Across he grid, producion and consumpion are perfecly synchonised, wihou any capabiliy for sorage. If he wo ge ou of balance, even for a momen, he frequency and volage of he power flucuaes. Furhermore, endusers rea his produc as a service a heir convenience, and here is very lile shor erm elasiciy of demand o price. The ask of he grid operaor, herefore, is o be coninuously monioring he demand process and o call on hose generaors who have he echnical capabiliy and he capaciy o respond quickly o he flucuaions in demand.

4 Mos spo markes for elecriciy are defined on hourly inervals (alhough he Briish marke is half-hourly), and herefore i is clear ha hroughou he day and hroughou he year, a wide variey of plan will be in acion and herefore seing he prices a differen imes. However, when we look a comparison of demand and prices, we see ha he price series exhibi much greaer complexiy han migh iniially be expeced simply from he aciviy of scheduling differen plan o mee flucuaions in demand. In Figure 1, he demand evoluion for hree price-seing periods in he day is shown for almos a year, and his should be compared wih he corresponding marke prices in Figure 2. I is clear ha spo elecriciy prices display a rich srucure much more complicaed han a simple funcional rescaling of demand o reflec he marginal coss of generaion. A number of salien characerisics of he ypical elecriciy spo price series have been noed in he lieraure, and apparenly correspond o he profiles of Figure 2. These include mean-reversion o a long-run level (e.g. Johnson and Barz, 1999), muli-scale seasonaliy (inra-day, weekly, seasonal), calendar effecs, erraic exreme behaviour wih fas-revering spikes as opposed o smooh regime-swiching (e.g. Kaminski, 1997) and non-normaliy manifesed as posiive skewness and lepokurosis. Spo prices also display excessive volailiy, orders of magniude higher han oher commodiies and finacial asses, wih annualised values of 200%, o rmore. This volailiy is ime-varying (e.g. Escribano e al. 2001) wih evidence of heeroscedasiciy boh in uncondiional and condiional variance. The former reflecs he influences of demand, capaciy margin and rading volume on volailiy levels. The laer describes he observed clusering of ranquil or unsable periods (GARCH effecs), specifying volailiy as a funcion of is lagged values and previous disurbances. There is also evidence ha condiional variance reacs asymmerically o posiive and negaive pas shocks and in addiion, displays an inverse o financial asses leverage effec (Kniel and Robers, 2001). There are a number of marke microsrucure elemens, which help o explain hese unusual ime series characerisics. The simples observaion is ha wih a diversiy of

5 plan, of differen echnologies and fuel efficiencies on he sysem, a differen levels of demand, differen plan will be seing he marke-clearing price. Furhermore, we would expec such a diversiy of plan on he sysem for a leas wo reasons. The firs and obvious one, is obsolescence. Wih power plan lasing for some 40 years, new echnologies will come in and be more efficien. So prices will be flucuaing because of he varying efficiencies of he se of plan being used for generaion a any paricular momen in ime. The more suble, and more significan, reason for diversiy is, however, again due o he insananeous naure of he produc. The mos efficien plan, wih he lowes marginal coss (he baseload plan), will operae mos of he ime (eg period 16), bu during some of he peaks in demand (eg period 36 in winer), some of he power plans (he peaking plan) may only be operaing for a few hours. The recovery of capial coss on peaking plan, hrough marke prices, may have o be achieved over a relaively few hours of operaion compared o he 8760 hours in a normal year for which a baseload plan, wihou mainenance breaks, could, in principle, serve. This will favour boh he consrucion of low capial/high operaing cos plan for peaking purposes and he overrecovery of marginal coss in operaion, wih he consequence ha prices are much higher in he peaks. So, whils he fundamenal naure of fuel price convergence has a mean-revering implicaion, he insananeous producion process of following a highly variable demand profile, wih a diversiy of plan coss, creaes he high spo price volailiy. Oher facors also come ino play in he shor erm. There may be echnical failures wih plan, causing even more expensive sandby generaors o come online. The ransmission sysem may become congesed so ha raher expensive, bu locally necessary plan ges called upon. And, of course, here may be unexpeced flucuaions in demand. All of hese evens show up in spo prices, and reflec he fundamenal economic and echnical naure of pricing elecriciy as a real-ime, non-sorable, commodiy.

6 There is a furher imporan characerisic of elecriciy markes, wih maor implicaions for price behaviour, and ha is heir oligopolisic naure. Mos power markes are characerised by a few dominan players, and even in hose less common siuaions where here may appear o be sufficien compeiors o achieve efficien prices, a paricular imes and in special locaions, individual companies may have he abiliy o influence prices. Of he academic research on liberalised elecriciy markes, by far he bulk of work ha has been published has been done on he analysis of, and sraegies for he miigaion of, he abuse of marke power by he generaing companies. As a resul of he presence of his marke power, prices are generally much higher, and even more volaile, han he fundamenals sugges. Parsimonious Sochasic Modelling of Spo Elecriciy Prices One class of mehodological work on empirical price modeling is inspired by a desire o adap some of he familiar models from financial asses, o he characerisics of elecriciy, in order o evaluae elecriciy derivaives and suppor rading. Table 1 summarises he progression of models wihin his heme of research. One of he earlies examples is Kaminski (1997), where he spiky characerisic is addressed hrough a random walk ump-diffusion model, adoped from Meron (1976). However, he model ignores anoher fundamenal feaure of elecriciy prices, he mean-reversion in he baseline regime. This propery is esablished in Johnson and Barz (1998). The comparison of alernaive models (Geomeric Brownian moion and mean-reversion wih/wihou umps) across several deregulaed markes suggess a mean-revering model wih a ump componen as more adequae specificaion. Alhough his ype of financial asse modelling addresses crucial characerisics of price dynamics, namely meanreversion and spikes, i sill assumes deerminisic price volailiy, which clearly conradics empirical evidence.

7 Table 1. Sochasic Models for Spo Elecriciy Prices. Random-walk Mean-reversion dp dp = µ P d + σ dw = a( µ P ) d + σ dw Mean-reversion wih umps Consan parameers dp = a( P ) d + σ dw + kdq ( λ) µ Time-varying parameers (long-run level, ump inensiy and volailiy (GARCH)) P = f + X dx dv = a µ X ) d + σ dw + kdq ( λ ) v ( = k ( θ v ) d + v σdz, σ = v 1/ 2 v 1/ 2 λ = λ1w in er + λ2spring + λ3summer + λ4 Auumn P denoes he spo price, W a Weiner process, f a deerminisic componen, q a Poisson process wih inensiy λ ha describes he ump occurrence and k = k µ, σ ) a random variable ha describes he ump magniude. ( J J This class of modelling was, however, furher exended in Deng (2000) wih addiional non-lineariies in he price dynamics, such as regime-swiching and sochasic volailiy. These aspecs allow richer dynamics o emerge, alhough hey are no capured simulaneously in a single specificaion. In he same aricle, a mulivariae framework is consruced for he oin-dynamics of elecriciy price and a correlaed sae variable. Defining he correlaed process as demand, weaher or fuel price, he mulivariae model allows for cross-commodiy hedging. Analyical resuls are also derived for several elecriciy derivaives under he hree proposed models, which differeniaes he paper from he simulaion approaches previously implemened (eg Kaminski, 1997). Generalising previous modelling, Escribano e al. (2001) suggess a price formulaion ha capures wo addiional price feaures; volailiy clusering in he form of GARCH effecs and seasonaliy (emphasised by Lucia and Schwarz, 2001), boh in he deerminisic componen of prices and he ump inensiy.

8 I should be emphasised ha he coninuous-ime models reduce o familiar formulaions in discree ime. For insance, mean reversion is equivalen o an AR(1) process: α α α ( s ) P a β P + n, where a = µ ( 1 e ), β = e, n = e σdw ( s. = o+ o 1 Similarly, ump diffusion implies: o 1 ) 1 α o + φ P-1 + σε1 wih prob 1- λ P =, where ε 1, α o + φ P-1 + µ J + σ J ε 2 wih prob λ ε 2 ~ N(0,1) Despie heir inuiive inerpreaion and non-linear behaviour, ump-diffusion models presen some limiaions. Firsly, i is assumed ha all shocks affecing he price series die ou a he same rae. In realiy however, wo ypes of shocks exis implying differen reversion raes; large disurbances, which diminish rapidly due o economic forces, and moderae ones, which migh persis for a while. Esimaion bias is ineviable, as Maximum Likelihood mehods end o capure he smalles and mos frequen umps in he daa. As emphasised by Huisman and Mahieu (2001), sochasic ump-models do no disenangle mean-reversion from he reversal of spikes o normal levels. Secondly, he model assumpions for ump inensiy (consan or seasonal) are convenien for simulaing he disribuion of prices over several periods of ime, bu resricive for acual shor-erm predicions for a paricular ime. The Poisson assumpion for umps may be valid empirically, bu only provides he average probabiliy of umps for paricular ransiions. Tigh demand-supply condiions and hence spikes, could emerge irrespecively of season, for insance due o anomalous fuel prices or a echnical failure in supply. For his reason, a causa, srucural represenaion, insead of probabilisic seasonal formulaions could be more appealing for he parameers of he ump disribuion for specific day ahead forecasing. An alernaive modelling framework o ump-diffusion is regime-swiching, and his may be more suiable for acual price forecasing. This can replicae he price disconinuiies, observed in pracice, and could deach he effecs of mean-reversion and spike reversal, aliased in ump-diffusion. Ehier and Moun (1999) assume wo laen marke saes and

9 an AR(1) price process under boh he regular and he abnormal regime and consan ransiion probabiliies, i.e. P µ = φ + S ( P 1 µ S ) ε 1, where 2 ε ~ N(0, σ s ). This reains however he misspecificaion and also imposes saionariy in he irregular spike process. The model suggesed by Huisman and Mahieu (2001), allows an isolaion of he wo effecs assuming hree marke regimes; a regular sae wih mean-revering price, a ump regime ha creaes he spike and finally, a ump reversal regime ha ensures wih cerainy reversion of prices o heir previous normal level. This regime-ransiion srucure is however resricive, as i does no allow for consecuive irregular prices. This consrain is relaxed in de Jong and Huisman (2002). The wo-sae model proposed assumes a sable mean-revering regime and an independen spike regime of log-normal prices. Regime independence allows for muliple consecuive regimes, closed-form soluions and ranslaes o a Kalman Filer algorihm in he implemenaion sage. Formally: ln P ln P M, = ln PM, 1 + a( µ ln PM, 1 S, = µ + ε s S, ) + ε where P M and P S denoe he mean-revering and spike regimes respecively, 2 2 ε M, ~ N(0, σ M ) and ε S, ~ N(0, σ s ). M,, Assuming a regime-swiching price model, day-ahead price forecass can be derived as: i) The expeced price across regimes i.e. a linear combinaion of he prediced prices across regimes wih weighs he prediced regime probabiliies. s ˆ ˆ i P = P Pˆr( S ) 1 = i I, where I denoes he informaion se up o day. i= ii) The predicion of he regime wih he higher prediced probabiliy i.e. ˆ ˆ i P P, where Pˆr( S = i I ) = > + +

10 The former scheme ypically over-esimaes regular prices, whereas he laer generally selecs he normal regime. Boh findings reflec he fac ha he irregular saes in he semi-deregulaed elecriciy markes are recurren bu no persisen. Thus, he prediced probabiliies of he exreme sae very rarely exceed a convenional hreshold level. Whereas his problem migh be averaged ou when simulaing several periods ahead wih he inenion o price a financial insrumen, i is criical for he precision required in a day-ahead predicion. Imposing a lower hreshold for he selecion of he irregular regimes would seem arbirary. An inuiive alernaive would be o specify he regimeprobabiliy as a logisic funcion of a sraegic effec, e.g. Margin. Convergence complicaions could sill emerge due o he non-linear naure of he problem, he nondisoin probabiliy densiies across regimes and possibly he inabiliy of an S-shaped funcion o capure kinked effecs. A poenially more accurae descripion of elecriciy prices is proposed in Bysrom (2001). Afer assuming an AR-GARCH price process wih a seasonal componen in volailiy, he residuals are modelled wih disribuions from exreme value heory. This approach avoids he esimaion complexiies and forecasing limiaions presen in he previous sochasic models due o sudden and fas-revering spikes. Finally, i should be noed ha he saionariy of elecriciy prices is no presen in he same way in differen power markes. If he modelling involves daily average or byperiod spo prices, a mean-revering process wih a seasonal rend, proposed for insance in Lucia and Schwarz (2002), seems appealing for some markes. However, discrepancies do exis. In Akins and Chen (2002), ime and frequency-domain ess reec he null hypoheses of I (1) and I (0) processes for he elecriciy prices in Albera. Long memory feaures are subsequenly idenified in he price evoluion and described wih auoregressive fracional difference (ARFIMA) models. In Sevenson (2002), a uni roo is idenified in he Vicorian marke, possibly because all hourly prices are reained in he same daa se and no divided by load period.

11 A common feaure of he finance-inspired sochasic models reviewed in his secion is heir main inenion o replicae he saisical properies of spo prices wih he ulimae obecive of derivaives evaluaion. In order o reain simpliciy and/or analyical racabiliy, he models include only a few facors and ypically focus on daily average prices, which are sensiive however o ouliers. Albei convenien for opions pricing, he previous properies, i.e. aggregaion of inra-day informaion and modelling of nonrobus measures, are resricive from a forecasing prospecive. Esimaion complexiies and forecasing limiaions are furher enhanced due o he abrup and fas-revering naure of price spikes. Also, a suble issue sysemaically negleced is he assessmen of model adequacy wih marke daa. In his framework, alhough sochasic models have been subsanially adaped o he peculiariies of elecriciy, hey have sill go a long way o go in revealing he main componens of price srucure. Recen papers (e.g. Kniel and Robers, 2001) have emphasised he need o explore his srucure and include i in price specificaions. A realed challenge is o explore how he sensiiviies of prices o influenial facors vary hroughou he day as a response o he fundamenals of inra-day variaion in demand, plan-operaing consrains as well as he generaors sraegic poenials. Forward Spo Price Dynamics in Elecriciy In financial markes, spo and forward prices are usually linked wih an analyical r ( T ) formula derived from non-arbirage condiions: F(, T ) = e S( ), where F(, T ) is he forward price of he underlying asse a ime wih mauriy a T, S() is he spo price of he asse and r he risk-free ineres rae. For sorable commodiies, he above relaionship is adaped o include he convenience yield y: F(, T ) = ( e r y) ( T ) S(), where T refers o delivery dae. In elecriciy markes, however, he convenience link beween forward and spo price clearly does no exis in he sorable sense. The dominan percepion, as expressed in Skanze and Illic (2000), is ha he spo price S() reflecs only he curren sae of demand and supply, and is independen, due o non-sorabiliy, of

12 forward prices F(, T ) wih mauriies in he medium and long fuure. The limied impac of long-erm price movemens on shor-erm dynamics is furher illusraed wih he low correlaion beween shor and long-erm forward prices, idenified in Koekebakker and Ollmar (2001), which would be an unconvenional feaure for sorable commodiies. Spo price volailiy however, srongly encourages conrac coverage and hence raises forward elecriciy prices. Due o his causaliy, forward price is ofen specified as a funcion of expeced spo price, is variance and a random disurbance. Harris (2003) emphasises he link beween spo and forward in erms of risk aversion and risk managemen. The impac of forward prices wih mauriy T on he spo price S(T) is also idiosyncraic. The efficien markes hypohesis (EMH), saing ha forward prices are unbiased esimaors of fuure spo prices, is ofen invalid for sorable commodiies, where a posiive risk premium ends o exis for holding a fuures conrac. In elecriciy however, he sign of he forward premium seems indefinie or irregular. To assess efficiency in US elecriciy fuures markes, Avsar and Goss (2001) adop he forecas error approach. Under he EMH, no sysemaic relaionship should exis beween curren predicion error of he spo price, S ( + k) F(, + k), and prior errors, S( ) F( k, ), for he same and relaed commodiies. A solid economeric analysis indicaes ime-varying risk premia, which reflec volailiy in he form of an M-GARCH erm. The significan negaive impac of rading volume on forecas errors suggess a violaion of he raional expecaions hypohesis, possibly due o agens learning. In a simplisic analysis ha does no accoun for fundamenals, Boerud e al. (2002) repor a negaive risk premium for he Scandinavian fuures markes. This peculiariy is aribued o he difference in flexibiliy beween demand and generaion side, which creaes a higher incenive o he former o hedge heir posiions. Beyond saing he irregulariy of he premium sign, Longsaff and Wang (2002) explore he economic properies of percenage forward premia regressing hem on ex-ane risk facors. These include a measure of price risk, an indicaor of demand uncerainy and he expeced load, perceived as a proxy for he probabiliy of spike occurrence. Significan srucure is

13 deeced in forward premia wih srong inra-day variaion, bu many aspecs of pricing remain vague, as indicaed by he very low R 2. A his poin, i should be emphasised ha even for shor ime differences, elecriciy prices could display persisen deviaions. In Borensein e al. (2001), absence of convergence is idenified beween he California day ahead and hourly real-ime markes. This discrepancy is inerpreed as rading inefficiency. Proposed explanaions include ransacion coss, informaion barriers or dominance of risk aversion. The seleced usificaion is ha a shock occurred in he price formaion process following he revision of marke rules and he sraegic behaviour of one company. The above reasoning sill underesimaes he fac ha he differen iming of he wo markes implies differen informaion uncerainies and plan flexibiliy requiremens, which are convered o coss. This remark is also consisen wih Walls (1999), where mauriy effecs appear much sronger for elecriciy han oher energy fuures. Forward Curve Modelling One class of approaches inspired by he financial asses lieraure abandons he modelling of spo price dynamics and focuses insead on he erm srucure of forward/fuures commodiy prices across differen mauriies. This modelling has been more appealing o maure markes, such as he Nordic and Vicoria, in Ausralia, which inroduced hedging ools early in heir liberalisaion process. Clewlow and Srickland (1999) adap he mulifacor model of Corazar and Schwarz (1994). Formally, he general risk-adused process for fuures prices is: df(, T ) F(, T ) n = σ i (, T ) dwi ( ) i= 1 d 1 2 n 2 ln F(, T ) = σ i (, T ) d + i= 1 n i= 1 σ (, T ) dw ( ) i i

14 where F(,T) denoes he fuures price a ime for delivery a T, w i are independen Brownian moions under he equivalen-maringale measure and σ i are volailiy funcions of spo prices. Each facor i is associaed wih a volailiy funcion ha deermines he magniude and direcion of he shif of each poin in he forward curve due o he arrival of informaion relaed o a paricular source of uncerainy. The forward curve movemens are such ha preclude arbirage opporuniies when rading among fuure conracs. The number n of volailiy componens is suggesed from eigenvalue decomposiion of he covariance marix of forward reurns. Usually, here are volailiy facors ha shif, il and bend he forward curve describing he relaive movemens of shor, medium and long-erm conracs. The mahemaical formulaion of his procedure is summarised below: Consider M forward conracs wih relaive mauriiesτ 1,..., τ,... τ M and values recorded on N ime poins (e.g. days). This equaion can be discreised for small ime changes as follows: 1 ln F(, + τ ) = 2 n n 2 σ i (, + τ ) + i= 1 i= 1 σ (, + τ ) W i Le x k denoe he reurn of he forward conrac on day k, x he sample mean, σ l he covariance beween he reurns of conracs and l and Σ = (σ l ) he sample covariance marix of forward reurns. Then, x k = ln F(, + τ ) = ln( F(, + τ )) ln( F(, + τ k k k k k k )), k =1,2,..,N. σ l N 1 = ( x N k = 1 k x )( x lk x )' The symmeric marix Σ can be facorised as Σ = ΓΛΓ, where Λ = diag{ λ } is a diagonal marix of eigenvalues and Γ an orhogonal marix of corresponding eigenvecors v = v i ( i ). The Principal Componen Analysis of forward reurns, described above, reveals uncorrelaed (orhogonal) dimensions of he variabiliy in forward reurns. l i

15 From he M calculaed eigenvalues, he n larger are seleced, which accoun for a fracion of he oal variaion equal o n i= 1 M i= 1 λi. The forward curve movemen a poin due o facor λ i i is calculaed as: σ (, + τ ) = v λ. i i i General volailiy funcions are proposed from which exising models are derived as special cases and analyical formulae are obained for various hedging producs. A differen approach for modelling he elecriciy forward curve is proposed in Aude e al. (2002). Forward prices are specified as log-normally disribued, srongly correlaed for adacen mauriies and wih lower han spo volailiy, which is assumed deerminisic. Despie his consrain, which conradics he empirical evidence of sochasic volailiy, he model capures aribues of he erm srucure no addressed in previous specificaions. Whereas forward curve modelling allows a parsimonious descripion of he erm srucure, is exension o forecasing remains an open quesion. Complexiies also arise a he esimaion level. Firsly, as in convenience yield modelling, he parameers are esimaed by equaing observed marke prices (or derivaive prices and implied volailiies) o he ones derived by he heoreical model. In he forward curve approach, he model validaion involves forward prices and heir sample covariance marix. This implies exclusive focus on he risk-neural forward price measure and absence of link o acual spo prices. Alhough his is a desirable feaure for commodiies where spo daa are unobservable or scarce, such an obsacle is rarely presen in elecriciy markes, which are usually immaure and in some cases possess limied forward daa. To avoid loss of criical informaion, he volailiy funcion could be linked wih spo volailiy σ S, e.g. df(, T ) F(, T ) n = σ ( ) σ ( T ) dz ( ) S i= 1 i i

16 Secondly, even in deep forward markes, reliance of esimaion only on linear payou asses may lead o poor esimaes of volailiy parameers, as heir impac on fuures prices could be delayed or no srong enough. Exploiing opions prices in addiion o fuures could correc some of hese problems. Thirdly, he forward price dynamics of elecriciy are influenced by several insiuional and marke srucure facors, which are no addressed in he heoreical models. Finally, he discreeness of prices and marke immauriy impose he need for esimaing forward prices for daes wih no ransacions. The subsiuion of missing values is ofen implemened wih a maximum smoohness crierion. An aemp o resolve some of he above issues is presened in wo papers which exploi a richer se of marke daa han simply forward prices. The firs uilises boh spo and forward prices, whereas he second links a boom-up model wih forward daa. More specifically, in Karesen and Husby (2001), observed fuures prices are assumed o equae he heoreical ones, implied from a spo price specificaion, plus some noise resuling from marke imperfecions such as bid-ask spread and risk preferences. Assuming his divergence is consisen wih marke incompleeness and allows model validaion wih richer informaion ses. Furher research could be conduced on he proper specificaion and esimaion of he parameers ha cause he discrepancy. In Fleen and Lemming (2003), a smooh forward curve is obained by consraining he heoreical prices, derived from an opimisaion boom-up model, wih he bid/ask prices observed in he marke. Alhough i may resric exreme deviaions, he heurisic of imposing bounds in prediced prices does no ensure he consisency of srucure beween observed and model prices. Depending on he model srucure, he spo and forward approaches are ofen equivalen for sorable commodiies. A reduced specificaion of he forward curve dynamics implies a process for he spo price and vice versa. In incomplee elecriciy markes however, his link is no as direc. If a sochasic differenial equaion is assumed for he dynamics of F(, T ), hen an explici specificaion is derived wih Io s lemma and he spo price is simply calculaed as S ( ) = F(, ). This ignores however ha he long-run

17 dynamics implici in he forward curve are inadequae o explain he spo variaions. The reverse ask enails deriving he forward price as expeced fuure spo price under he risk-neural measure. Alhough he presence of a risk premium can be addressed, he fundamenal issue of non-unique risk-neural measure in incomplee markes is raher negleced in he lieraure. Defining an opimal risk measure for he unhedgable par of he risk, according o a disance crierion ha also explois acual marke daa, is a challenging research issue. Srucural Modelling Moving away from he financial perspecive of seeking o develop parsimonious sochasic models ha can feed ino derivaive pricing formula, srucural models inend o uncover a richer srucure for elecriciy prices in order o undersand marke performance and enable o mos accurae forecasing, per se. They synhesise wo sources of informaion; hisoric marke prices and fundamenals such as load, weaher and plan daa. For insance, a simple regression model ha relaes spo price o lagged price and demand values is suggesed in Nogales e al. (2002). Their model wasrefined by adusing he number of lags unil he assumpion of uncorrelaed errors was saisfied. I is expressed as: P = c + ) d p ω ( B) D + ω ( B P + where D is he demand a ime, ω d (B) and ω p (B) polynomial funcions of he backshif operaor B : Bx = x 1 ε wih order d and p respecively. The predicive abiliy of his model seems limied, however, in he case of markes wih sraegic marke power and complex rading environmens. Oher srucural formulaions address non-linear aspecs of elecriciy price dynamics, such as muliple price regimes and umps. Vucein e al. (2001) implemen a discovery algorihm of regression regimes, which reveals muliple price-load relaionships in spo

18 rading. The assumpion of a moderae swiching rae beween regimes, necessary for convergence, is unappealing for he sudden spikes in elecriciy, bu could describe smooh regime ransiions in he medium erm. As he regime-swiching process is no modelled, he algorihm is consrained o he analysis of pas daa, raher han forecasing. In Davison e al. (2002), prices are assumed o follow a mixure of wo normal disribuions. A nice propery of he model is ha, in conras wih Markov-regime swiching, he regime probabiliies are relaed empirically o a variable wih economic and sraegic aribues, he raio demand/supply. To derive a more general formulaion ha allows medium-erm forecasing, load is specified as a sinusoidal funcion and capaciy across he year as a wo-level caegorical variable. This approximaion ignores he ineracion beween prices and capaciy availabiliy decisions and could be replaced by a richer sochasic equaion. In Skanze e al. (2000), hourly price is specified as an exponenial funcion of demand and supply. Boh are assumed sochasic wih a deerminisic monhly componen plus a random erm. Due o he pronounced inra-day correlaion, he random erms are derived from a Principal Componen Analysis, similarly o Wolak (1997). To capure sochasic effecs, he loadings are specified as mean-revering o a sochasic mean. A disinc feaure of he model, compared o sandard ump-diffusion, is he uilizaion of echnical knowledge in he supply equaion. More specifically, a Markovian process is assumed for plan ouages wih parameers relaed o echnologies. This does no however include he possibiliy of sraegic capaciy wihholding, leading o price manipulaion, which, of course, has been a maor concern o regulaors in elecriciy markes, and has srong implicaions for accurae price forecasing.

19 The exisence of muliple, differen, componens in elecriciy pricing is refleced in Sevenson (2002). The price and demand series are decomposed ino muliple levels of resoluion wih wavele analysis and signal is differeniaed from noise wih a robus smooher-cleaner ransformaion. For he reconsruced daa, a hreshold auoregressive (TAR) model is suggesed wih demand as a criical variable, i.e. P ao + γd + a1 P = β o + πd + β1 P α P 1 q q β P q, if q, if D D < 0 0 Price changes are modelled due o he presence of a uni roo and assigned o one of wo regimes depending on wheher he change in demand is posiive or negaive. As his resricion excludes he impac on prices of oher fundamenals, such as supply, he model could be more flexible by defining he hreshold variable as a funcion of he raio demand/ supply. The smoohing procedure eliminaes he leakage of rapidly revering price spikes o more fundamenal resoluion levels, where informaion akes progressively longer o be impounded ino price. This allows a more reliable esimaion of he baseline regime. Price spikes however, are perceived as noise and despie heir informaion conen, are essenially removed from he daa. Non-parameric Modelling As he forecasing, raher han modeling, emphasis gradually becomes more pragmaic, several non-parameric echniques, such as geneic algorihms and neural neworks, have ineviably been adoped for price predicion. An indicaive lis includes neural neworks applicaions for he England -Wales pool by Ramsay and Wang (1997), for he California marke by Gao e al. (2000), Spain by Cenano Hernandez e al (2003) and Vicoria by Szkua e al. (1999); fuzzy regression models linking demand and price by Nakashima e al (2000); Fourier and Harley ransformaions by Nicolaisen e al. (2000). Alhough nonparameric models end o be flexible, can handle complexiy and hereby promising for

20 shor-erm predicions, hey do no provide srucural insighs and forecass of he price disribuion, which limis heir applicaion o risk managemen. Elecriciy Price Forecasing: Research Challenges and Pracice Alhough much of he saisical characerisics of elecriciy prices are replicaed wih exising sochasic models, heir idiosyncraic price srucure has no been fully represened adequaely in he empirical research lieraure. The research inspired by he needs of risk managemen is more concerned wih capuring he disribuion of prices over a period of ime, han in he acual levels of prices a paricular imes. For he laer case, he srucural approach is he mos appropriae bu unresolved modelling issues include: i) The facors refleced upon spo prices, for insance economic fundamenals, plan consrains, sraegic behaviour, perceived risks, forward conracing, rading inefficiencies and marke design effecs. ii) The magniude, relaive imporance and inra-day variaion of he above impacs on prices. iii) The non-linear naure and dynamics of srucural effecs as markes evolve. iv) The response of volailiy o fundamenals and shocks, and paricularly he sources of residual uncerainy when srucural componens are removed from prices. v) The srucural characerisics of marke cycles and he pricing scheme under emporal marke irregulariies, such as plan failure vi) How o fully endogenise he marke drivers of regime swiching, such as reserve margin. vii) The feedback of elecriciy prices on inpu variables, eg gas prices, when here is increasing evidence of coinegraion beween he wo commodiies. viii) Elecriciy markes end o be iner-conneced hrough neworks ha occasionally become congesed; again regime swiching beween spaial