Occasional Paper series

Transcription

1 Occasional Paper series No 84 / Shor-erm forecasing of GDP using large monhly daases a pseudo real-ime forecas evaluaion exercise by Karim Barhoumi, Szilard Benk, Riccardo Crisadoro, Ard Den Reijer, Audronė Jakaiiene, Pior Jelonek, Anónio Rua, Gerhard Rünsler, Karsen Ruh and Chrisophe Van Nieuwenhuyze

2 OCCASIONAL PAPER SERIES NO 84 / april 2008 Shor-erm forecasing of GDP using large monhly daases: a pseudo real-ime forecas evaluaion exercise by Karim Barhoumi, Szilard Benk, Riccardo Crisadoro, Ard Den Reijer, Audrone Jakaiiene, Pior Jelonek, Anónio Rua, Gerhard Rünsler 1, Karsen Ruh and Chrisophe Van Nieuwenhuyze * In 2008 all publicaions feaure a moif aken from he 10 banknoe. This paper can be downloaded wihou charge from hp:// or from he Social Science Research Nework elecronic library a hp://ssrn.com/absrac_id= * The projec described in his paper was conduced under he auspices of he Eurosysem working groups on Economeric Modelling and on Forecasing. The auhors would like o hank he members of he wo working groups, in paricular L. Reichlin, A. d Agosino, D. Giannone and C. Schumacher, for heir helpful commens. 1 Corresponding auhor: DG Research, European Cenral Bank, Kaisersrasse 29, Frankfur am Main, Germany, Gerhard.Ruensler@ecb.europa.eu

3 European Cenral Bank 2008 Address Kaisersrasse Frankfur am Main, Germany Posal address Posfach Frankfur am Main Germany Telephone Websie hp:// Fax All righs reserved. Any reproducion, publicaion or reprin in he form of a differen publicaion, wheher prined or produced elecronically, in whole or in par, is permied only wih he explici wrien auhorisaion of he or he auhor(s). The views expressed in his paper do no necessarily reflec hose of he European Cenral Bank. ISSN (prin) ISSN (online)

4 CONTENTS ABSTRACT 4 CONTENTS NON-TECHNICAL SUMMARY 5 1 INTRODUCTION 6 2 MODELS uarerly models Recursive mean and quarerly auoregressive model (AR) uarerly vecor auoregressive models (VAR) forecas averages Bridging monhly daa wih quarerly GDP Bridge equaions forecas average across indicaors Bridging wih facors Generalised principal componens 10 3 PSEUDO REAL-TIME FORECAST DESIGN Forecas design 10 4 DATA 11 5 RESULTS Forecas accuracy Encompassing ess 16 6 CONCLUSIONS 17 REFERENCES 19 EUROPEAN CENTRAL BANK OCCASIONAL PAPER SERIES SINCE

5 ABSTRACT This paper evaluaes differen models for he shor-erm forecasing of real GDP growh in en seleced European counries and he euro area as a whole. Purely quarerly models are compared wih models designed o exploi early releases of monhly indicaors for he nowcas and forecas of quarerly GDP growh. Amongs he laer, we consider small bridge equaions and forecas equaions in which he bridging beween monhly and quarerly daa is achieved hrough a regression on facors exraced from large monhly daases. The forecasing exercise is performed in a simulaed real-ime conex, which akes accoun of publicaion lags in he individual series. In general, we find ha models ha exploi monhly informaion ouperform models ha use purely quarerly daa and, amongs he former, facor models perform bes. Keywords: Bridge models, Dynamic facor models, real-ime daa flow JEL classificaion: E37, C53 4

6 NON-TECHNICAL SUMMARY NON-TECHNICAL SUMMARY Official esimaes of GDP growh are released wih a considerable delay. For he euro area as a whole, he firs official number is a flash esimae, which is published six weeks afer he end of he quarer. Meanwhile, economic analysis mus rely on monhly indicaors which arrive wihin he quarer such as, e.g. indusrial producion, reail sales and rade, surveys, and moneary and financial daa. In providing he saring poin for a longer-erm analysis, he assessmen of he curren sae of he economy is herefore an imporan elemen in macroeconomic forecasing. This paper performs a forecasing evaluaion of models used in cenral banks for compuing early esimaes of curren quarer GDP and shor-erm forecass of nex-quarer GDP. The models are designed o bridge early releases of monhly indicaors wih quarerly GDP. The paper considers a range of models for his purpose, including radiional bridge equaions and dynamic facor models. The key feaures of he evaluaion sudy presened in his paper are as follows. Firs, we examine he forecas performance under he real-ime flow of daa releases, aking accoun of he non-synchronous release of monhly informaion hroughou he quarer. Second, we use en large daases. In addiion o he euro area as a whole we consider daases from six euro area Member Saes and hree new Member Saes of he European Union. The main finding obained for he euro area counries is ha bridge models, which imely exploi monhly releases, fare considerably beer han quarerly models. Amongs hose, dynamic facor models, which exploi a large number of releases, do generally beer han averages of radiional bridge equaions. Resuls for he new Member Saes, on he oher hand, are difficul o inerpre. All models perform quie badly wih respec o naïve benchmarks, bu, given he shor evaluaion sample, i is hard o undersand wha drives he resuls. 5

7 1 INTRODUCTION This paper performs a forecasing evaluaion of models used in cenral banks for compuing early esimaes of curren quarer GDP and shor-erm forecass of nex-quarer GDP. These models are designed o bridge early releases of monhly indicaors wih quarerly GDP. Official esimaes of GDP growh are released wih a considerable delay. For he euro area as a whole, he firs official number is a flash esimae, which is published six weeks afer he end of he quarer. Meanwhile, economic analysis mus rely on monhly indicaors which arrive wihin he quarer such as, e.g. indusrial producion, reail sales and rade, surveys, and moneary and financial daa. In providing he saring poin for a longer-erm analysis, he assessmen of he curren sae of he economy is cerainly an imporan elemen in macroeconomic forecasing. This holds even more so as he longer-erm predicabiliy of quarerly GDP growh has declined since he 1990s (D Agosino, Giannone and Surico, 2006). A key feaure of his paper is ha we examine he forecas performance aking ino accoun he real-ime daa flow, ha is, he nonsynchronous release of monhly informaion hroughou he quarer. To his end, we replicae he design of he forecas exercise proposed by Rünsler and Sédillo (2003) for he euro area and by Giannone, Reichlin and Sala (2004) and Giannone, Reichlin and Small (2005) for he Unied Saes, which has also been applied for euro area aggregae daa by Angelini e al. (2008a) and Angelini, Banbura and Rünsler (2008b). We examine a wider range of models han previous sudies and consider, beside euro aggregae daa, individual counry daases. Macroeconomic indicaors are subjec o imporan differences in publicaion lags. Monhly indusrial producion daa, for insance, are released abou six weeks afer he end of he respecive monh for he euro area, while survey and financial daa are available righ a he end of he monh. Our forecas evaluaion exercise is designed o replicae he daa availabiliy siuaion ha is faced in real-ime applicaion of he models. In addiion, he models are reesimaed only from he informaion available a he ime of he forecas. However, our design differs from a perfec real-ime evaluaion insofar as we use final daa vinages and hence ignore revisions o earlier daa releases. In order o undersand he imporance of imely monhly informaion, he paper considers boh purely quarerly models and bridge equaions developed o link monhly releases wih quarerly GDP growh. Bridge equaions are used by many insiuions and have been sudied in various papers (Baffigi, Golinelli and Parigi, 2004; Diron, 2006; Rünsler and Sédillo, 2003). Tradiional bridge equaions can only handle few variables. To exploi informaion in he releases of several indicaors, he sandard approach is o average equaions using differen regressors. Recenly, Giannone, Reichlin and Sala (2004) and Giannone, Reichlin and Small (2008) have proposed o use facors exraced from large monhly daases o perform bridging which exploi a large number of indicaors wihin he same model (bridging wih facors). They propose o use he Kalman filer o esimae he facors and handle missing daa. 2 When bridging wih facors, however, one can consider alernaive esimaion mehods for he facors han ha based on he Kalman filer. Mehods ha have been used in he Eurosysem include he principal componen esimaor of he facors (Sock and Wason, 2002b) and he frequency domain-based wo-sep esimaor of Forni e al. (2005). I is herefore naural for his sudy o consider hese esimaors in he bridging wih facors framework. However, hese mehods have o be complemened wih some ool o handle missing daa. We will fill he missing daa of each series on he basis of univariae forecass following common pracice wih bridge equaions. 2 Beside he US and euro area applicaions cied above, he mehod is also used a Norges Bank (Aasvei and Trovik, 2007) and he Reserve Bank of New Zealand (Maheson, 2007). 6

8 I is imporan o sress ha while here are several sudies ha apply facor models for forecasing euro area daa (Marcellino e al. (2003) for euro area daa, Aris e al. (2005) for he Unied Kingdom, Bruneau e al. (2007) for France, Den Reijer (2007) for he Neherlands, Duare and Rua (2007) for Porugal, Schumacher (2007) for Germany, and Van Nieuwenhuyze (2005) for Belgium, among ohers), his paper considers he bridge version of hese models which is appropriae for real-ime shor-erm forecasing and can be meaningfully compared wih radiional bridge equaions. I INTRODUCTION Our model comparison is performed for he euro area as a whole as well as for six euro area counries. Moreover, we also assess he abovemenioned models for hree new members of he European Union. We end up wih en large monhly daases, wih an average dimension of more han one hundred series for each counry. Hence, we provide some cross-counry evidence regarding he relaive performance of he differen models considered. The paper is organised as follows. Secion 2 presens he models ha we consider in our exercise. Secion 3 discusses he pseudo realime forecas design, while secion 4 presens he daa. In secion 5 he empirical resuls are discussed. Finally, secion 6 concludes. 7

9 2 MODELS This secion describes several models ha may be used for forecasing GDP growh in he presence of large daases. We consider models ha rely solely on quarerly daa as well as models ha exploi he monhly naure of he available daa wih models ranging from he simple auoregressive process o he more sophisicaed dynamic facor models proposed in he lieraure. 2.1 UARTERLY MODELS RECURSIVE MEAN AND UARTERLY AUTOREGRESSIVE MODEL (AR) As benchmarks we use wo univariae ime series models for quarerly GDP growh y, i.e. a) average GDP growh, i.e. he naïve model y μ ε, and b) a firs-order auoregressive model, y μ= ρ( y μ ) + ε, (1) 1 where μ is a consan and ε is quarerly whie noise, N 0, σ. ε The forecasing performance of hese wo models will serve as a reference poin in forecas evaluaion. Given he differences in he saisical properies of GDP growh across counries, absolue measures of forecas performance are of limied use. We use he performance relaive o he above models insead UARTERLY VECTOR AUTOREGRESSIVE MODELS (VAR) FORECAST AVERAGES Anoher forecas ha uses purely quarerly daa can be obained from vecor auoregressive models. This approach has been repored o perform well, for example, for he Unied Kingdom (see Camba-Mendez e al., 2001). We run bivariae VARs including quarerly GDP and he quarerly aggregae of a single monhly indicaor, and average he forecass across indicaors. 2 ε 1. We consider a se of k monhly indicaors from he daase and calculae heir quarerly aggregaes { x i,, x 2,,..., x k, }. 2. For each indicaor x i,, we run a quarerly bivariae VAR, which includes he indicaor and GDP growh, pi zi, = μ + As zi, s + εi,, i = 1,..., k (2) i s= 1 z i, y, x i, ; from his VAR, we y of GDP growh. The lag lengh (p i ) of each VAR is deermined from he Schwarz informaion crierion (SIC). wih produce forecass i, h 3. We form he average of he k forecass from he individual indicaors, y k 1 k h i = 1 y i, h. y i, h These forecasing mehods do no exploi early monhly releases and hence hey do no deal wih ragged edges due o he non-synchronous flow of daa releases. 2.2 BRIDGING MONTHLY DATA WITH UARTERLY GDP BRIDGE EUATIONS FORECAST AVERAGE ACROSS INDICATORS Bridge equaions are a widely used mehod o forecas quarerly GDP from monhly daa (see, for example, Baffigi, Golinelli and Parigi, 2004). Two seps are involved: (i) he monhly indicaors are forecas over he horizon; (ii) he quarerly aggregaes of he obained forecass are used o predic GDP growh. In averaging across a large number of indicaors we follow he same bivariae approach as in secion 2.2 (see also Kichen and Monaco, 2003). 1. We consider a se of monhly indicaors { x1,, x 2,,..., x k,} and forecas he individual indicaors x i, over he relevan horizon from univariae auoregressive models, i, pi X = ρ X + u, i = 1,..., k, (3) s= 1 s i, s i, 8

10 wih coefficiens ρ s and whie noise erm 2 u i, N 0, σ i. 2. For each indicaor x i,, we consider he bridge equaion qi = i + βis xi, s + εi, s= 0 y μ, (4) which relaes quarerly GDP growh o he quarerly aggregae of he monhly indicaor, evaluaed in he hird monh of each quarer (see Mariano and Murasawa, 2003). Again, lag lenghs p i and q i in he equaions (3) and (4) are deermined from he SIC. We produce a forecas of GDP growh, y i, h, by insering he quarerly aggregaes x i, h of he forecass i, h x ino equaion (4). 3. We form he average of he k resuling forecass y i, h from he individual indicaors, as in sep 3 in secion BRIDGING WITH FACTORS Giannone, Reichlin and Sala (2004) and Giannone, Reichlin and Small (2005) propose he idea of bridging wih facors. They consider he bridge equaion y = f μ + β ' + ε, (5) where f is a quarerly aggregae of common facors driving all he monhly indicaors. Given a large se of monhly ime series x = (x 1,..., x n )', we consider he following facor srucure x = Λ f + ξ (6) which relaes he n 1 vecor of monhly ime series x o he r 1 vecor of common facors f = (f 1,..., f r ) via a marix of facor loadings Λ and o he idiosyncraic componen ξ = (ξ 1,..., ξ n )'. The number of saic facors r is ypically much smaller han he number of series n. The procedure works in wo seps. Firs he facors are exraced from he monhly indicaors. We will consider wo differen approaches for exracing he facors. 1. Simple principal componens (PC) following Sock and Wason (2002). 2. Two-sep approach (KF) based on principal componens and Kalman filering (Doz, Giannone and Reichlin, 2007). In his approach he common facors f are assumed o follow vecor auoregressive process which is driven by a vecor of innovaions u = (u 1,..., u q )' which are called he common shocks: 3 f p = s = 1 A f i s + Bu (7) The esimaion by PC requires he seing of he number of common facors r only. The lag lengh p and he number of common shocks q need no be specified since he PC esimaor does no ake ino accoun he dynamic properies of he common facors. The laer is explicily aken ino accoun by he KF approach, for which all he hree parameers mus be se. The forecas of GDP is obained in a second sep. The Kalman filer delivers he forecass of he common facors needed for predicing GDP, since i akes ino accoun heir dynamic properies. The forecas of GDP growh y h is obained by insering ino he bridge equaion he quarerly aggregaes of he esimaed common facors and heir forecas f h. Forecass of he facors are no direcly obained when facors are exraced using PC, since in his procedure he dynamics of he common facors are no explicily considered. For his reason, he h-seps ahead forecas for GDP growh is compued wih a direc approach, from he bridge equaion y μ β' f ε, where GDP appears wih a h lead of h periods and here is hence no need o forecas monhly facors. I remains o specify how o deal wih ragged edges due o he non-synchronous flow of daa releases. The KF esimaor deals efficienly wih ragged edges by replacing he missing observaions wih opimal predicions based on 3 For more deails on he generaliy of such represenaion, see Forni, Giannone, Lippi and Reichlin (2007). 2 MODELS 9

11 he enire se of monhly indicaors. Concerning PC we deal wih ragged edges by filling he missing monhly indicaors wih predicions based on univariae auoregressions, as done for he radiional bridge equaions. Again, he lag lengh is deermined from he SIC. Alernaive mehods are also sudied for robusness (see secion 5). The facors exraced using he KF are appropriae combinaions of presen and pas observaions wih weighs derived by aking ino consideraion he persisence of he common facors and he heerogeneiy in he informaional conen of every monhly indicaor relaive o he common facors. On he oher hand, he facors exraced by PC are linear combinaions only of he mos recen observaions since he PC esimaor does no ake ino consideraion he persisence of he common facors. Moreover, in PC all monhly indicaors are considered o be equally informaive abou he common facors GENERALISED PRINCIPAL COMPONENTS Anoher facor model ha accouns for facor dynamics is given by he generalised principal componens model (GPC) as pu forward by Forni e al. (2005). Wihin his framework, no specific model is posulaed for he facors. Therefore hey can no be prediced direcly, as i is he case wih he KF approach. In his paper, we deal wih his issue by effecively running a quarerly model. We combine GDP growh and he quarerly aggregaes of he monhly series in our daase, from which facors are esimaed. The GDP forecas is hen obained as a forecas of he common componen of GDP, as provided by he facor model. 4 3 PSEUDO REAL-TIME FORECAST DESIGN In his secion, he general principles underlying he forecasing exercise, which are applied o all models, are described. 3.1 FORECAST DESIGN The forecas evaluaion exercise is designed o predic quarerly GDP growh from monhly indicaors, which are published wihin he quarer. While flash esimaes of GDP growh are released around six weeks afer he end of he quarer, a considerable amoun of monhly daa on real aciviy wihin he same quarer is published earlier. There may be gains in making use of his informaion when producing shorerm forecass for GDP. Wih our forecas design, we aim a replicaing he real-ime applicaion of he models as closely as possible. We do no have real-ime daases a hand. However, following Rünsler and Sédillo (2003) and Giannone, Reichlin and Small (2005) we ake accoun of publicaion lags in he individual monhly series and consider a sequence of forecass o replicae he flow of monhly informaion ha arrives wihin a quarer. More precisely, we consider a sequence of eigh forecass for GDP growh in a given quarer, obained in consecuive monhs. The iming is illusraed in Table 2 and is bes explained using an example. Assume ha our objecive is o forecas GDP growh in he second quarer of We sar forecasing in January 2007: his forecas refers o nex quarer GDP and we denoe i as he firs monh one quarer ahead forecas. In moving forward in ime we produce a forecas in each monh, and wih he GDP Again, as wih bridge equaions and model PC, we deal wih ragged edges by filling he missing monhly observaions wih predicions based on univariae auoregressions. We do so before aggregaing he daa o quarerly frequency. Furher, parameers r and q are o be specified. They are deermined from he recursive minimum RMSE measure Possible alernaive soluions which are no considered in his paper include: (i) using a monhly inerpolaion of GDP among he variables in x and aking he projecion of he common componen of his variable for he quarerly GDP forecas (Alissimo e al, 2001); (ii) exracing monhly smooh facors and regressing GDP growh on heir appropriaely ransformed values (Alissimo e al. 2007). While one may add a forecas of he idiosyncraic componen, D Agosino and Giannone (2006) repor some evidence ha his componen is highly unforecasable.

12 flash esimae being published in mid-augus run he final forecas on 1 Augus. We denoe he laer as he second monh preceding quarer forecas, which is acually a backcas. This sequence of forecass is applied o each quarer of our ou-of-sample period. Anoher issue concerns he unbalancedness of he available daa. The individual monhly series are published wih differen delays. As a resul, he number of missing observaions a he end of he sample differs across series. Survey and financial daa, for insance, are available righ a he end of he monh, bu indusrial producion daa are published, for example, wih a delay of six weeks for he euro area. Similar lags are found for oher official saisics. In his respec, Giannone, Reichlin and Small (2005) and Banbura and Rünsler (2007) have shown ha ignoring unbalancedness in he daa may have srong effecs on he resuls. In his paper, we fully accoun for unbalancedness. We download our daases a he beginning of he monh, when mos of he survey and financial marke daa for he previous monh are already available. For each forecas, we apply in a recursive way he daa release paern ha we find in our daases o he ime a which he forecass are made. Formally, our pseudo real-ime daases X are defined as follows: given our main se of monhly observaions, T n marix X T, as downloaded on a cerain day of he monh, we define wih n marix X he observaions from he original daa X T up o period, bu wih elemens X (-h,i) eliminaed, if observaion X T (T-h,i) is missing in X T (for i = 1,..,n, and h 0). A forecas y h made in period is based on informaion se X. In all cases, we also reesimae and re-specify he models in each poin in ime based on informaion se X. Given he absence of well agreed informaion crieria, he specificaion of facor models, i.e. he choices of he numbers of saic (r) and dynamic facors (q) and he number of lags p in equaion (6), is based on a recursive minimum RMSE crierion. In each monh of he evaluaion period, we simply selec he specificaion ha has provided he bes forecass in he pas. More precisely, we calculae he average RMSE across all horizons and selec he specificaion wih minimum average RMSE. We repea his in each individual monh of he evaluaion period. We limi he specificaion search o values of r 8, q r, and p 3. In addiion, we consider forecas averages across all specificaions. For hose models ha use only quarerly daa, he same rules can be applied. A each poin in ime, we form he quarerly aggregaes x i, of individual series x i, from pseudo real-ime daases X and rea an observaion in x i, as missing if he monhly daa are no complee. Naurally, he forecass hen remain unchanged for hree consecuive monhs, and are updaed only once new quarerly daa arrives, depending on publicaion lags. 4 DATA The daa used in his paper comprise en large daases ha have been compiled for he euro area as a whole as well as for six euro area counries (Belgium, Germany, France, Ialy, Neherlands, Porugal) and hree new Member Saes (Lihuania, Hungary and Poland). The daases were downloaded in eiher early July or Augus The daases have an average dimension of more han one hundred series for each counry and all series are available from January 1991 up o mid-2006, apar from he new Member Saes where he sample period is shorer (see Table 1 for deails on he daases). Addiionally, quarerly real GDP series were also colleced for he corresponding sample period. All daa are seasonally adjused. For he analysis, he daa are differenced o be saionary. For rending daa (such as indusrial producion, employmen, reail sales) we ake logarihms beforehand, which amouns o calculaing raes of change, while survey and financial daa are no logarihmised. We use hree-monh 4 DATA 11

13 Table 1 Daases No of series Producion and sales Surveys of which Financial Prices Oher Sample sar Euro area EA M1 Belgium BE M1 Germany DE M1 France FR M1 Ialy IT M1 Neherlands NL M1 Porugal PT M1 Lihuania LT M1 Hungary HU M1 Poland PL M1 Sources: differences of he monhly daa, i.e. he raes of change agains he same monh of he previous quarer, (x x 3 )/3. 5 This implies ha he quarerly aggregae of he series is given by x x x 1 x 2 )/3 from a log-linear approximaion. In applicaion, daa X are sandardised o mean zero and variance one in a recursive manner. For he facor models, we also clean he daa from ouliers in a recursive manner. 6 5 RESULTS Concerning he ou-of-sample period, for he euro area counries, we evaluae he forecas performance of he various models over he period from o For new Member Saes, he shor samples require runcaing he evaluaion period o o FORECAST ACCURACY Taking ino accoun he number of models considered and he differen model selecion crieria, balancing mehods, ec. we end up wih almos fory specificaions for each counry. In order o make he presenaion of he resuls racable, we limi our presenaion o he besperforming specificaions while discussing he sensiiviy of he resuls obained. 8 Firs, regarding quarerly VARs and radiional bridge equaions, we considered wo alernaive ses of indicaors. The firs se comprises all indicaors in he daase. The second se conains only hose indicaors ha expers in cenral banks regard as being he mos imporan Table 2 Timing of forecas exercise (Example: forecass for second quarer) uarer o be forecas Forecas made on firs day of One quarer ahead 1 January 2 February 3 March Curren 1 April 2 May 3 June Preceding 1 July 2 Augus Sources: From a heoreical perspecive, monh-on-monh differences, x x 1 may be preferred as hey allow for a more precise modelling of dynamics by avoiding a moving average srucure of he residuals. From a pracical perspecive, using hree-monh differences has he advanage ha noise in he daa is reduced and daa irregulariies are smoohed ou. We find ha hree-monh differences end o give beer forecass. The resuls are available from he auhors upon reques. Oulier deecion was based on a simple rule applied o he differenced series: we idenified hose observaions as ouliers, which were five imes larger in absolue value han he 20% quanile of he series disribuion. We eiher se hese ouliers as missing values (model KF) or replace hem wih he value of he cu-off poin. When using recursive RMSE crierion for he facor model specificaions, we use a burning in phase saring in o find he iniial specificaion. All he resuls are available from he auhors upon reques. 12

14 when monioring economic aciviy. While differences are minor, he laer fares slighly beer. We herefore repor he resuls from he second se (labelled as VAR respecively in Table 3). Second, as concerns facor models, we have considered alernaive ways o specificaion search in addiion o he recursive RMSE crierion as described in secion 3.1. As one alernaive opion, we have combined informaion crieria proposed by Bai and Ng (2002, 2007) o deermine he number of saic and dynamic facors wih he SIC o deermine lag lengh p in equaion (6). In addiion, we have considered unweighed forecas averages across all specificaions. Again, we find he differences o be raher small, bu for all facor models, he recursive RMSE selecion slighly ouperforms he alernaives considered. Third, for he PC and GPC esimaion mehod we have also considered alernaive mehods o deal wih ragged edges owing o he synchroniciy of daa releases. Precisely, in addiion o he univariae models, we consider alernaives in which he predicions are obained from mulivariae models. Firs we shif he series wih missing observaions forward in ime: if he las m observaions are missing in ~ series i, lagged series x i, x i, m is used in place of x i,. Moreover, for he PC esimaes we have also considered he EM algorihm developed by Sock and Wason, 2002a o handle missing observaions. The differences are, on average, small, bu univariae models repored here end o ouperform he alernaive mehods. 9 The main resuls for he preferred specificaions are shown in Table 3. We repor he RMSE of each model relaive o he naïve benchmark of consan growh. A number lower han one indicaes ha he model s forecass are more accurae han he average growh over he pas sample. In addiion o he individual counries, we repor in he righ panel he mean RMSE across he euro area counries (excluding he euro area as a whole) and new Member Saes. In he boom panel we repor he rank across 9 The PC-EM algorihm esimaes he facors from he available observaions and uses hese esimaes o predic missing observaions. This procedure is ieraed unil convergence. 5 RESULTS Table 3 Resuls overview (forecass for euro area counries and for NMS) Average RMSE for preceding, curren and one-quarer-ahead forecass relaive o he naive forecas EA BE DE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC Ranks of models according o he RRMSE measure EA BE DE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC Sources: AR denoes a univariae auoregressive model for GDP; VAR and BE denoe he quarerly bivariae VAR and bridge equaion models respecively. KF, PC and GPC denoe he 3 versions of facor models, based on he Kalman filer, principal componens and generalised principal componens respecively. See Table 1 for an explanaion of counry abbreviaions; EA denoes daa for he euro area aggregae, while EuroA and NMS denoe averages of he various measures across he six euro area Member Saes and he hree new Member Saes included in he invesigaion respecively. 13

15 models and, in he las wo columns, he mean rank for euro area counries and new Member Saes. The findings differ qualiaively among he euro area counries and he new Member Saes. The wo groups of counries are herefore discussed separaely. The resuls for he euro area counries included in he sudy migh be summarised as follows: a. Models ha use monhly daa end o ouperform hose models ha use purely quarerly daa. Bridge equaion and facor models, ha incorporae early releases, produce forecass ha are more accurae han hose based on quarerly models. These resuls highligh he imporance of exploiaion of monhly releases. b. Facor-based esimaes are in general more accurae han forecass based on simple bridge equaions. Wih he excepion of he Neherlands (and one minor excepion in he case of Ialy), he hree facor models rank ahead of he alernaive models. This indicaes ha bridging wih facors exraced from many monhly ime series is preferable o he average of many small bridge equaions each consruced wih individual monhly series. c. Among he facor models he mos accurae forecass are hose based on facors exraced by he KF proposed by Giannone, Reichlin and Small (2005). The KF mehods aain rank one for all counries bu France and he Neherlands. For France, model PC fares slighly beer, while for he Neherlands he quarerly VAR performs bes. 10 d. Esimaes of GDP growh a euro area aggregae level are more accurae han he esimaes of GDP growh in individual Member Saes. The esimaes based on he common facors exraced by he KF improve upon he naïve forecas by 25 percen in he euro area. The accuracy relaive o he naïve model is much less pronounced for individual counries and for several counries we find lile improvemen over he naïve consan growh model. The differences in he average RMSE across counries are small. However, one can esablish significan differences from considering he cross-counry perspecive. Assume ha he ranks of he individual models are independen across counries and consider he null hypohesis ha wo models perform equally well. Under he null hypohesis, he probabiliy ha model 1 is found o perform beer han model 2 in k of n counries is found from he binomial disribuion wih 0.5 n k j = 1 ( ). For n=7 one can esablish ha he probabiliy ha model 1 performs beer han model 2 in six or all seven cases amouns o p=0.063 and p=0.008 respecively. Hence, we can esablish from he rank saisics ha he improvemen of facor models exraced by KF and PC over he bridge equaions, quarerly VARs and he facors exraced by GPC is significan. Equivalenly, he forecass based on facors exraced using KF are significanly more accurae han hose based on facors exraced by PC. As regards he hree new Member Saes, in general he model-based forecass are no uniformly beer han he naïve forecass. These findings may be relaed o he shor samples a hand (daa sar only in ), he rapid ransiion of he economies, which implies unsable relaionships among series, and possibly oher issues regarding he qualiy of he daa (for example, a lack of seasonally adjused monhly daa means i is necessary o use 12-monh differences of he daa). Tables 4 o 6 show he corresponding measures for averages of he RMSE over he individual quarers of he forecas horizon. One can see ha 10 Alhough no repored in his paper, for he Neherlands, he KF model based on informaion crieria performs bes across all specificaions including he quarerly VARs. n j 14

16 5 RESULTS Table 4 Resuls overview preceding quarer Forecass for euro area counries and for NMS Average RMSE for preceding quarer forecass relaive o he naive forecas EA BE DE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC Rank of models according o he RRMSE measure EA BE DE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC AR denoes a univariae auoregressive model for GDP; VAR and BE denoe he quarerly bivariae VAR and bridge equaion models respecively. KF, PC and GPC denoe he 3 versions of facor models, based on he Kalman filer, principal componens and generalised principal componens respecively. See Table 1 for an explanaion of counry abbreviaions; EA denoes daa for he euro area aggregae, EuroA and NMS denoe averages of he various measures for he six euro area Member Saes and he hree new Member Saes included in he invesigaion respecively. he relaive performance of he models remains sable across horizons. The facor models, in paricular, coninue o ouperform he quarerly models and bridge equaions, wih a model based on facors exraced by he KF performing bes for he preceding and curren quarer forecass. The differences across mehods are less pronounced for he one-quarer-ahead forecass when he relaive RMSE ends o one, which represens non-forecasabiliy. Table 5 Resuls overview curren quarer (forecass for euro area counries and for NMS) Average RMSE for curren quarer forecass relaive o he naive forecas EA BE DE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC Rank of models according o he RRMSE measure EA BE DE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC Sources: AR denoes a univariae auoregressive model for GDP; VAR and BE denoe he quarerly bivariae VAR and bridge equaion models respecively. KF, PC and GPC denoe he 3 versions of facor models, based on he Kalman filer, principal componens and generalised principal componens respecively. See Table 1 for an explanaion of counry abbreviaions; EA denoes daa for he euro area aggregae, while EuroA and NMS denoe averages of he various measures across he six euro area Member Saes and he hree new Member Saes included in he invesigaion respecively. 15

17 Table 6 Resuls overview one quarer ahead Forecass for euro area counries and for NMS Average RMSE for one-quarer-ahead forecass relaive o he naive forecas EA BE DE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC Rank of models according o he RRMSE measure EA BE GE FR IT NL PT LT HU PL EuroA NMS AR VAR BE KF PC GPC Sources: AR denoes a univariae auoregresive model for GDP; VAR and BE denoe he quarerly bivariae VAR and bridge equaion models respecively. KF, PC and GPC denoe he 3 versions of facor models, based on he Kalman filer, principal componens and generalised principal componens respecively. See Table 1 for an explanaion of counry abbreviaions; EA denoes daa for he euro area aggregae,while EuroA and NMS denoe averages of he various measures for he six euro area Member Saes and he hree new Member Saes included in he invesigaion respecively. 5.2 ENCOMPASSING TESTS Forecas encompassing ess are anoher means o assess he relaive performance of models. The encompassing es beween wo alernaive models 1 and 2 is based on a regression of he acual daa y on forecass f 1, and f 2, from wo models (see, e.g. Clemens and Hendry, 1998: 228ff), y = λ + ( 1 λ) f + u, 0 λ 1. (8) f 1, 2, Parameer λ gives he opimal weigh of model 1 in he combined forecas. In he exreme case, a value of λ = 1 indicaes ha model 1 dominaes model 2, i.e. forecass f 2, from model 2 do no conain any informaion beyond he informaion conained in forecass f 1,. Hence, forecass from model 2 can be disregarded. Equivalenly, a value of λ = 0 implies ha forecass from model 1 can be disregarded. In he inermediae case of 0 < λ < 1, combinaions of forecass from he wo models migh be considered. Table 7 shows encompassing ess of he models shown in Table 3 agains he bes-performing one, KF. Here, a large value of λ means ha a model based on facors esimaed by he KF dominaes he alernaive model. The ess are shown for he forecass obained in he second monh of he curren quarer, which represens he cenre of our forecas horizon. For he euro area counries, he resuls indicae some dominance of esimaes based on he facor model wih KF agains models AR, VAR and bridge equaions. Esimaes of λ always exceed a value of 0.5 and are in many close o one. The hypohesis of λ = 0, i.e. ha he esimaes based on facors exraced by he KF would no add informaion o forecass from hese alernaive models is uniformly rejeced. The opposie hypohesis of λ = 1, i.e. ha models AR(1), BE and VAR do no add informaion o forecass from he KF-based facor model is rejeced only in he case of Germany. Furhermore, he KF-based esimaes of he facor model also end o aain high weighs agains he alernaive facor models. Wih he excepion of model GPC in case of Belgium, λ is esimaed larger han 0.5, while he hypohesis of λ = 0 is rejeced in mos cases. We have also performed encompassing ess for oher forecas horizons. Wih one excepion, he findings remain reasonably robus across 16

18 6 CONCLUSIONS Table 7 Encompassing ess agains model KF (seleced models) (forecass for euro area counries and for NMS) Poin esimae of parameer λ in he encompassing regression y = λ f 1, + (1-λ) f u 2, + Second monh curren quarer forecass EA BE DE FR IT NL PT LT HU PL AR VAR BE PC GPC Tes of he null hypohesis of λ = 1 EA BE DE FR IT NL PT LT HU PL AR * ** ** ** VAR * ** * ** BE * * ** ** PC * ** GPC * ** ** ** ** and * denoe rejecion of he null hypohesis of λ = 1 a he 5% and 10% levels, respecively. Tes of he null hypohesis of λ = 0 EA BE DE FR IT NL PT LT HU PL AR VAR BE PC GPC and + denoe rejecion of he null hypohesis of λ = 0 a he 5% and 10% levels, respecively. Sources: AR denoes a univariae auoregressive model for GDP; VAR and BE denoe he quarerly bivariae VAR and bridge equaion models, respecively. KF, PC and GPC denoe he 3 versions of facor models, based on he Kalman filer, principal componens and generalised principal componens respecively. See Table 1 for an explanaion of counry abbreviaions; EA denoes he euro area aggregae horizons. The excepion is ha he dominance of esimaes based on he KF agains he esimaes based on PC is los for higher horizons, i.e. he one-quarer-ahead quarer forecass. A possible reason is relaed o he efficiency of model KF in dealing wih unbalanced daa. While his advanage may be paricularly imporan for he very shor horizons, i may become less imporan for he nex quarer forecass. 11 For he new Member Saes, he ranking among forecass mehods canno be esablished. This is expeced given ha he evaluaion and esimaion samples are boh very shor. 6 CONCLUSIONS This paper has performed a large-scale forecas exercise, involving en large daases for en European counries and one large daase for he euro area economy. We have compared simple quarerly models wih models exploiing more imely monhly daa o obain early esimaes and shor-erm forecass of quarerly GDP growh. Amongs hese models we have considered boh radiional bridge equaions and facor models adaped o handle unsynchronised daa releases. The forecas design has aimed a replicaing he real-ime applicaion of he models as closely as possible. I deviaes from a real-ime applicaion only insofar as we had o use final daa releases, as such real-ime daa are no readily available. The main message of he resuls obained for he euro area counries is ha models ha exploi imely monhly releases fare beer han quarerly 11 The resuls ha he gains from using he KF are less pronounced for longer horizons are in line wih findings based on he Mone Carlo exercise performed by Doz, Giannone and Reichlin (2007). 17

19 models. Amongs hose, facor models, which exploi a large number of releases, do generally beer han averages of bridge equaions. This suggess ha he idea of using facors o bridge monhly wih quarerly informaion is promising and should be more sysemaically explored in he Eurosysem. We have also ried o esablish a ranking beween differen esimaors and beween differen mehods o handle unbalanced daa a he end of he sample. Differences beween differen approaches were found o be small, wih he excepion of he experimen based on he euro area aggregae daase where he Kalman-filer-based procedure proposed by Giannone, Reichlin and Sala (2004) and Giannone, Reichlin and Small (2005) gives significanly beer resuls. Resuls for he new Member Saes, on he oher hand, are difficul o inerpre. All models perform quie badly wih respec o naïve benchmarks, bu, given he shor evaluaion sample, i is hard o undersand wha drives he resuls. do sugges ha differences beween mehods are minimal. 5. Models ha handle he daa flow problem of shor-erm forecasing in a unified framework can be exended o provide an inerpreaion of he conribuions of daa releases o he forecas and o he uncerainy around he forecas along he lines suggesed by Angelini e al. (2008), Banbura and Rünsler (2007) and Giannone, Reichlin and Small (2005). 6. Resuls for he new Member Saes should be furher evaluaed. In order o perform he evaluaion and he comparison, he presen sudy is based on very shor esimaion samples which make he resuls unreliable. However, a presen i is possible o use a leas en years of daa for he new Member Saes. Resuls should be revaluaed using he longer sample. On he basis of his firs evaluaion we can ouline an agenda for more deailed sudies on shor-erm forecasing mehods: 1. Evaluae he design of bridge equaions which are rouinely used in some insiuions. 2. The bridge models can be furher exended and refined boh in erms of idenifying key monhly releases and exending he class of models. Bayesian VARs exended o handle he bridge problem, for example, should be given furher consideraion. 3. For facor-based bridge equaions, furher hough should be given o variables selecion (size of he daase) and daa ransformaions. 4. Our evaluaion does no clearly disinguish beween mehods of esimaion and mehods of filling missing observaions a he end of he sample. This could be he subjec of a more deailed evaluaion alhough our resuls 18

20 REFERENCES REFERENCES Aasvei, K. A. and T. G. Trovik (2007), Nowcasing Norwegian GDP: The role of asse prices in a small open economy. Norges Bank working paper 2007/09. Alissimo, F., A. Basanei, R. Crisadoro, M. Forni, M. Hallin, M. Lippi, L. Reichlin and G. Veronese (2003), A real-ime coinciden indicaor for he euro area business cycle. CEPR discussion paper No Alissimo, F., R. Crisadoro, M. Forni, M. Lippi and G. Veronese (2007), New Eurocoin: Tracking economic growh in real ime. Bank of Ialy, Tema di Discussione, No 631. Angelini, E., G. Camba-Mendez, D. Giannone, L. Reichlin and G. Rünsler (2008a), Shor-erm forecass of euro area GDP. Working Paper series, forhcoming. Angelini, E., M. Banbura, and G. Rünsler (2008b), Esimaing and forecasing he euro area monhly naional accouns from a dynamic facor model. Working Paper series, forhcoming. Aris, M.J., A. Banerjee and M. Marcellino (2005), Facor forecass for he UK. Journal of Forecasing 24, Bai, B.J. and S. Ng (2002), Deermining he number of facors in approximae facor models. Economerica 71(1), Bai, B.J. and S. Ng (2007), Deermining he number of primiive shocks in facor models. Journal of Business and Economic Saisics 25(1), Baffigi, A., R. Golinelli, and G. Parigi (2004), Bridge models o forecas he euro area GDP. Inernaional Journal of Forecasing 20(3), Banbura, M. and G. Rünsler (2007), A look ino he facor model black box: publicaion lags and he role of hard and sof daa in forecasing GDP. Working Paper No 715. Bruneau, C., O. de Band, A. Flageolle, and E. Michaux (2007), Forecasing inflaion using economic indicaors: he case of France. Journal of Forecasing 26 (1), Camba-Mendez, G., G. Kapeianos, M. Smih and R. Weale (2001), An auomaic leading indicaor of economic aciviy: forecasing GDP growh for European counries. Economerics Journal 4(1), Clemens, M. and D. Hendry (1998), Forecasing Economic Time Series. Cambridge: Cambridge Universiy Press. D Agosino, A., D. Giannone and Surico, F. (2006), (Un)predicabiliy and macroeconomic sabiliy. Working Paper No 605. D Agosino, A. and D. Giannone (2006), Comparing alernaive predicors based on large-panel facor models. Working Paper No

21 Den Reijer, A.H.J. (2007), Forecasing Duch GDP using large scale facor models. mimeo. Doz, C., D. Giannone and L. Reichlin (2007), A Two-Sep Esimaor for Large Approximae Dynamic Facor Models Based on Kalman Filering. CEPR discussion paper Duare, C. and A. Rua (2007), Forecasing inflaion hrough a boom-up approach: How boom is boom? Economic Modelling 24, Durbin, J. and S.J. Koopman (2001), Time Series Analysis by Sae Space Mehods. Oxford: Oxford Universiy Press. Forni, M., D. Giannone, M. Lippi and L. Reichlin (2007), Opening he black box srucural facor models wih large gross-secions. Working Paper No 712. Forni, M., M. Hallin, M. Lippi and L. Reichlin (2000), The generalized dynamic facor model: idenificaion and esimaion. The Review of Economics and Saisics, 82, Forni, M., M. Hallin, M. Lippi, and L. Reichlin (2005), The generalized dynamic facor model: one-sided esimaion and forecasing. Journal of he American Saisical Associaion, 100, Giannone, D., L. Reichlin and L. Sala (2004), Moneary Policy in Real Time. Mark Gerler and Kenneh Rogoff, ediors, NBER Macroeconomics Annual 2004, pages MIT Press. Giannone, D., L. Reichlin and D. Small (2005), Nowcasing GDP and inflaion: he real-ime informaional conen of macroeconomic daa releases. Federal Reserve Bank Finance and Economics Discussion Series , Journal of Moneary Economics forhcoming. Kichen, J. and R. Monaco (2003), Real-ime forecasing in pracice: The U.S. Treasury saff s real-ime GDP forecas sysem. Business Economics 38 (4), Mariano, R. and Y. Murasawa (2003), A new coinciden index of business cycles based on monhly and quarerly daa. Journal of Applied Economerics, 18, Marcellino, M., Sock, J.H. and M. Wason (2003), Macroeconomic forecasing in he euro area: counry specific versus euro wide informaion. European Economic Review 47, Maheson, T. (2007), An analysis of he informaional conen of New Zealand daa releases: he imporance of business opinion surveys. Reserve Bank of New Zealand Discussion Paper DP2007/13. Rünsler, G. and F. Sédillo (2003), Shor-erm esimaes of euro area real GDP by means of monhly daa. Working Paper No 276. Schumacher, C. (2007), Forecasing German GDP using alernaive facor models based on large daa ses. Journal of Forecasing 26(4), Sock, J.H. and M. Wason (1999), Forecasing inflaion. Journal of Moneary Economics 44,