Lecture 13 Auto/cross-correlation

Transcription

1 Lecure 13 Auo/cross-correlaion 1 Generalized Regression Model The generalized regression model's assumpions: (A1) DGP: y = X + is correcly specified. (A) E[ X] = 0 (A3 ) Var[ X] = Σ =. (A4) X has full column rank rank(x)=k-, where T k. We assume ha he s in he sample are no longer generaed independenly of each oher. Ignoring heersocedasiciy, we have a new Σ: E[ i j X] = ij if i j = if i=j 1

2 Auocorrelaion: Examples We keep he linear model: y ' X - Firs-order auoregressive auocorrelaion: AR(1) 1 u - Fifh-order auoregressive auocorrelaion: AR(5) - Third-order moving average auocorrelaion: MA(3) u u u u u Noe: This example is described as hird-order moving average auocorrelaion, denoed MA(3), because i depends on he hree previous innovaions as well as he curren one. 8 Implicaions for OLS Similar o he heeroscedasiciy resuls: - OLS is unbiased, consisen (we need addiional assumpions), asympoic normaliy (we need addiional assumpions and definiions), bu inefficien. - OLS sandard errors are incorrec, ofen biased downwards. A very imporan excepion: The lagged dependen variable y = x + y = -1 + u. Now, Cov[y -1, ] 0 => IV Esimaion Useful sraegy: OLS esimaes wih he Newey-Wes (NW) robus esimaion of he covariance marix. Recall NW s HAC esimaor of Q * : S T = S 0 + (1/T) l w L (l) =l+1,...,t (x -l e -l e x + x e e -l x -l )

3 Newey-Wes esimaor The performance of NW esimaors depends on he choice of he kernel funcion i.e., w L (l)- and runcaion lag (L). These choices affec he resuling es saisics and render esing resuls fragile. NW sandard errors perform poorly in Mone Carlo simulaions (end o be downward biased), presumably because he moivaion for he kernel is o guaranee psd of he covariance marix raher han o produce good sandard errors wihou he psd guaranee. Kiefer and Vogelsang (00) use differen weighs (w=1-1/t). This esimaor is inconsisen, bu i works beer han kernels ha yield consisen esimaes. Whiney Newey, USA Kenneh D. Wes, USA Newey-Wes esimaor: Implemenaion To implemen he HAC esimaor, we need o deermine: lag order i.e., runcaion lag (L) or bandwidh-, and kernel choice (w l (L)). (1) Truncaion lag (L) No opimal formula; hough selecing L o minimize MSE is popular. The choice of L maers. In general, for ARMA models we have: - Shorer lags: Larger Bias, Smaller Variance - Longer lags: Smaller Bias, Larger Variance To deermine L, we use: - Trial and error, informed guess. - Rules of humb. For example: L+1 = 0.75T 1/3. () Kernel Choice - In heory, he kernel choice maers. - In pracice, a leas for psd kernels, i does no seem o maer. 3

4 Newey-Wes esimaor: Implemenaion () Kernel Choice w L (x) Based on he resuls of Andrews (1991), he QS kernel is usually implemened o calculae HAC SE. x Newey-Wes esimaor: Improvemens Oher han finding a suiable kernel funcion and a proper L, he performance of HAC esimaors may be improved by: (1) Pre-whiening he daa -Andrews and Monahan (199). Regress x i e on is lagged values. () Compuing sample auocovariances based on forecas errors, insead of OLS residuals -Kuan and Hsieh (006). Replace e wih one-sep-ahead forecas errors: fe = y X b -1, where b -1 is he recursive OLS esimaors based on he subsample of he firs 1 observaions. 4

5 Newey-Wes esimaor: Inconsisency Recall ha a key assumpion in esablishing consisency is ha L as T, bu L/T 0. In pracice, L/T is never equal o 0, bu some posiive fracion. Under his assumpion, he NW esimaor is no longer consisen. Thus, radiional -and F-ess no longer converge in disribuion o Normal and χ RVs, bu hey do converge in disribuion o RVs ha do no depend on he unknown value of Ω. Tess are sill possible. Kiefer and Vogelsang (005) derive limiing disribuion for S T, which is complicaed, bu he 95% criical values (CV) for -ess can be consruced using he following polynomial: CV (L/T) = (L/T) ((L/T) (L/T) 3. Newey-Wes esimaor: KVB The (kernel) HAC esimaion requires he choices of he kernel funcion and L. Such choices are somewha arbirary in pracice. To avoid hese difficulies, Kiefer, Vogelsang, and Bunzel (000), KVB, proposed an approach ha yields an asympoically pivoal es wihou consisen esimaion of he asympoic covariance marix. Idea: Use a normalizing marix o eliminae he nuisance parameers in Q -1/, he marix square roo of Q * T & impose no runcaion. Le φ = (1/ T) j=1,...,t x j e j Normalizing marix: C T = (1/T) =l,...,t φ φ = (1/T ) =1,...,T ( j=l,..., x j e j ) ( j=l,..., e j x j ) 5

6 Newey-Wes esimaor: KVB Normalizing marix: C T = (1/T) =l,...,t φ φ = (1/T ) =1,...,T ( j=l,..., x j e j ) ( j=l,..., e j x j ) This normalizing marix is inconsisen for Q * T bu is free from he choice of kernel and L. (Since Q * T is no esimaed, i is robus o heeroscedasiciy and auocorrelaion!) We will use his C T marix o calculae ess. For example, o es r resricions H 0 : (R β q=0), we have he following saisic W + T = T (Rˆb T q) [R (X X) 1 C T (X X) 1 R] -1 (Rˆb T q). Alhough he asympoic disribuion of W + T is non-sandard, i can be simulaed -Lobao (001). Newey-Wes esimaor: KVB KV (00) showed ha C T is algebraically equivalen o Q *,B T (where B sands for Barle kernel) wihou runcaion - i.e., L(T)=T. Then, usual W based on Q *,B T wihou runcaion is he same as W + T/. KVB derive he (non-sandard) asympoic disribuion of he convenional -es of H 0 : β i =r ; bu using heir robus version, + : where δ i is he i-h diagonal elemen of (X X) 1 C T (X X) 1, W is a sandard Wiener process, and B(r) is a Brownian Bridge i.e., B k (r) = W k (r) rw k (1), 0 r 1. This disribuion is symmeric, bu more disperse han he N(0,1). 6

7 Newey-Wes esimaor: KVB KVB repor he quaniles of he asympoic disribuion of he usual -es, using C T and using he NW HAC SE, wihou runcaion - i.e., L(T) = T. (Noaion: Q * =Σ kernel ) Remark: KV (00) shows ha under cerain assumpions he -es wih NW s SE wihou runcaion are also asympoically pivoal. Newey-Wes esimaor: KVB - Remarks An advanage of esing wih KVB s C T marix is ha is asympoic disribuion usually provides good approximaion o is finie-sample counerpar. Tha is, he empirical size is close o he nominal size (α). This is no he case for he NW HAC SE: in finie samples, hey are downward biased. Tess are usually over-sized i.e., no conservaive. KV (00b) show ha, for Q *,k wih he runcaion lag equal o sample size, T, Q *,B T compares favorably wih Q *,QS T in erms of power. This is in conras wih he resul in HAC esimaion, where he laer is usually preferred o oher kernels. Reference: Kiefer, N. M., T. J. Vogelsang and H. Bunzel (000). Simple robus esing of regression hypohesis, Economerica, 68,

8 Implicaions for OLS: Relaive Efficiency We define relaive efficiency of GLS agains OLS as: -1 1 X X i RE i 1 1 X X X X X X Le y = x + y = -1 + u. Also, le x also follow an AR(1) process: x = θ x -1 +ξ. Then, when T is large, i can be shown ha RE Var ˆ GLS Var b OLS The relaive efficiency can be very poor for large for any given θ. For example, suppose =θ=.7, RE Then, if he SE[ GLS ]= 1, SE[b] = 1.71 (= ); ha is, he OLS sandard error is abou 71% bigger ha is GLS counerpar. i Implicaions for OLS: Relaive Efficiency RE Var ˆ GLS Var b OLS The OLS esimaors can be quie reasonable for a low degree of auocorrelaion for any given θ, for example, when =.3 and θ=.9, hen RE The inefficiency of OLS is difficul o generalize. We end o see increase inefficiency wih increasing values of he disurbance variances. In pracice, i is wors in low frequency -i.e., long period (year)- slowly evolving daa. Can be exremely bad. GLS vs. OLS, he efficiency raios can be 3 or more. Given he poenial efficiency gain, i makes sense o es for auocorrelaion. 8

9 Tesing for Auocorrelaion: LM ess There are several auocorrelaion ess. Under he null hypohesis of no auocorrelaion of order p, we have H 0 p = 0. Under H 0, we can use OLS residuals. Breusch Godfrey (1978) LM es. Similar o he BP es: - Sep 1. (Auxiliary Regression). Run he regression of e i on all he explanaory variables, z. In our example, e = X β + α 1 e α p e -p + v - Sep. Keep he R from his regression. Le s call i R e. Then, calculae eiher (a) F = (R e /p)/[(1-r e )/(T-(p+1)], which follows a F p,(t-(p+1) or (b) LM = T R e d χ p. Tesing for Auocorrelaion: Pormaneu ess Box-Pierce (1970) es. I es H 0 p = 0 using he sample correlaion r j: r j: = Σ =1,...T-j e e -j / Σ =1,...T e Then, under H 0 Q = T Σ j=1,...,p r j d χ p Ljung-Box (1978) es. A variaion of he Box-Pierce es. I has a small sample correcion. LB = T (T-) Σ j=1,...,p r j /(T-j) The LB saisic is widely used. Bu, he Breusch Godfrey (1978) LM es condiions on X. Thus, i is more powerful. 9

10 Tesing for Auocorrelaion: Durbin-Wason The Durbin-Wason (1950) (DW) es for AR(1) auocorrelaion: H 0 0 agains H 1 0. Based on simple correlaions of e. d T ( e T 1 e 1 ) I is easy o show ha when T, d (1 - ). is esimaed by he sample correlaion r. e Under H 0 is 0. Then, d should be disribued randomly around. Small values of d lead o rejecion of H 0. The disribuion depends on X. Durbin-Wason derived bounds for he es. In he presence of lagged dependen variables, Durbin s (1970) h es should be used: h = r sqr{t/(1-t s )} GLS: The AR(1) Model (A1) holds: y = X + Bu, is no longer whie noise: = -1 + u, < 1. u is whie noise error ~D(0,σ u ) Noe: This characerizes he disurbances, no he regressors. Noaion: Le L be he lag operaor, such ha L q z =z -q. Then, (1 - L) = u. Afer some algebra, we ge = u + u -1 + u u = Σ j=0 j u -j = Σ j=0 (L) j u (a moving average) Var[ ] = Σ j=0 j Var[u -j ] = Σ j=0 j σ u = σ u /(1- ) 10

11 GLS: The AR(1) Model Afer some algebra, we ge Σ = σ. T 1 1 T 1 u T 3 Ω 1 1 T 1 T T 3 1 (Noe, race Ω = n as required.) GLS: The AR(1) Model Then, if we wan o esimae his model o gain efficiency, we can use he ransformaion marix P= -1/ : Ω 1/ 1/ Ω y= y1 y y y y y 3... T T 1 Problem: To use we need. We need o esimae i =>FGLS 11

12 FGLS: Eliminaing Auocorrelaion Le's coninue wih he firs-order auocorrelaion: y = X +, = -1 + u, < 1. Subrac y -1 = X Form pseudo-differences: y* = y - y -1 = (X - X -1 ) = X* + u =>Now, u is uncorrelaed. We can use OLS o esimae! Bu, we need o esimae (or know i, rare) o ransform he model and do GLS. An ieraive wo-sep algorihm seems naural: - Firs sep: Esimae - Second sep: Esimae GLS i.e., OLS in ransformed model. FGLS Esimaion: Cochrane-Orcu y - y -1 = (X - X -1 ) => y = y -1 + X - X -1 + u We have a linear model, bu i is nonlinear in parameers. This is no a problem: Non-linear esimaion is possible. Before oday s compuer power, Cochrane Orcu s (1949) ieraive procedure was an ingenious way o do NLLS. Seps: (1) Do OLS. Ge residuals, e. Then esimae wih a regression of e agains e -1. We use r o denoe he esimaor of. () FGLS Sep. Using r ransform he model o ge y* and X*. Do OLS o esimae. Ge residuals, e*. Go back o (1). (3) Ierae unil convergence. 1

13 FGLS Esimaion: Cochrane-Orcu If we do no wan o lose he firs observaion, we can use he Prais- Winsen (1945) ransformaion of he firs observaion: sqr{1- } y 1 & sqr{1- } X 1 A grid search around can speed up he algorihm considerably. This is he Hildreh-Lu (1960) procedure. The ieraive wo-sep esimaion procedure can be easily exended o AR(p) models. FGLS & MLE Esimaion We need o esimae => We need a model for = (θ). In he AR(1) model, we had = (). - FGLS esimaion is done using Cochrane-Orcu or NLLS. - MLE can also be done, say assuming a normal disribuion for u, o esimae and simulaneously. For he AR(1) problem, he MLE algorihm works like he Cochrane-Orcu algorihm. For an AR() model, Beach-Mackinnon (1978) propose an MLE algorihm ha is very fas o converge. For an AR(p) models, wih p > 3, MLE becomes complicaed. Two-sep esimaion is usually done. 13

14 Auocorrelaion as a Common Facor From he firs-order auocorrelaed model => y = y -1 + X - X -1 + u (*) We can generalize (*) using he lag operaor L i.e., L y = y -1 : (1- L) y = (1- L) X + u Then, dividing by (1- L): y = X + u /(1- L) = X + We can hink of a model wih auocorrelaion as a misspecified model. The common facor (1- L) is omied. See Mizon (1977). We can generalize (*) even more by inroducing more common lags: (1- B(L)) y = (1- B(L)) X + u B(L): funcion of L,L,...,L q ;. Common Facor Tes From he firs-order auocorrelaed model => y = y -1 + X - X -1 + u (*) We can hink of (*) as a special case of a more general specificaion: y = λ 1 y -1 + X λ + X -1 λ 3 + u Resricions needed o ge (*): λ 3 = - λ 1 λ We can es he validiy of he resricions. We can use a LR es: LR = T ln (RSS R RSS u ) χ d 1 The es is known as he common facor es. We can also use an F- es. 14

15 Building he Model Old (pre-lse school) view: A feaure of he daa Accoun for auocorrelaion in he daa. Differen models, differen esimaors Conemporary view: Why is here auocorrelaion? Wha is missing from he model? Build in appropriae dynamic srucures Auocorrelaion should be buil ou of he model Use robus procedures (Newey-Wes) insead of elaborae models specifically for he auocorrelaion. 15