Estimation of Vector Autoregressive Processes

Transcription

1 Estimation of Vector Autoregressive Processes Based on Chapter 3 of book by H.Lütkepohl: New Introduction to Multiple Time Series Analysis Yordan Mahmudiev Pavol Majher December 13th, 2011 Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

2 Contents 1 Introduction 2 Multivariate Least Squares Estimation Asymptotic Properties of the LSE Example Small Sample Properties of the LSE 3 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is known Estimation when the process mean is unknown 4 The Yule-Walker Estimator 5 Maximum Likelihood Estimation The Likelihood Function The ML Estimators Properties of the ML Estimator Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

3 Introduction Introduction Basic Assumptions: y t = (y 1t,..., y Kt ) R K available time series y 1,..., y T, which is known to be generated by stationary, stable VAR(p) process where y t = ν + A 1 y t A p y t p + u t (1) ν = (ν 1,..., ν K ) is K 1 vector of intercept terms A i are K K coefficient matrices u t is white noise with nonsingular covariance matrix Σ u moreover p presample values for each variable, y p+1,..., y 0 are assumed to be available Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

4 Multivariate Least Squares Estimation Notation Y := (y 1,..., y T ) (K T ) B := (ν, A 1,..., A p ) (K (Kp + 1)) Z t := (1, y t, y t 1,..., y t p+1 ) ((Kp + 1) 1) Z := (Z 0,..., Z T 1 ) ((Kp + 1) T ) U := (u 1,..., u T ) (K T ) y := vec(y ) (KT 1) β := vec(b) ((K 2 p + K) 1) b := vec(b ) ((K 2 p + K) 1) u := vec(u) (KT 1) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

5 Multivariate Least Squares Estimation Estimation (1) using this notation, the VAR(p) model (1) can be written as Y = BZ + U, after application of vec operator and Kronecker product we obtain vec(y ) = vec(bz) + vec(u) = (Z I K )vec(b) + vec(u), which is equivalent to y = (Z I K )β + u note that covariance matrix of u is Σ u = I T Σ u Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

6 Multivariate Least Squares Estimation Estimation (2) multivariate LS estimation (or GLS estimation) of β minimizes note that S(β) = u (I T Σ u ) 1 u = = [ y (Z I K )β ] (IT Σ 1 u ) [ y (Z I K )β ] S(β) = y (I T Σ 1 u the first order conditions S(β) β )y + β (ZZ Σ 1 u )β 2β (Z Σ 1 u )y = 2(ZZ Σ 1 u )β 2(Z Σ 1 u )y = 0 after simple algebraic exercise yield the LS estimator ˆβ = ((ZZ ) 1 Z I K )y Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

7 Multivariate Least Squares Estimation Estimation (3) the Hessian of S(β) 2 S β β = 2(ZZ Σ 1 u ) is positive definite ˆβ is minimizing vector the LS estimator can be written in differen ways ˆβ = β + ((ZZ ) 1 Z I K )u = = vec(yz (ZZ ) 1 ) another possible representation is ˆb = (I K (ZZ ) 1 Z)vec(Y ), where we can see that multivariate LS estimation is equivalent to OLS estimation of each of the K equations of (1) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

8 Multivariate Least Squares Estimation Asymptotic Properties of the LSE Asymptotic Properties (1) Definition A white noise process u t = (u 1 t,..., u K t) is called standard white noise if the u t are continuous random vectors satisfying E(u t ) = 0, Σ u = E(u t u t ) is nonsingular, u t and u s are independent for s t and E u it u jt u kt u mt c for i, j, k, m = 1,..., K, and all t for some finite constant c. we need this property as a sufficient condition for the following results: Γ := plim ZZ 1 T T t=1 T exists and is nonsingular vec(u t Z t 1) = 1 d (Z I K )u N (0, Γ Σ u) T T Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

9 Multivariate Least Squares Estimation Asymptotic Properties of the LSE Asymptotic Properties (2) the above conditions provide for consistency and asymptotic normality of the LS estimator Proposition Asymptotic Properties of the LS Estimator Let y t be a stable, K-dimensional VAR(p) process with standard white noise residuals, ˆB is the LS estimator of the VAR coefficients B. Then ˆB p B T and. T ( ˆβ β) = T vec(ˆb B) d T N (0, Γ 1 Σ u ) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

10 Multivariate Least Squares Estimation Asymptotic Properties of the LSE Asymptotic Properties (3) Proposition Asymptotic Properties of the White Noise Covariance Matrix Estimators Let y t be a stable, K-dimensional VAR(p) process with standard white noise residuals and let B be an estimator of the VAR coefficients B so that T vec( B B) converges in distribution. Furthermore suppose that where c is a fixed constant. Then Σ u = (Y BZ)(Y BZ), T c T ( Σ u UU /T ) p T 0. Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

11 Multivariate Least Squares Estimation Example Example (1) three-dimensional system, data for Western Germany ( ) fixed investment y 1 disposable income y 2 consumption expenditures y 3 Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

12 Multivariate Least Squares Estimation Example Example (2) assumption: data generated by VAR(2) process LS estimates are the following stability of estimated process is satisfied, since all roots of the polynomial det(i 3  1 z  2 z 2 ) have modulus greater than 1 Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

13 Multivariate Least Squares Estimation Example Example (3) we can calculate the matrix of t-ratios these quantities can be compared with critical values from a t-distribution d.f. = KT K 2 p K = 198 or d.f. = T Kp 1 = 66 for a two-tailed test with significance level 5% we get critical values of approximately ±2 in both cases apparently several coefficients are not significant model contains unnecessarily many free parameters Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

14 Multivariate Least Squares Estimation Small Sample Properties of the LSE Small Sample Properties difficult do analytically derive small sample properties of LS estimation numerical experiments are used, such as Monte Carlo method example process ( ) ( ) ( ) y t = + y t 1 + y t 2 + u t ( ) 9 0 Σ u = time series generated of length T = 30 (plus 2 presample values) u t N (0, Σ u ) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

15 Multivariate Least Squares Estimation Small Sample Properties of the LSE Empirical Results Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

16 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is known Process with Known Mean (1) The process mean µ is known The mean-adjusted VAR(p) is given by (y t µ) = A 1 (y t 1 µ) A p (y t p µ) + u t One can use LS estimation by defining: Y 0 (y t µ,..., y T µ) A (A 1,..., A p ) y t µ Yt 0 ( ). X Y0 0,..., Y T 0 1 y t p+1 µ Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

17 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is known Process with Known Mean (2) y 0 vec(y 0 ) α vec(a) Then the mean-adjusted VAR(p) can be rewritten as: Y 0 = AX + U or y 0 = (X I K ) α + u where u is defined as before. The LS estimator is: ) ˆα = ((XX ) 1 X I K y 0 or  = Y 0 X (XX ) 1 If y t is stable and u t is white noise, it follows that d T (ˆα α) N (0, Σˆα ) where ˆα = Γ Y (0) 1 ( Σ u with ( ) Γ Y (0) E Y 0 t ). Y 0 t Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

18 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is unknown Process with Unknown Mean (1) Usually the process mean is not known and we have to estimated it: ȳ = 1 T T y t t=1 Plugging in for each y t expressed from (y t µ) = A 1 (y t 1 µ) A p (y t p µ) + u t and rewriting gives: Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

19 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is unknown Process with Unknown Mean (2) ] ȳ = µ + A 1 [ȳ + 1 T (y 0 y T ) µ ] A p [ȳ + 1 T (y p y 0 y T p+1... y T ) µ + 1 T T u t t=1 The exact meaning of elements such as y p+1 for p>1 is unclear (presample observations). Equivalently: (I K A 1... A p ) (ȳ µ) = 1 T z T + 1 T u t T t=1 [ ] where z T = p i 1 A i (y 0 j y T j ) i=1 j=1 Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

20 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is unknown Process with Unknown Mean (3) ( z T / ) T Obviously E T E (z T ) = 0 ( Moreover, as y t is stable var z T / ) T = 1 z T / T converges to zero in mean square and = 1 T Var (z T ) T 0 (I K A 1... A p ) (ȳ µ) has the same asymptotic distribution 1 T as T u t t=1 By the CLT 1 T d T u t N (0, Σu ) t=1 Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

21 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is unknown Process with Unknown Mean (4) therefore, if y t is stable and u t is white noise: T (ȳ µ) d N (0, Σȳ) with Σȳ = (I K A 1... A p ) 1 Σ u (I K A 1... A p ) -1 another way of estimating the mean is obtained from the LS estimator: ˆµ = ( I K Â1... Âp) 1 ˆν these two ways are asymptotically equivalent Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

22 Least Squares Estimation with Mean-Adjusted Data Estimation when the process mean is unknown Process with Unknown Mean (5) Replacing µ with ȳ in the vectors and matrices from before, e.g. Ŷ 0 (y t ȳ,..., y T ȳ) gives the corresponding LS estimator: ( ( ˆα = X X ) ) 1 X I K ^y 0 This estimator is asymptotically equivalent to LS estimator for a process with known mean ˆα T (ˆα α) d N (0, Γ Y (0) 1 Σ u ) ( with Γ Y (0) E Y 0 t ( Y 0 t ) ) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

23 The Yule-Walker Estimator The Yule-Walker Estimator (1) Recall from the lecture slides that for VAR(1) it holds: A 1 = Γ y (0)Γ y (1) 1 and in general Γ y (h) = A 1 Γ y (h 1) = A h 1 Γ y(0) Γ y (h 1) Extending to VAR(p): Γ y (h) = [A 1,..., A p ]. or: Γ y (h p) Γ y (0)... Γ y (p 1) [Γ y (1),..., Γ y (p)] = [A 1,..., A p ]..... Γ y ( p + 1)... Γ y (0) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

24 The Yule-Walker Estimator The Yule-Walker Estimator (2) [Γ y (1),..., Γ y (p)] = AΓ Y (0) Hence, A = [Γ y (1),..., Γ y (p)] Γ Y (0) 1 If p presample observations are available, the mean µ can be estimated by: ȳ = 1 T y t T + p t= p+1 Then ˆΓ y (h) = 1 T T +p h t= p+h+1 (y t ȳ ) (y t h ȳ ) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

25 The Yule-Walker Estimator The Yule-Walker Estimator (3) The Yule-Walker Estimator has the same asymptotic properties as the LS estimator for stable VAR processes. However, it could be less attractive for small samples. The following example shows that asymptotically equivalent estimators can give different results for small samples (here T=73) ( ) ȳ = ˆµ = I 3  1  2 ˆν = Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

26 The Yule-Walker Estimator The Yule-Walker Estimator (4) ) Â = (Â1, Â2 = Â YW = Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

27 Maximum Likelihood Estimation The Likelihood Function Maximum Likelihood Estimation Assume that the VAR(p) is Gaussian, i.e. u =vec (U) = u 1. u T N (0, I T Σ u ) The probability density of u is 1 f u (u) = I (2π) KT /2 T [ ( u 1/2 exp 1 2 u I T 1 u ) ] u Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

28 Maximum Likelihood Estimation The Likelihood Function The Log-Likelihood Function From the probability density of u, a probability density for y vec (Y ), f y (y), can be derived After some modification the log-likelihood function is given by: ln l (µ, α, u ) = KT 2 ln2π T 2 ln u 1 2 tr [ (Y 0 AX ) 1 u From ln(l) µ, ln(l) α, and ln(l) u which can be solved for the estimators ( Y 0 AX )] we get the system of normal equations, Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

29 Maximum Likelihood Estimation The ML Estimators The ML Estimators The three ML Estimators: µ = 1 ( ) 1 p I K à i T α = i=1 T ( ) p y t à i y t i t=1 i=1 ( ( X X ) 1 X I K ) (y µ ) Σ u = 1 T (Ỹ 0 à X ) (Ỹ 0 à X ) Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

30 Maximum Likelihood Estimation Properties of the ML Estimator Properties of the ML Estimator (1) The estimators are asymptotically consistent and asymptotically normal distributed: µ µ Σ µ 0 0 T α α d N 0, 0 Σ α 0 σ σ 0 0 Σ σ ) ( where σ = vech ( Σ u and Σ µ = I K p ) 1 ( Ã i u I K p i=1 Ã i i=1 ) 1 Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

31 Maximum Likelihood Estimation Properties of the ML Estimator Properties of the ML Estimator (2) Σ α = Γ Y (0) 1 Σ u Σ σ = 2D + K (Σ u Σ u ) ( ) D K + where D K is given by vec (Σ u )=D K vech (Σ u ) and D K + Moore-Penrose generalized inverse. σ 11 σ 11 σ 12 σ σ σ = vech (Σ u ) = vech σ 21 σ 22 σ 23 = σ 31 σ σ 31 σ 32 σ σ 32 σ 33 is the Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32

32 Maximum Likelihood Estimation Properties of the ML Estimator Thank you for your attention! Yordan Mahmudiev, Pavol Majher () Estimation of VAR Processes December 13th, / 32