Lecture 2: ARMA(p,q) models (part 3)

Transcription

1 Lecture 2: ARMA(p,q) models (part 3) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept Jan Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

2 Motivation Introduction Characterize the main properties of ARMA(p,q) models. Estimation of ARMA(p,q) models Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

3 Road map Introduction 1 ARMA(1,1) model Definition and conditions Moments Estimation 2 ARMA(p,q) model Definition and conditions Moments Estimation 3 Application 4 Appendix Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

4 ARMA(1,1) model Definition and conditions 1. ARMA(1,1) 1.1. Definition and conditions Definition A stochastic process (X t ) t Z is said to be a mixture autoregressive moving average model of order 1, ARMA(1,1), if it satisfies the following equation : X t = µ + φx t 1 + ɛ t + θɛ t 1 t Φ(L)X t = µ + Θ(L)ɛ t where θ 0, θ 0, µ is a constant term, (ɛ t ) t Z is a weak white noise process with expectation zero and variance σ 2 ɛ (ɛ t WN(0, σ 2 ɛ )), Φ(L) = 1 φl and Θ(L) = 1 + θl. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

5 ARMA(1,1) model Definition and conditions Simulation of an ARMA(1,1) with (-0.5;-0.5) 4 Simulation of an ARMA(1,1) with (-0.5;0.5) Simulation of an ARMA(1,1) with (0.9;-0.5) 10 Simulation of an ARMA(1,1) with (0.9;0.5) Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

6 ARMA(1,1) model Definition and conditions The properties of an ARMA(1,1) process are a mixture of those of an AR(1) and MA(1) processes : The (stability) stationarity condition is the one of an AR(1) process (or ARMA(1,0) process) : φ < 1. The invertibility condition is the one of a MA(1) process (or ARMA(0,1) process) : θ < 1. The representation of an ARMA(1,1) process is fundamental or causal if : φ < 1 and θ < 1. The representation of an ARMA(1,1) process is said to be minimal and causal if : φ < 1, θ < 1 and φ θ. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

7 ARMA(1,1) model Definition and conditions If (X t ) is stable and thus weakly stationary, then (X t ) has an infinite moving average representation (MA( )) : X t = = µ 1 φ + (1 φl) 1 (1 + θl)ɛ t µ 1 φ + a k ɛ t k k=0 where : a 0 = 1 a k = φ k + θφ k 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

8 ARMA(1,1) model Definition and conditions If (X t ) is invertible, then (X t ) has an infinite autoregressive representation (AR( )) : i.e. (1 θ L) 1 (1 φl)x t = X t = = µ 1 θ + ɛ t µ 1 θ + b k X t k + ɛ t k=1 where θ = θ, and : b k = θ k θ k 1 φ. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

9 1.2. Moments ARMA(1,1) model Moments Definition Let (X t ) denote a stationary stochastic process that has a fundamental ARMA(1,1) representation, X t = µ + φx t 1 + ɛ t + θɛ t 1. Then : E [X t ] = Proof : See Appendix 1. µ 1 φ m γ X (0) 1 + 2φθ + θ2 V(X t ) = 1 φ 2 σɛ 2 γ X (1) (φ + θ)(1 + φθ) Cov [X t, X t 1 ] = 1 φ 2 σɛ 2 γ X (h) = φγ X (h 1) for h > 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

10 ARMA(1,1) model Moments Definition The autocorrelation function of an ARMA(1,1) process satisfies : 1 if h = 0 (φ+θ)(1+φθ) ρ X (h) = if h = 1 1+2φθ+θ 2 φρ X (h 1) if h > 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

11 ARMA(1,1) model Moments The autocorrelation function of an ARMA(1,1) process exhibits exponential decay towards zero : it does not cut off but gradually dies out as h increases. The autocorrelation function of an ARMA(1,1) process displays the shape of that of an AR(1) process for h > 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

12 ARMA(1,1) model Moments Partial Autocorrelation : The partial autocorrelation function of an ARMA(1,1) process will gradually die out (the same property as a moving average model). Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

13 Estimation ARMA(1,1) model Estimation Same techniques as before, especially those of MA models. Yule-Walker estimator : the extended Yule-Walker equations could be used in principe to estimate the AR coefficients but the MA coefficients need to be estimated by other means. In the presence of moving average components, the least squares estimator becomes nonlinear and the corresponding estimator is the conditional nonlinear least squares estimator (see estimation of MA(q) models). It has to be solved with numerical methods. Taking explicit distributional assumption for the error term, the conditional or exact maximum likelihood estimator can be computed (using also numerical or optimization methods). Other methods are also available : the Kalman filter, the generalized method of moments, etc. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

14 ARMA(1,1) model Estimation Estimation of ARMA(p,q) models (true DGP: µ = 0, φ = 0.9, and θ = 0.5) Coefficient Std. Error t-statistic p-value. Akaike info criterion Schwarz criterion ARMA(1,1) C AR(1) MA(1) ARMA(2,2) C AR(1) AR(2) MA(1) MA(2) ARMA(2,1) AR(1) AR(2) MA(1) AR(2) C AR(1) AR(2) AR(1) C AR(1) MA(2) C MA(1) MA(2) MA(4) C MA(1) MA(2) MA(3) MA(4) Note: C, AR(j), and MA(j) are respectively the estimate of the constant term, the jth autogressive term, and the jth moving average term. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

15 ARMA(p,q) model Definition and conditions 2. ARMA(p,q) 2.1. Definition and conditions Definition A stochastic process (X t ) t Z is said to be a mixture autoregressive moving average model of order p and q, ARMA(p,q), if it satisfies the following equation : X t = µ + φ 1 X t φ p X t p + ɛ t + θ 1 ɛ t θ q ɛ t q t Φ(L)X t = µ + Θ(L)ɛ t where θ q 0, φ p 0, µ is a constant term, (ɛ t ) t Z is a weak white noise process with expectation zero and variance σ 2 ɛ (ɛ t WN(0, σ 2 ɛ )), Φ(L) = 1 φ 1 L φ p L p and Θ(L) = 1 + θ 1 L + + θ q L q. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

16 ARMA(p,q) model Definition and conditions Main idea of ARMA(p,q) models Approximate Wold form of stationary time series by parsimonious parametric models AR and MA models can be cumbersome because one may need a high-order model with many parameters to adequately describe the data dynamics (see the effective Fed fund rate application) By mixing AR and MA models into a more compact form, the number of parameters is kept small... Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

17 ARMA(p,q) model Definition and conditions The properties of an ARMA(p,q) process are a mixture of those of an AR(p) and MA(q) processes : The (stability) stationarity conditions are those of an AR(p) process (or ARMA(p,0) process) : z p Φ(z 1 ) = 0 z p φ 1 z p 1 φ p = 0 z i < 1. for i = 1,, p. The invertibility conditions are those of an MA(q) process (or ARMA(0,q) process) : z q Θ(z 1 ) = 0 z q + θ 1 z q θ q = 0 z i < 1. for i = 1,, q. The representation of an ARMA(p,q) process is fundamental or causal if it is stable and invertible The representation of an ARMA(1,1) process is said to be minimal and causal if it is stable, invertible and the characteristic polynomials z p Φ(z 1 ) and z q Θ(z 1 ) have no common roots. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

18 ARMA(p,q) model Definition and conditions Definition The representation of a mixture autoregressive moving average process of order p and q defined by : X t = µ + φ 1X t φ px t p + ɛ t + θ 1ɛ t θ qɛ t q, is said to be a minimal causal (fundamental) representation (ɛ t) is the innovation process if : (i) All the roots of the characteristic equation associated to Φ z p φ 1z p 1 φ p = 0 are of modulus less than one, z i < 1 for i = 1,, p ; (ii) All the roots of the characteristic equation associated to Θ z q + θ 1z q θ q = 0 are of modulus less than one, z i < 1 for i = 1,, q ; (iii) The characteristic polynomials z p Φ(z 1 ) and z q Θ(z 1 ) have no common roots. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

19 ARMA(p,q) model Definition and conditions If (X t ) is stable and thus weakly stationary, then (X t ) has an infinite moving average representation (MA( )) : X t = = µ 1 p k=1 φ k µ 1 p k=1 φ k + Φ(L) 1 (1 + θl)ɛ t + a k ɛ t k k=0 where : a 0 = 1 a k < k=0 Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

20 ARMA(p,q) model Definition and conditions If (X t ) is invertible, then (X t ) has an infinite autoregressive representation (AR( )) : i.e. Θ(L) 1 Φ(L)X t = X t = = µ 1 q k=1 θ q µ 1 q k=1 θ k + + ɛ t b k X t k + ɛ t k=1 where θ k = θ k. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

21 ARMA(p,q) model Moments 2.2. Moments of an ARMA(p,q) The properties of the moments of an ARMA(p,q) are also a mixture of those of an AR(1) and MA(1) processes. The mean is the same as the one of an AR(p) model (with a constant term) : E(X t ) = µ 1 p k=1 φ k m. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

22 ARMA(p,q) model Moments Autocorrelation : The autocorrelation function of an ARMA(p,q) process exhibits exponential decay towards zero : it does not cut off but gradually dies out as h increases (possibly damped oscillations. The autocorrelation function of an ARMA(p,q) process displays the shape of that of an AR(p) process for h > max(p, q + 1). Partial Autocorrelation : The partial autocorrelation function of an ARMA(p,q) process will gradually die out (the same property as a MA(q) model). Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

23 2.3. Estimation ARMA(p,q) model Estimation Same techniques as in previous models... 1 Conditional least squares method 2 Maximum likelihood estimator (conditional or exact) 3 Generalized method of moments 4 Etc Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

24 Application 3. Application Effective Fed fund rate : 1970 : :01 (monthly observations) An ARMA(1,2) captures better the dynamics of the effective Fed fund rate. ML estimation of the effective Fed fund rate : ARMA(1,2) Coefficients Estimates Std. Error P-value µ φ θ θ Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

25 Application Effective Fed fund rate: ARMA(1,2) specification Residual Actual Fitted Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

26 Application Effective Fed fund rate: diagnostics of the ARMA(1,2) specification Autocorrelation Actual Theoretical Partial autocorrelation Actual Theoretical Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

27 Application Effective Fed fund rate: Impulse response function of the estimated ARMA(1,2) specification 1.0 Impulse Response ± 2 S.E Accumulated Response ± 2 S.E Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

28 4. Appendix Appendix 1. Moments of an ARMA(1,1). Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

29 Appendix 1. Moments of an ARMA(1,1) The properties of the moments of an ARMA(1,1) are a mixture of those of an AR(1) and MA(1) processes. The mean is the same as the one of an AR(1) model (with a constant term) : E(X t ) = E (µ + φx t 1 + ɛ t + θɛ t 1 ) = µ + φe(x t 1 ) + E(ɛ t ) + θe(ɛ t 1 ) = µ + φe(x t ) since E(X t ) = E(X t j ) for all j (stationarity property) and E(ɛ t j ) = 0 for all j(white noise). Therefore, E(X t ) = µ 1 φ m. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

30 Appendix Autocovariances Trick : Proceed in the same way and note that the Yule-Walker equations are the same as those of an AR(1) for h > 1. For h = 0 : γ X (0) = E [(X t m)(x t m)] = E [(φ(x t 1 m) + ɛ t + θɛ t 1) (X t m)] = φe [(X t m)(x t 1 m)] + E [ɛ t(x t m)] + θ E [ɛ t 1(X t m)] } {{ } 0 = φγ X (1) + σɛ 2 +θe [(φ(x t 1 m) + ɛ t + θɛ t 1) ɛ t 1] } {{ } AR(1) part = φγ X (1) + σ 2 ɛ + θφe [(X t 1 m)ɛ t 1] + θe [ɛ tɛ t 1] + θ 2 E = φγ X (1) + σ 2 ɛ (1 + θ(φ + θ)). [ ] ɛ 2 t 1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

31 Appendix For h = 1 : γ X (1) = E [(X t m)(x t 1 m)] = E [(φ(x t 1 m) + ɛ t + θɛ t 1 ) (X t 1 m)] = φe [(X t 1 m)(x t 1 m)] + E [ɛ t (X t 1 m)] +θe [ɛ t 1 (X t 1 m)] = φγ X (0) +θσɛ 2 } {{ } AR(1) part Solving for γ X (0) and γ X (1) : γ X (0) V(X t ) = γ X (1) Cov(X t, X t 1 ) = 1 + 2φθ + θ2 1 φ 2 σ 2 ɛ (φ + θ)(1 + φθ) 1 φ 2 σ 2 ɛ. Florian Pelgrin (HEC) Univariate time series Sept Jan / 32

32 Appendix For h > 1 : γ X (h) = E [(X t m)(x t h m)] = E [(φ(x t 1 m) + ɛ t + θɛ t 1 ) (X t h m)] = φe [(X t 1 m)(x t h m)] + E [ɛ t (X t h m)] +θe [ɛ t 1 (X t h m)] = φγ X (h 1) } {{ } AR(1) part since E [ɛ t j (X t h m)] = 0 for all h > j (ɛ t ) is the innovation process. The expression of the autocovariance of order h displays the same difference or recurrence equation as in an AR(1) model only the initial value γ X (1) changes! Florian Pelgrin (HEC) Univariate time series Sept Jan / 32