7. Autocorrelation (Violation of Assumption #B3)

Transcription

1 7. Autocorrelation (Violation of Assumption #B3) Assumption #B3: The error term u i is not autocorrelated, i.e. Cov(u i, u j ) = 0 for all i = 1,..., N and j = 1,..., N where i j Where do we typically find autocorrelation: Time-series data sets (less frequently in cross-sectional data sets) 169

2 Outlook: Strong similarities between heteroskedasticity and autocorrelation with respect to consequences and estimation procedures (GLS, FGLS estimators) Example: [I] Estimation of a price-revenue function (monthly data) Variables: y i = monthly revenue quantity (in 1000 pieces) x i = selling price (in euros) 170

3 Month Obs. Price Revenue Month Obs. Price Revenue 01: : : : : : : : : : : : : : : : : : : : : : : : Dependent Variable: REVENUE Method: Least Squares Date: 11/17/04 Time: 13:50 Sample: 2002: :12 Included observations: 24 Variable Coefficient Std. Error t-statistic Prob. C PRICE R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion Log likelihood F-statistic Durbin-Watson stat Prob(F-statistic)

4 Example: [II] Linear model with a single regressor: y i = α + β x i + u i (i = 1,..., 24) Price increase of 1 euro decreases the monthly revenue by pieces 172

5 Revenue Price

6 Evidently: Line connecting the residuals rarely crosses the regression line we frequently find that positive u i 1 -values are followed by positive u i -values negative u i 1 -values are followed by negative u i -values Cov(u i 1, u i ) 0 (violation of Assumption #B3) Question: Impact on estimation and testing procedures 174

7 7.1 Consequences Note: We assume an explicit pattern of autocorrelation (alternative patterns are not considered here) Definition 7.1: (AR(1) process) Let u 1,..., u N be the error terms of the linear regression model. Furthermore, let ρ R be a (constant) parameter and let e 1,..., e N denote additional error terms that satisfy all B-assumptions (#B1 #B4). If u i = ρu i 1 + e i, (i = 2,..., N) we say that the error term u i follows a first-order autoregressive process (in symbols: u i AR(1)). 175

8 Remarks: An AR(1) process regresses u i on its predecessor value u i 1 plus the new random shock e i For ρ = 1 or ρ = 1 we have so-called random walks (important stochastic processes) For ρ > 1 processes become explosive in this lecture: 1 < ρ < 1 Now: Expected values, (co)variances, correlation coefficients of an AR(1) process 176

9 Theorem 7.2: (Moments of an AR(1) process) Let the error term u i (i = 1,..., N) follow an AR(1) process according to Definition 7.1 where 1 < ρ < 1. Furthermore, let Var(e i ) σ 2 e denote the constant variance of all e i. We then have for all (admissible) i = 1,..., N: (Proof: class) E(u i ) = 0, Var(u i ) = σ2 e 1 ρ 2 σ2, Cov(u i, u i τ ) = ρ τ σ 2 e 1 ρ 2 = ρτ σ 2 0, Corr(u i, u i τ ) = ρ τ. 177

10 Obviously: If the error term u i follows an AR(1) process with 1 < ρ < 1, then the Assumptions #B1, #B2 are satisfied whereas #B3 is violated Now: Autocorrelation in matrix notation (u i s follow an AR(1) process) Notation: u = u 1 u 2. u N, u 1 = u 0 u 1. u N 1, e = e 1 e 2. e N 178

11 Matrix representation: [I] Linear regression model y = Xβ + u with AR(1) error terms ( 1 < ρ < 1) u = ρu 1 + e Theorem 7.2 yields Cov(u) 179

12 Matrix representation: [II] Due to σ 2 σ 2 e /(1 ρ 2 ) we obtain Cov(u) = σ 2 Cov(u 1, u 2 ) Cov(u 1, u N ) Cov(u 2, u 1 ) σ 2 Cov(u 2, u N )... Cov(u N, u 1 ) Cov(u N, u 2 ) σ 2 = σ 2 ρσ 2 ρ N 1 σ 2 ρσ 2 σ 2 ρ N 2 σ 2... ρ N 1 σ 2 ρ N 2 σ 2 σ 2 = σ 2 Ω 180

13 Matrix representation: [II] where Ω = 1 ρ ρ N 1 ρ 1 ρ N 2... ρ N 1 ρ N

14 Question: Is there any transformation of the autocorrelated model so that the parameter vector β remains unchanged autocorrelation vanishes the transformed model y = X β + u satisfies all #A-, #B-, #C- assumptions? (cf. Section 6, Slide 111) 182

15 Hope: If yes, then the OLS estimator of the transformed model (the GLS estimator) would be BLUE (cf. Section 6, Slides ) Result: In analogy to the line of argument given on Slides under heteroskedasticity the following result obtains: there exists a regular matrix P so that the transformed model Py = PXβ + Pu satisfies all #A-, #B-, #C-assumptions 183

16 Form of P in the autocorrelated model: [I] P has to satisfy the following equations: P P = Ω 1 and PΩP = I N (see Slides 117, 120) First, the inverse of Ω from Slide 181 is given by Ω 1 = 1 1 ρ 2 (check it) 1 ρ ρ 1 + ρ 2 ρ ρ. 1 + ρ ρ 2 ρ ρ 1 184

17 Form of P in the autocorrelated model: [II] The form of P is given by P = 1 1 ρ 2 1 ρ ρ ρ ρ 1 185

18 Form of P in the autocorrelated model: [III] transformed model: y = X β + u where y = Py = X = PX = 1 ρ 2 u 1 e 2. 1 ρ 2 y 1 y 2 ρy 1, u = Pu =. y N ρy N 1 e N 1 ρ 2 1 ρ 2 x 11 1 ρ 2 x K1 1 ρ. x 12 ρx 11.. x K2 ρx K1. 1 ρ x 1N ρx 1(N 1) x KN ρx K(N 1) 186

19 Remarks: The transformed model y = X β + u satisfies all #A-, #B-, #C-assumptions The parameter vector β remains unchanged consequences of autocorrelation parallel those of heteroskedasticity 187

20 Consequences of autocorrelation: [I] The OLS estimator β = ( X X ) 1 X y is still unbiased, but no longer BLUE (cf. Theorem 6.1, Slide 109) The covariance matrix of the OLS estimator is given by Cov ( β ) = σ 2 ( X X ) 1 X ΩX ( X X ) 1 The GLS estimator is BLUE β GLS = [ X X ] 1 X y = [ X Ω 1 X ] 1 X Ω 1 y 188

21 Consequences of autocorrelation: [II] Its covariance matrix is given by ( Cov β GLS) = σ 2 [ X Ω 1 X ] 1 (cf. Theorem 6.3, Slide 123) Unbiased estimator of σ 2 : ˆσ 2 = û û N K 1 = (Pû) Pû N K 1 189

22 Impact of neglecting autocorrelation: [I] OLS estimator of β is unbiased, but inefficient β = ( X X ) 1 X y The estimator ˆσ 2 (X X) 1 of the covariance matrix Cov ( β ) is biased The estimator ˆσ 2 û û = N K 1 of the error-term variance is biased 190

23 Impact of neglecting autocorrelation: [II] test statistics are based on biased estimators hypothesis tests are likely to be unreliable (t-, F -tests) 191

24 7.2 Diagnostics Graphical analysis: [I] First, estimation of the model y = Xβ + u by OLS, i.e. β = ( X X ) 1 X y calculation of the residuals û = y X β 192

25 Graphical analysis: [II] Plot of the residuals versus time slow swings around zero positive autocorrelation fast swings around zero negative autocorrelation Scatterplot of û i 1 versus û i positive slope positive autocorrelation negative slope negative autocorrelation Example: [I] Price-revenue function on Slides

26 Dependent Variable: Revenue Method: Least Squares Date: 11/17/04 Time: 13:50 Sample: 2002: :12 Included observations: 24 Variable Coefficient Std. Error t-statistic Prob. C PRICE R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion Log likelihood F-statistic Durbin-Watson stat Prob(F-statistic) OBS RESID(-1) RESID

27 Residuals :01 02:04 02:07 02:10 03:01 03:04 03:07 03:10 i = 1,..., u i u i 1

28 Obviously: Positive dependence between û i 1 and û i indication of positive autocorrelation a conceivable specification of an AR(1) error-term process could be with 0 < ρ < 1 u i = ρu i 1 + e i 196

29 Now: Use the pair of residuals (û i 1, û i ) to estimate ρ Model û i = ρû i 1 + e i (i = 2,..., N) OLS estimator of ρ: ˆρ = N i=2 (ûi 1 û ) ( û i û ) N (ûi 1 û ) 2 = N i=2 N û i 1 û i i=2 û 2 i 1 i=2 197

30 Remarks: For the sum of the residuals computed via OLS we always have N i=1 û i = 0 and thus û = 1 N (cf. Von Auer, 2007, p. 57) N i=1 û i = 0 Since for i = 1 there is no residual û i 1 = û 0, we only have i = 2,..., N observations 198

31 OLS estimate of ρ in the price-revenue example: ˆρ = Dependent Variable: RESID Method: Least Squares Date: 11/20/04 Time: 19:15 Sample(adjusted): 2002: :12 Included observations: 23 after adjusting endpoints Variable Coefficient Std. Error t-statistic Prob. RESID(-1) R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion Log likelihood Durbin-Watson stat Question: Is ρ significantly different from zero? Durbin-Watson test for autocorrelation 199

32 Durbin-Watson test: [I] Most popular test for autocorrelation (due to Durbin & Watson, 1950, 1951) Tests for both, positive (ρ > 0) and negative autocorrelation (ρ < 0) Test statistic: calculate the residuals û i via OLS test statistic: DW = N i=2 (ûi û i 1 ) 2 N / û 2 i i=1 200

33 Durbin-Watson test: [II] Relation to the OLS estimator ˆρ from Slide 197: DW 2(1 ˆρ) Properties: Since 1 < ρ < 1 it follows that 0 < DW < 4 no autocorrelation: ˆρ 0 DW 2 positive autocorrelation: ˆρ 1 DW 0 negative autocorrelation: ˆρ 1 DW 4 201

34 Durbin-Watson test: [III] Test for positive autocorrelation: hypotheses: H 0 : ρ 0 versus H 1 : ρ > 0 distribution of DW under H 0 depends on sampling size (N) number of exogenous regressors (K) specific values of the regressors x 1i,..., x Ki exact calculation by econometric software 202

35 Durbin-Watson test: [IV] distribution under H 0 of DW has lower and upper bounds exact critical values at the α-level (d α ) have lower and upper bounds (i.e. d L α d α d U α) (for α = 0.05 see Von Auer, 2007, p. 402) explicit decision rule: reject H 0 : ρ 0 if DW < d L α do not reject H 0 : ρ 0 if DW > d U α no decision if d L α DW d U α 203

36 Durbin-Watson test: [V] Test for negative autocorrelation: hypotheses: H 0 : ρ 0 versus H 1 : ρ < 0 explicit decision rule: Reject H 0 : ρ 0 if DW > 4 d L α Do not reject H 0 : ρ 0 if DW < 4 d U α No decision if 4 d U α DW 4 dl α 204

37 Example: Estimation of the price-revenue function (Slides 170, 171) Test for positive autocorrelation at the 5% level: H 0 : ρ 0 versus H 1 : ρ > 0 We have N = 24, K = 1, DW = , d0.05 L = 1.27, du 0.05 = 1.45 and thus DW = < 1.27 = d0.05 L reject H 0 at the 5%-level 205

38 Drawbacks of the Durbin-Watson test: Frequently there is no decision (e.g. if DW [d L α, d U α]) when testing for pos. autocorrelation) DW-Test is unreliable if predecessor values like y i 1, y i 2,... are used as regressors (so-called lag-models) DW-test only tests for AR(1)-autocorrelation 206

39 7.3 Feasible Estimation Procedures Now: Estimation of the autocorrelated model y = Xβ + u with AR(1) error-terms u = ρu 1 + e ( 1 < ρ < 1) 207

40 Problem: From the data set (X, y) we do not have direct knowledge about the autocorrelation parameter ρ FGLS estimation Two feasible estimation procedures: GLS approach (Hildreth & Lu) FGLS approach (Cochrane & Orcutt) 208

41 1. Method by Hildreth & Lu: [I] Search algorithm Consider the GLS estimator β GLS = [ X Ω 1 X ] 1 X Ω 1 y where Ω 1 = 1 1 ρ 2 (cf. Slides 184, 188) 1 ρ ρ 1 + ρ 2 ρ ρ. 1 + ρ ρ 2 ρ ρ 1 209

42 1. Method by Hildreth & Lu: [II] Perform GLS estimation for distinct ρ-values ( 1 < ρ < 1) compute the sum of squared residuals û û for each estimation Find the ρ-value with the minimal sum of squared residuals GLS estimator of β associated with this ρ-value is called Hildreth-Lu estimator 210

43 Example: Data on price-revenue function ρ u * ' u * α β Hildreth-Lu estimates: ˆα HL = , ˆβ HL =

44 2. Method by Cochrane & Orcutt: [I] Iterative multi-step procedure Procedure: 1. OLS estimation of the model y = Xβ + u 2. Save the residuals û = y X β 212

45 2. Method by Cochrane & Orcutt: [II] 3. Consider the resgression û = ρû 1 + e and estimate ρ by the OLS estimator ˆρ = N û i 1 û i i=2 N û 2 i 1 i=2 4. Use ˆρ to apply the FGLS estimator [ β FGLS = X Ω 1 1 X] X Ω 1 y 213

46 2. Method by Cochrane & Orcutt: [III] 5. Improvement due to iteration: Compute the new residuals û (2) FGLS = y X β Re-estimate ρ as in Step #3 Find the new FGLS estimator β FGLS(2) Repeat Step #5 until the FGLS estimator of β does not exhibit any further (substantial) change 214

47 Example: [I] Consider the price-revenue example Estimate OLS estimate Iteration #1 Iteration #2 Iteration #3 ρ α β Cochrane-Orcutt estimates: ˆα CO = , ˆβ CO = (possibly further iterations) 215

48 Example: [II] Contrasting both estimation results: Parameter OLS estimate Hildreth-Lu Cochrane-Orcutt