How To Find Out If A Model Is Unbiased Or Not

Transcription

1 Econ 507. Econometric Analysis. Spring 2009 April 14, 2009

2 The Classical Linear Model: 1 Linearity: Y = Xβ + u. 2 Strict exogeneity: E(u) = 0 3 No Multicollinearity: ρ(x) = K. 4 No heteroskedasticity/ serial correlation: V (u) = σ 2 I n. Gauss/Markov Theorem: ˆβ = (X X) 1 X Y is best linear unbiased. ˆV ( ˆβ) = S 2 (X X) 1 is an unbiased estimate of V ( ˆβ) = σ 2 (X X) 1.

3 What happens if we drop the homoskedasticity assumption? ˆβ (the OLS estimator) is still linear and unbiased (Why?). Though linear and unbiased, ˆβ is not the minimum variance estimate (inefficient). ˆV ( ˆβ) = S 2 (X X) 1 is biased. This makes standard t and F tests invalid.

4 Intuition The presence of heteroskedastic errors should not alter the central position of the OLS line (unbiasedness). OLS weigths all observations equally, but in this case it makes more sense to pay more attention to observations where the variance is smaller.

5 The plan: what to do with heteroskedasticity. 1 Before abandoning OLS we will see how to test for heteroskedasticity. 2 Strategy 1: Propose another more efficient and unbiased estimator for β (weighted least squares (WLS)) and a suitable estimator for its variance. 3 Strategy 2: Keep using OLS (it is still unbiased, though inefficient), but find a replacement for its variance (the old one is biased under heteroskedasticity).

6 Testing for heteroscedasticity a) The White test H 0 : no heteroscedasticity, H A : there is heterocedasticity of some form. Consider a simple case with K = 3: Y i = β 1 + β 2 X 2i + β 3 X 3i + u i 1,..., n

7 Steps to implement the test: 1 Estimate by OLS, save squared residuals in e 2. 2 Regress e 2 on all variables, their squares and all possible non-redundant cross-products. In our case, regress e 2 on 1, X 2, X 3, X 2 2, X2 3, X 2X 3, and obtain R 2 in this auxiliar regression. 3 Under H 0, nr 2 χ 2 (p). p = number of explanatory variables in the auxiliar regression minus one. 4 Reject H o if nr 2 is too large.

8 Intuition: The auxiliar model can be seen as trying to model the variance of the error term. If the R 2 of this auxiliar regression were high, then we could explain the behavior of the squared residuals, providing evidence that they are not constant. Caveats: Valid for large samples. Informative if we do not reject the null (no heterocedasticity). When it rejects the null: there is heterocedasticity. But we do not have any information regarding what causes heterocedasticity. This will cause some trouble when trying to construct a GLS estimator, for which we need to know in a very specific way what causes heterocedasticity.

9 b) The Breusch-Pagan/Godfrey/Koenker test Mechanically very similar to White s test. Checks if certain variables cause heterocedasticity. Consider the following heteroscedastic model: Y = Xβ + u, u i normal, with E(u) = 0 and V (u i ) = h(α 1 + α 2 Z 2i + α 3 Z 3i α p Z pi ) where h( ) is any positive function with two derivatives. When α 2 =... = α p = 0, V (u i ) = h(α 1 ), a constant!! Then, homoscedasticity H 0 : α 2 = α 3 =... = α p = 0,and H A : α 2 0 α α p 0.

10 Steps to implement the test: 1 Estimate by OLS, and save squared residuals e 2 i. 2 Regresss e 2 i on the Z ik variables, k = 2,..., p and get (ESS). The test statistic is: 1 2 ESS χ2 (p 1) χ 2 (p) under H 0, asymptotically. We reject if it is too large.

11 Comments: Intuition is as in the White test (a model for the variance). By focusing on a particular group, if we reject the null we have a better idea of what causes heterocedasticity. Accepting the null does not mean there isn t heterocedasticity (why?). Also a large sample test. Koenker (1980) has proposed to use nra 2 as a test, which is still valid if errors are non-normal.

12 Estimation and inference under heteroscedasticity For simplicity, consider the two variable case Y i = β 1 + β 2 X i + u i where now the error term is heteroskedastic, that is V (u i ) = σi 2, i = 1,..., n We will assume all the other classical assumptions still hold

13 Divide each observation of the linear model by σ i : Y i 1 X i = β 1 + β 2 + u i σ i σ i σ i σ i Yi = β 1 X1i + β 2 X2i + β k Xki + u i Note that V (u i ) = V (u i/σ i ) = 1, then the residuals of this transformed model are homoscedastic. Then, if we know σi 2, the BLUE is simply the OLS estimator using the transformed variables.

14 In our case, the OLS with the transfored model is ˆβ 2,wls = n i=1 x i y i n i=1 x 2 i ˆβ 1,wls = Ȳ ˆβ 2,wls X with Yi = Y i /σ i, Xi = X i/σ i and lowercase letters are deviations from sample means, as usual. This is the weighted least squares estimator.

15 The name weighted least squares come from the fact that the estimator can be obtained by solving the following minimization problem n 1 min σ 2 e 2 i i=1 i that is, errors enter the SSR weighted by the inverse of the variance for each observation: we pay more attention to observations with smaller variance.

16 Problem: in practice we do not know σi 2. This leads to two strategies 1 Use WLS: Pros: estimates will be unbiased and efficient. Cons: we need to know the variances in advance (or make assumptions) 2 Keep OLS but change its variance estimator: Think again about the effects of heteroscedasticity on standard estimation procedures. OLS is still unbiased though not efficient (not that bad...). But, S 2 (X X) 1 is biased, which invalidates inference (this is bad!). Then, a second strategy: keeping OLS for β and look for a valid estimator for its variance. Pros: no assumptions needed, unbiased. Cons: we will lose efficiency with respect to the WLS case (if available).

17 1) Known variance structure: WLS Consider our simple two-variable case: Y i = β 1 + β 2 X i + u i Strategy: assume some particular forms of heteroscedasticity. a) V (u i ) = σ 2 Xi 2 σ 2 is an unknown constant. Divide all observations by X i Y i 1 = β 1 + β 2 + u i X i X i X i Y i = β 1 X 0i + β 2 + u i Note E(u i ) = E(u i/x i ) 2 = σ 2 X2 i = σ 2 Xi 2 Errors of the transformed model are homocedastic. Do OLS on the transformed model!

18 We do not need to know σ 2. What have just divided by the part of the standard error that varies over observations, that is, by X i. This strategy provides a WLS. Careful with interpretations. The intercept of the transformed model is the slope of the original model and that the slope of the transformed model is the intercept of the original one.

19 b) V (u) = σ 2 X i Y i 1 = β 1 + β 2 Xi + u i Xi Xi Xi Y i = β 1 X 0i + β 2 X 1i + u i As for implementation and interpretation, note that the transformed model has no intercept, and that the coefficient of the first explanatory variable corresponds to the intercept of the original model, and the coefficient of the second variable corresponds to the slope of the original model. Problem with these strategies: it is difficult to find an exact form for heterocedasticity

20 2) Unknown variance structure Alternative strategy: retain OLS (still unbiased though not efficient) and look for valid estimators for its variance. Variance matrix of ˆβ OLS under heteroscedasticity can be shown to be: Ω = diag(σ 2 1, σ2 2,..., σ2 n). V ( ˆβ OLS ) = (X X) 1 X ΩX(X X) 1

21 White (1980): a consistent estimator for X ΩX is X DX, D = diag(e 2 1, e2 2,..., e2 n), e i s OLS residuals Then, a heteroscedasticity consistent estimator of the variance matrix is: ˆV ( ˆβ OLS ) HC = (X X) 1 X DX(X X) 1 Strategy: use OLS but replace S 2 (X X) 1 by White s consistent estimator. This strategy is not efficient, but it does not require assumptions about the structure of heteroscedasticity.

22 Summary Heteroskedasticity makes OLS inefficient and invalidates the standard estimator of its variance, and hence invalidates inference (t tests, F tests, etc.). The WLS estimator is efficient and unbiased but it depends on knowing the variance structure. In practice is seldom available. In practice it is more common to keep OLS and replace its variance estimator by White s consistent method, which does not require any assumptions.