Model Specification: Precision and Bias

Transcription

1 Econ 507. Econometric Analysis. Spring 2009 April 1, 2009

2 The Classical Linear Model: 1 Linearity: Y = Xβ + u. 2 Strict exogeneity: E(u X) = 0 3 No Multicollinearity: ρ(x) = K, w.p.1. 4 No heteroskedasticity/ serial correlation: V (u X) = σ 2 I n. Gauss/Markov: ˆβ = (X X) 1 X Y is best linear unbiased.

3 Result: ˆβ is unbiased, that is E( ˆβ) = β. Proof ˆβ = (X X) 1 X Y = (X X) 1 X (Xβ + u) = (X X) 1 X Xβ + (X X) 1 X u = β + (X X) 1 X u E( ˆβ) = β + E ( (X X) 1 X u ) = β + (X X) 1 X E (u) = β Note the crucial role of the E(u) = 0 assumption.

4 Specification errors, bias and imprecision So far we have considered that our linear model Y = Xβ + u is correct Consider the following case Y = X 1 β 1 + X 2 β 2 + u where all classical assumptions hold K 1 and K 2 are the columns of X 1 and X 2. Trivially, our original model corresponds to X = [X 1 X 2 ], with K = K 1 + K 2.

5 Consider the following scenarios regarding β 2 and the corresponding estimation strategies: Omission of relevant variables: β 2 0, but we wrongly proceed as if β 2 = 0, that is, we regress Y on X 1 only. Inclusion of irrelevant variables: β 2 = 0, but we wrongly proceed as if β 2 might be 0, that is, regress Y on X 1 and X 2 when we could have ignored X 2.

6 Biases The Unbiasedness Property Let us compare results for the estimation of β 1 in the two scenarios I) Omission of relevant variables First note that in this case with u = X 2 β 2 + u. Let ˆβ 1 = (X 1 X 1) 1 X 1 Y. Y = X 1 β 1 + u This is the OLS estimator when we regress Y on X 1 only (omiting X 2.

7 Result: ˆβ 1 will be biased unless X 2 X 1 = 0. X 2 X 1 = 0 implies that X 2 and X 1 are uncorrelated. In words: the omission or relevant (β 2 0) variables leads to biases in the estimation, unless the omited variables are uncorrelated with the included variables.

8 Proof: ˆβ 1 = (X 1X 1 ) 1 X 1Y = (X 1X 1 ) 1 X 1(X 1 β 1 + X 2 β 2 + u) = β 1 + (X 1X 1 ) 1 X 1X 2 β 2 + (X 1X 1 ) 1 X 1u E( ˆβ 1 ) = β 1 + (X 1X 1 ) 1 X 1X 2 β 2 + (X 1X 1 ) 1 X 1E(u) = β 1 + (X 1X 1 ) 1 X 1X 2 β 2 This expression is equal to β 1 if X 1 X 2 = 0.

9 II) Inclusion of Irrelavant Variables In this case we would estimate β 1 jointly with β 2 by regressing Y on X 1 and X 2, that is, ˆβ 1 is a subvector of ˆβ = [ ˆβ1 ˆβ 2 ] = (X X) 1 X Y It is important to see that under the classical assumptions ˆβ and hence ˆβ 1 will be unbiased. Why?

10 Variances The Unbiasedness Property Result: the estimator that omits X 2 has smaller variance.

11 Bias-variance trade-off To summarize: In practice we do not know which model holds (the large one or the small one)? The trade-off: estimating a small model (omit variables) implies a gain in precision and a likely bias. A large model is less likely to be biased and will be more inefficient. Variable omission does not necessarily lead to biases.

12 Ommited Variable Bias: an example Computer generated data, but based on Appleton, French and Vanderpump ( Ignoring a Covariate: an Example of Simpon s Paradox, The American Statistician, 50, 4, 1996) Y = risk of death. SM OKE = consumption of cigarrettes.

13 . reg y smoke Source SS df MS Number of obs = F( 1, 98) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = y Coef. Std. Err. t P> t [95% Conf. Interval] smoke _cons

14 . reg y smoke age Source SS df MS Number of obs = F( 2, 97) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = y Coef. Std. Err. t P> t [95% Conf. Interval] smoke age _cons cor y smoke age (obs=100) y smoke age y smoke age