HH Chapter 9. Nov 3, 2005

Transcription

1 Nov 3, 2005

2 Topics of Output Variable Selection

3 Data Scatter Plot Added Variable Plots hh/datasets/usair.dat Response SO2 measurements in 41 metropolitan areas Predictors temp firms popn wind precip rain Model for log(so2) as a function of temp, log(firms), log(popn), wind, precip, rain.

4 Scatterplot Matrix Scatter Plot Added Variable Plots log(so2) temp log(firms) log(popn) wind precip rain

5 Correlations Scatter Plot Added Variable Plots lso2 temp lfirms lpopn wind precip rain lso temp lfirm lpopn wind precip rain

6 av.plots Added Variable Plot Added Variable Plot Added Variable Plot Scatter Plot Added Variable Plots log(so2) others temp others log(so2) others log(firms) others log(so2) others log(popn) others Added Variable Plot Added Variable Plot Added Variable Plot log(so2) others log(so2) others log(so2) others wind others precip others rain others

7 summary(poll.lm3) (abbreviated) Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values Signif. codes: 0 *** ** 0.01 * Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) *** temp ** log(firms) log(popn) wind * precip rain Residual standard error: on 34 degrees of freed R-Squared: , Adjusted R-squared: F-statistic: on 6 and 34 DF, p-value: 7.12e-05

8 Residual Standard Error Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values Residual standard error: on 34 degrees of freedom Residuals e i = Y i Ŷ i Unbiased estimate of σ 2 : ˆσ 2 = ei 2 /(n p 1) Sum of squared residuals over degrees of freedom (n - # estimated coefficients in regression) residual standard error is ˆσ

9 F statistic Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values F-statistic: on 6 and 34 DF, p-value: 7.12e-05 Null Hypothesis: β 1 = β 2 =... = β p = 0 Alternative Hypothesis: at least one coefficient is not zero test statistic: (Ŷi Ȳ ) 2 /p F = ˆσ 2 distribution: F with p and n p 1 degrees of freedom under the null hypothesis

10 R2 Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values Coefficient of Determination or R-Squared: R 2 (Yi Ŷ i ) 2 = 1 (Yi Ȳ )2 Unexplained Variation = 1 Total Variation = Fraction of variation explained by the regression 55% of the variation in log(so2) can be explained by the linear model

11 Adjusted R-squared Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values Adjusted R-squared: R2 increases as we add more and more predictors R2 = 1 with p = n (even if the predictors are unrelated to Y!) R2 is only useful to compare models with the SAME response and the SAME number of predictors To take into account model complexity Adjusted R2 = 1 ˆσ2 S 2 y

12 Coefficients Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values For a 1 unit increase in X j, expect Y to increase by ˆβ j (with everything else held constant) (1 α)100%ci: ˆβ j ± t α/2,n p 1 SE(ˆβ j ) Need to be able to interpret after transformation back to original units of SO2 ŜO2 = e ˆβ 0 e ˆβ 1 Temp e ˆβ 2 log firm e ˆβ 3 log popn e ˆβ 4 wind e ˆβ 5 precip e ˆβ 6 rain ŜO2 = e ˆβ 0 e ˆβ 1 Temp firmˆβ 2 popnˆβ 3 e ˆβ 4 wind e ˆβ 5 precip e ˆβ 6 rain

13 Temperature Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values a 1 unit increase in temperature: exp( ) =.94 95% interval (0.89, 0.98) (exp(ci))

14 Number of Firms Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values a 2 fold increase (doubling) of number of firms: = % interval (0.98, 1.70) (2 CI )

15 t-statistics Residual Standard Error F statistic R2 and adjusted R-squared coefficients and their standard errors (in original units) t-statistics & p-values Null hypothesis: β j = 0 Alternative hypothesis β j 0 test statistic ˆβ j /SE(ˆβ j ) Distribution: under null, Student-t with n - p - 1 degrees of freedom Conclusion: reject null of p-value α Does not allow simultaneous tests of two or more coefficients only one coefficient at a time!

16 Signs of VIF coefficient of popn in multiple regessions has a negative sign coefficient of popn in simple linear regression (correlation) has a positive sign! Coefficient is adjusted for other variables so sign change may be meaningful Maybe a symptom of multicollinearity

17 Variance Inflation Factor VIF vif(log(so2)~temp + log(firms) + log(popn) + wind + precip + rain, data=pollution) temp lfirm lpopn wind precip rain VIF(X j ) = 1/(1 R 2 j ) (R2 j is percent of variation in X j that can be explained by the other X s) Values greater than 5 indicate multicollinearity (redundancy and instability) Variance of coefficient is inflated by VIF over variances with no multicollinearity present (SE is inflated by square root of VIF)

18 Problems with VIF Variables may appear to be unimportant (when they are) (large VIF may lead to larger SE s and smaller t statistics) Coefficient estimates are unstable and hard to interpret (can estimate combinations of coefficients but not individual coefficients)

19 Dealing with VIF Collect more data to reduce correlation among teh explanatory variables Keep all variables and drop interpretation of coefficients (focus on prediction) Eliminate redundant variables (subjective) Automatic Variable selection Ridge or other Shrinkage estimates