3. Hypothesis tests and confidence intervals in multiple regression

Transcription

1 3. Hypothesis tests and confidence intervals in multiple regression Contents of previous section: Definition of the multiple regression model OLS estimation of the coefficients Measures-of-fit (based on estimation results) Some problems in the regression model (omitted-variable bias, multicollinearity) Now: Statistical inference based on OLS estimation (hypothesis tests, confidence intervals) 39

2 3.1. Standard errors for the OLS estimators Recall: OLS estimators are subject to sampling uncertainty Given the OLS assumptions on Slide 18 the OLS estimators are normally distributed in large samples, that is ˆβ j N(β j, σ 2ˆβ j ) for j = 0,..., k Now: How can we estimate the (unknown) OLS estimator s variance σ 2ˆβ and its standard deviation σ 2ˆβ j j σˆβ j 40

3 Definition 3.1: (Standard error) We call an appropriately defined estimator of the standard deviation σˆβ j the standard error of ˆβ j and denote it by SE(ˆβ j ). Natural question: What constitutes a good estimator of σˆβ j? Answer: The analytical formula of a good estimator crucially hinges on whether the errors u i are homoskedastic or heteroskedastic (see Definition 2.2 on Slide 8) 41

4 Homoskedasticity / heteroskedasticity Important notes: The way we defined the terms homoskedasticity and heteroskedasticity in Definition 2.2 on Slide 8 implies that homoskedasticity is a special case of heteroskedasticity ( heteroskedasticity is more general than homoskedasticity ) Since the OLS assumptions on Slide 18 place no restrictions on the conditional variance of the error terms u i, they apply to both the general case of heteroskedasticity and the special case of homoskedasticity Theorem 2.4 on Slide 19 is valid under both concepts 42

5 Corollary 3.2: (To Theorem 2.4, Slide 19) Given the OLS assumptions on Slide 18, the OLS estimators are unbiased, consistent, and normally distributed in large samples (asymptotically normal) irrespective of whether the error terms are heteroskedastic or homoskedastic. Classical econometrics: In classical econometrics the default assumption is that the error terms are homoskedastic Given our OLS assumptions plus homoskedasticity, the OLS estimators are efficient (optimal) among all alternative linear and unbiased estimators of the regression coefficients β 0,..., β k (Gauss-Markov theorem) 43

6 Classical econometrics: [continued] Under heteroskedasticity there are more efficient estimators than OLS, namely the so-called (feasible) Generalized Least Squares (GLS) estimators (see the lectures Econometrics I+II) Mathematical aspects: There exist specific formulas for the standard errors SE(ˆβ j ) both under heteroskedasticity and homoskedasticity Since homoskedasticity is a special case of heteroskedasticity the standard errors under homoskedasticity have a simpler structural form (homoskedasticity-only standard errors) 44

7 Mathematical aspects: [continued] The homoskedasticity-only standard errors are valid only under homoskedasticity, but lead to invalid statistical inference under heteroskedasticity The more general standard errors under heteroskedasticity were proposed by Eicker (1967), Huber (1967), and White (1980) (Eicker-Huber-White standard errors) The Eicker-Huber-White standard errors produce valid statistical inference irrespective of whether the error terms are heteroskedastic or homoskedastic (heteroskedasticity-robust standard errors) 45

8 Homoskedasticity-only and heteroskedasticity-robust standard errors for the house-prices dataset Dependent Variable: SALEPRICE Method: Least Squares Date: 07/02/12 Time: 16:50 Sample: Included observations: 546 Variable Coefficient Std. Error t-statistic Prob. C LOTSIZE BEDROOMS BATHROOMS STOREYS R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid 1.80E+11 Schwarz criterion Log likelihood Hannan-Quinn criter F-statistic Durbin-Watson stat Prob(F-statistic) Dependent Variable: SALEPRICE Method: Least Squares Date: 28/02/12 Time: 09:43 Sample: Included observations: 546 White heteroskedasticity-consistent standard errors & covariance Variable Coefficient Std. Error t-statistic Prob. C LOTSIZE BEDROOMS BATHROOMS STOREYS R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid 1.80E+11 Schwarz criterion Log likelihood Hannan-Quinn criter F-statistic Durbin-Watson stat Prob(F-statistic)

9 Practical issues: Heteroskedasticity arises in many econometric applications It is prudent to assume heteroskedastic errors unless you have compelling reasons to believe otherwise Rule of thumb: to be on the safe side, always use heteroskedasticity-robust standard errors Many software packages (like EViews) report homoskedasticity-only standard errors as their default setting It is up to the user to activate the option of heteroskedasticity-robust standard errors (in EViews: use the command ls(cov=white)) 47

10 Autocorrelated errors Problem: Particularly in time-series regressions (that is when the index i represents distinct points in time) we often encounter autocorrelated error terms: Corr(u i, u j ) 0 for some i j Under autocorrelation the OLS coefficient estimators are still consistent, but the usual OLS standard errors become inconsistent Statistical inference based on the usual OLS standard errors becomes invalid 48

11 Solution: Standard errors should be computed using a heteroskedasticity- and autocorrelation-consistent (HAC) estimator of the variance Such HAC standard errors become relevant in the Sections 6 and 9 A well-known (special) HAC estimator is the so-called Newey- West variance estimator (see Newey and West, 1987) For a more formal discussion of HAC estimators see Stock and Watson (2011, Section 15.4) 49

12 3.2. Hypothesis tests and confidence intervals for a single coefficient Testing problem: Consider one of the k regressors, say X j, and the corresponding regression coefficient β j We aim at testing the two-sided problem that the unknown β j takes on some specific value β j,0 In technical terms: H 0 : β j = β j,0 vs. H 1 : β j β j,0 50

13 Testing procedure: Compute the standard error of ˆβ j, SE(ˆβ j ) Compute the so-called t-statistic: t = ˆβ j β j,0 SE(ˆβ j ) (3.1) Compute the p-value: p-value = 2 Φ ( t act ), (3.2) where t act is the value of the t-statistic actually computed and Φ( ) is the cdf of the standard normal distribution Reject H 0 at the 5% significance level if p-value < 0.05 (or, equivalently, if t act > 1.96) 51

14 Remarks: Our testing procedure makes use of the result that the sampling distribution of the OLS estimator ˆβ j is approximately normal for moderate and large sample sizes Under H 0 the mean of this distribution is β j,0 The t-statistic (3.1) is approximately N(0, 1) distributed The phrasing t-statistic stems from the fact that for finite sample sizes and under some additional (classical) assumptions on the multiple regression model the t-statistic (3.1) follows the t-distribution with n k 1 degrees of freedom Given these restrictive assumptions the p-values should be computed from the quantiles of the t n k 1 -distribution 52

15 Remarks: [continued] Since these additional assumptions are rarely met in realworld applications and since sample sizes are typically moderate or even large, we base inference on the p-values (3.2) computed from the normal distribution Attention: Many software packages (like EViews) assume the validity of the classical assumptions and report p-values based on the t n k 1 -distribution in the default setting p-values should be corrected manually (see the following example) 53

16 p-values for the house-prices dataset based on the t- and the normal distribution, respectively Dependent Variable: SALEPRICE Method: Least Squares Date: 28/02/12 Time: 09:43 Sample: Included observations: 546 White heteroskedasticity-consistent standard errors & covariance Variable Coefficient Std. Error t-statistic Prob. C LOTSIZE BEDROOMS BATHROOMS STOREYS R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid 1.80E+11 Schwarz criterion Log likelihood Hannan-Quinn criter F-statistic Durbin-Watson stat Prob(F-statistic) Dependent Variable: SALEPRICE Method: Least Squares Date: 28/02/12 Time: 09:43 Sample: Included observations: 546 White heteroskedasticity-consistent standard errors & covariance Variable Coefficient Std. Error t-statistic Prob. C LOTSIZE BEDROOMS BATHROOMS STOREYS R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid 1.80E+11 Schwarz criterion Log likelihood Hannan-Quinn criter F-statistic Durbin-Watson stat Prob(F-statistic)

17 Remarks: [continued] A (1 α) two-sided confidence interval for the coefficient β j is an interval that contains the true value of β j with a (1 α) probability It contains the true value of β j in 100 (1 α)% of all possible randomly drawn samples Equivalently, it is the set of values of β j,0 that cannot be rejected by an α-level hypothesis test H 0 : β j = β j,0 vs. H 1 : β j β j,0 When the sample size is large, we approximate the (1 α) confidence interval for β j by [ˆβ j u 1 α/2 SE(ˆβ j ), ˆβ j + u 1 α/2 SE(ˆβ j ) ], (3.3) where u α denotes the α-quantile of the N(0, 1)-distribution 55

18 3.3. Tests of joint hypotheses Now: Testing hypotheses on two or more regression coefficients (joint hypotheses) Example: H 0 : β 1 = 0 and β 2 = 0 vs. H 1 : β 1 0 and/or β 2 0 (two restrictions) 56

19 General form of joint hypothesis: Consider the k + 1 regression coefficients β 0, β 1,..., β k and k + 1 prespecified real numbers β 0,0, β 1,0,..., β k,0 For q out of the k + 1 coefficients we consider joint null and alternative hypotheses of the form H 0 : β j = β j,0, β m = β m,0,..., for a total of q restrictions, H 1 : one or more of the q restrictions under H 0 does or do not hold 57

20 Remarks: A special case is given by considering the k restrictions with β 1,0 = 0, β 2,0 = 0,..., β k,0 = 0, that is H 0 : β 1 = 0,..., β k = 0 H 1 : at least one of the β m is nonzero for m = 1,..., k (overall-significance test of the regression model) It is tempting to conduct the test by using the usual t- statistics to test the k restrictions one at a time: Test #1: H 0 : β 1 = 0 vs. H 1 : β 1 0 Test #2: H 0 : β 2 = 0 vs. H 1 : β 2 = Test #k: H 0 : β k = 0 vs. H 1 : β k 0 58

21 Remarks: [continued] This approach is unreliable:... testing a series of single hypotheses is not equivalent to testing those same hypotheses jointly. The intuitive reason for this is that in a joint test of several hypotheses any single hypothesis is affected by the information in the other hypotheses. (Gujarati and Porter, 2009, p. 238) Solution: Joint-hypotheses testing on the basis of the F -statistic EViews-example: overall-significance test for the house-prices dataset 59

22 F -test (overall-significance test) for the house-prices dataset Dependent Variable: SALEPRICE Method: Least Squares Date: 28/02/12 Time: 09:43 Sample: Included observations: 546 White heteroskedasticity-consistent standard errors & covariance Variable Coefficient Std. Error t-statistic Prob. C LOTSIZE BEDROOMS BATHROOMS STOREYS R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid 1.80E+11 Schwarz criterion Log likelihood Hannan-Quinn criter F-statistic Durbin-Watson stat Prob(F-statistic)

23 Form and null-distribution of the F -statistic: The exact formula of the F -statistic used for testing the (general) problem of q restrictions given on Slide 57 depends on the specific assumptions imposed on the multiple regression model Also, the exact null-distribution of the F -statistic (F -distribution with exactly specified degree-of-freedom parameters n 1 and n 2 ) also depends on the these assumptions In EViews, the F -statistic and its null-distribution are computed under the classical assumptions of (1) normally distributed and (2) homoskedastic error terms u i In contrast, Stock and Watson (2011) do not assume normally distributed errors u i and consider a heteroskedasticityrobust F -statistic (Stock and Watson, 2011, pp ) 61

24 3.4. Testing single restrictions involving multiple coefficients General F -testing: Consider again the multiple regression model: Y i = β 0 + β 1 X 1i + β 2 X 2i β k X ki + u i (3.4) We aim at testing hypotheses involving some linear restrictions on the parameters of the k-variable model such as H 0 : β 2 = β 3 vs. H 1 : β 2 β 3 H 0 : β 3 + β 4 + β 5 = 3 vs. H 1 : β 3 + β 4 + β 5 3 H 0 : β 3 = β 4 = β 5 = 0 vs. H 1 : β 3 0 and/or β 4 0 and/or β 5 0 (the regressors X 3, X 4, X 5 are absent from the model) 62

25 General F -testing: [continued] All these hypotheses can be tested using a general F -statistic This general testing strategy distinguishes sharply between the so-called unrestricted regression model (3.4) and the restricted regression obtained from plugging the restriction specified under H 0 into the unrestricted regression (3.4) The general F -statistic then compares the sum of squared residuals obtained from the unrestricted regression (3.4), denoted by SSR UR, with the sum of squared residuals obtained from the restricted regression, denoted by SSR R (see Gujarati and Porter, 2009, pp ) 63

26 General F -testing: [continued] As in Section 3.3., the exact null-distribution of this general F -statistic (F -distribution with exactly specified degree-offreedom parameters) again depends on the assumptions imposed on the multiple regression model (in particular on the normality/nonnormality and the homoskedasticity/heteroskedasticity of the error terms u i ) EViews provides a fully-fledged framework for performing these general F -tests under the classical assumptions of normally distributed and homoskedastic error terms A thorough discussion will be given in the class 64

27 3.5. Confidence sets for multiple coefficients Definition 3.3: (Confidence set) A 95% confidence set for two or more coefficients is the set of numbers that contains the true population values of these coefficients in 95% of randomly drawn samples. Remarks: A confidence set is the generalization to two or more coefficients of a confidence interval for a single coefficient Recall Formula (3.3) on Slide 55 for constructing a confidence interval for the single coefficient β j 65

28 Remarks: [continued] Instead of using Formula (3.3), an equivalent way of constructing a say 95% confidence interval for the single coefficient β j consists in determining the set of all values β j,0 that cannot be rejected by a two-sided hypothesis test H 0 : β j = β j,0 vs. H 1 : β j β j,0 at the 5% significance level based on the t-statistic (3.1) on Slide 51 This approach can be extended to the case of multiple coefficients using the general F -testing approach described in Section

29 Example: [continued] Suppose you are interested in constructing a confidence set for the two coefficients β j and β m (for j, m = 0,..., k, j m) In line with Slide 57, consider testing a joint null hypothesis with the 2 restrictions H 0 : β j = β j,0, β m = β m,0 at the 5% level using the appropriate F -statistic The set of all pairs (β j,0, β m,0 ) for which you cannot reject H 0 at the 5% level constitutes a 95% confidence set for β j and β m 67

30 Remarks: In line with the confidence-interval formula (3.3), there are also analytical formulas for constructing confidence sets for multiple coefficients (not to be discussed here) EViews provides a fully-fledged framework for constructing confidence intervals and confidence sets (see class for details) Example: Confidence intervals and two-dimensional confidence sets for the house-prices dataset in EViews 68

31 Single-coefficient confidence intervals for the house-prices dataset Coefficient Confidence Intervals Date: 20/03/12 Time: 12:39 Sample: Included observations: % CI 95% CI 99% CI Variable Coefficient Low High Low High Low High C LOTSIZE BEDROOMS BATHROOMS STOREYS

32 95% two-dimensional confidence sets for the house-prices dataset LOTSIZE BEDROOMS 4,000 2, ,000 BATHROOMS 20,000 18,000 16,000 14,000 12,000 9,000 STOREYS 8,000 7,000 6, ,000-5, ,000 4,000 12,000 16,000 20,000 C LOTSIZE BEDROOMS BATHROOMS

33 Remarks: The vertical and horizontal dotted lines show the corresponding 95% confidence intervals for the single coefficients β j, β m The orientation of the ellipse indicates the estimated correlation between the OLS estimators ˆβ j and ˆβ m If the OLS estimators ˆβ j and ˆβ m were independent, the ellipses would be exact circles 71