Econometrics The Multiple Regression Model: Inference

Transcription

1 Econometrics The Multiple Regression Model: João Valle e Azevedo Faculdade de Economia Universidade Nova de Lisboa Spring Semester João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

2 in the Multiple Linear Regression Model Suppose you want to test whether a variable is important in explaining variation in the dependent variable: E.g., is the effect of tenure on wages statistically significant (ie, different from zero)? Is the effect of height on wages statistically significant? Or suppose you want to test whether a coefficient has a particular value E.g., is the effect of one additional year of schooling on expected monthly wages equal to 200? Need to take into account the sampling distribution of our estimators We will check whether under the maintained hypothesis (or null hypothesis) the observed values of certain test statistics are likely If they are not we reject the null João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

3 in the Multiple Linear Regression Model y = β 0 + β 1 x 1 + β 2 x β k x k + u Assumption MLR.6 (Normality) The distribution of the population error u is independent of x 1, x 2,..., x k and u is normally distributed with mean 0 and variance σ 2 : we write u Normal(0, σ 2 ) Independence assumption is stronger than MLR.4 (Zero Conditional Mean) assumption. Actually, it implies MLR.4 Also, normality and independence imply MLR.5 so that all the results regarding unbiasedness and variance of the estimators remain valid João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

4 Classical Linear Model Assumptions MLR.1 through MLR.6 are the Classical Linear Model (CLM) assumptions Under the CLM assumptions, OLS is not only BLUE, but is the minimum variance unbiased estimator: no other unbiased estimator has a variance smaller than OLS We can summarize the population assumptions of CLM as follows y X Normal(β 0, β 1 x 1, β 2 x 2,..., β k x k, σ 2 ) Normality is unrealistic in many cases (e.g., wages cannot be negative but under the normality assumption of u we can get negative wages) However, most results would hold in large samples without the normality assumption João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

5 Normal Sampling Distribution y f(y x). Normal distributions. E(y x) = b 0 + b 1 x x 1 x 2 Figure: The homoskedastic normal distribution with a single explanatory variable João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

6 Normal Sampling Distribution Since the OLS estimators are a linear function of the error term u, then (conditional on the x s): Theorem Under the CLM assumptions, conditional on the sample values of the independent variables, ˆβ j Normal[β j, Var( ˆβ j )], Therefore, ( ˆβ j β j ) sd( ˆβ Normal(0, 1) j ) where sd stands for standard deviation (squared root of the variance, derived in previous classes) João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

7 Normal Sampling Distribution Now, the σ 2 that appears in the expression for the standard deviation of the estimators must be estimated Also, conditional on the x s (n k 1)ˆσ 2 /σ 2 χ 2 n k 1 which implies: ( ˆβ j β j ) se( ˆβ j ) = ( ˆβ j β j ) sd( ˆβ j ) sd( ˆβ j ) se( ˆβ j ) = ( ˆβ j β j ) σ sd( ˆβ j ) ˆσ Normal(0, 1) χ 2 n k 1 n k 1 t n k 1 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

8 Normal Sampling Distribution Theorem Under the CLM assumptions MLR.1 through MLR.6, ( ˆβ j β j ) t n k 1, se( ˆβ j ) where k+1 is the number of unknown parameters in the population model y = β 0 + β 1 x β k x k + u (k slope parameters and the intercept β 0 ) João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

9 Performing a test on a coefficient Set the null hypothesis (and the alternative) E.g., H0 : β j = 0 (coefficient on experience in our wage regression) and H 1 : β j > 0 Choose a significance level (Probability of rejecting the null if the null is actually true) E.g., α = 0.05 Look at the sampling distribution of the test statistic t (random variable) involving the parameter: t = ( ˆβ j β j ) se( ˆβ j ) t (n k 1), Under the null hypothesis, the test statistic should be small across samples. Reject the null if the observed value of the test statistic is very unlikely (very large) João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

10 Performing a test on a coefficient One-side Tests For one-sided tests where the alternative is favored if t obs is large and positive (e.g., H 1 : β j > 0), reject the null if the observed test statistic, t obs, is larger than c, where c is implicitly given by: Prob[t > c H 0 is true]=α For one-sided tests where the alternative is favored if t obs is large and negative (e.g., H 1 : β j < 0), reject the null if the observed test statistic, t obs, is smaller than -c, where c is implicitly given by: Prob[t < c H 0 is true]=α For two-sided tests, where the alternative is favored if t obs is large in absolute value (e.g., H 1 : β j 0), reject the null if the absolute value of observed test statistic, t obs, is larger than c, where c is implicitly given by: Prob[ t > c H 0 is true]=α João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

11 One-Sided Alternative H 0 : β j = 0 H 1 : β j > 0 Fail to reject the null (1-α) Reject the null α Figure: Rejection region for a 5% significance level for alternative H 1 : β j > 0 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

12 Two-Sided Alternative H 0 : β j = 0 H 1 : β j 0 Fail to reject the null Reject the null (1-α) Reject the null α/2 α/2 Figure: Rejection region for a 5% significance level for alternative H 1 : β j 0 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

13 Example: Hypothesis Testing Independent Variable Coefficient Estimate Standard Error Intercept Education (in years) Labor Market Experience (in years) Square of Labor Market Experience (in years) n R t ratio Figure: Dependent Variable: Log of Wages The t ratios are the observed values of the test statistic for testing β j = 0 E.g = / João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

14 Example: Hypothesis Testing (Cont.) Choose α = 0.05 Test H 0 : β j = 0 against H 1 : β j 0 (coefficient on education) t obs = = t >1.96 Reject the null: the coefficient for education is significant at 5% significance level We use Normal approximation since n is large Fail to reject the null Reject the null Reject the null -c=-1.96 c=1.96 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

15 Example: Hypothesis Testing (Cont.) Choose α = 0.05 Test H 0 : β j = 0 against H 1 : β j > 0 (clearly more reasonable...) t obs = = t >1.645 Reject the null: the coefficient for education is significant at 5% significance level We use Normal approximation since n is large Fail to reject the null Reject the null c=1.645 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

16 Example: Hypothesis Testing (Cont.) Choose α = 0.05 Test H 0 : β j = 0.07 against H 1 : β j 0.07 (coefficient on education) t obs = = t >1.96 Reject the null: the coefficient for education is significant at 5% significance level We use Normal approximation since n is large Fail to reject the null Reject the null Reject the null -c=-1.96 c=1.96 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

17 p-value p-value: Given the observed value of the t statistic, what would be the smallest significance level at which the null H 0 : β j = 0 would be rejected against the alternative H 1 : β j 0? It is given by: Prob[ t > t obs H 0 true] 1- p-value p-value /2 p-value /2 -t obs t obs João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

18 Confidence Intervals A (1 α)% confidence interval is defined as: ˆβ j ± c se( ˆβ j ) where c is the (1 α 2 ) percentile in a t n k 1 distribution If the hypothesized value of a parameter (b j ) is inside the confidence interval, we would not reject the null β j = b j against β j b j at the significance level α João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

19 Testing multiple exclusion restrictions Unrestricted model: y = β 0 + β 1 x 1 + β 2 x 2 + β 3 x β k x k + u H 0 : β k q+1 = β k q+2 =... = β k = 0 H 1 : NotH 0 Restricted model: y = β 0 + β 1 x 1 + β 2 x 2 + β 3 x β k q x k q + u Under the null: Fstatistic = (SSR r SSR ur )/q SSR ur /(n k 1) F (q,n k 1) r stands for restricted and ur for unrestricted, q is number of restrictions Does SSRur decrease enough compared to SSR r? If F obs is too large we reject the null João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

20 Testing multiple exclusion restrictions H 0 : β k q+1 = β k q+2 =... = β k = 0 H 1 : NotH 0 Fstatistic = (SSR r SSR ur )/q SSR ur /(n k 1) F (q,n k 1) Fstatistic = (R 2 ur R 2 r )/q (1 R 2 ur )/(n k 1) F (q,n k 1) Obtained by dividing the numerator and the denominator above by SST This is different from testing significance of each coefficient individually!! It is a test of joint significance João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

21 Testing multiple exclusion restrictions: F test Reject the null if the observed test statistic, F obs, is larger than c, where c is implicitly given by: Prob[F > c H 0 istrue] = α Fail to Reject the null 1-α α Reject the null c João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

22 Example H 0 : β 2 = β 3 = 0 Independent Variable Coefficient Estimate Standard Error Unrestricted model Intercept Education (in years) Labor Market Experience (in years) Square of Labor Market Experience (in years) R Mean Square Error Restricted model Intercept Education (in years) t ratio R Mean Square Error Figure: Dependent Variable: Log of monthly wage, n=11064 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

23 Example (Cont.) α = 0.05 H 0 : β 2 = β 3 = 0 Fstatistic = ( )/2 ( )/( ) = > 3.00 Reject H 0 João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24

24 Overall significance of the model Under the null use: H 0 : β 1 = β 2 =... = β k = 0 H 1 : NotH 0 (SST SSR)/k F = SSR/(n k 1) SSE/k = SSR/(n k 1) = R 2 /k (1 R 2 )/(n k 1) F (k,n k 1) Testing general linear restrictions: in the practice sessions! João Valle e Azevedo (FEUNL) Econometrics Lisbon, March / 24