Economics 140A Hypothesis Testing in Regression Models

Transcription

1 Economics 140A Hypohesis Tesing in Regression Models While i is algebraically simple o work wih a populaion model wih a single varying regressor, mos populaion models have muliple varying regressors 1 + X + + k X k + U The classic assumpions are virually una eced by he presence of muliple varying regressors. The only change is ha we now assume ha here is no mulicollineariy among he regressors. The coe ciens have he inerpreaion of parial derivaives The coe cien measures he e ec on of a one uni change in X holding all oher regressors consan. Esimaion of he model is exacly as before (here is no simpliciy gained by working in deviaion-from-means form), so he OLS coe cien esimaors are (B 1 B k ) arg min ~B 1 ~ B k ~ B1 ~ B X ~ Bk X k As we discussed earlier, he probabiliy ha B i i is zero, so we do no rely on poin esimaes alone. Raher we focus on inerval esimaes, which conain informaion boh abou he variance and he shape of he disribuion of he esimaor. There is an ineresing parallel beween he model wih one regressor and he model wih muliple regressors. For he model wih one regressor he variance of he esimaor of he coe cien on X 1 is V (B 1 ) X 1 X 1 For he model wih muliple regressors V (B 1 ) S 1

2 where S 1 is he sum of squared residuals afer regressing X 1 on a consan and he oher regressors. We see ha he denominaor of he variance for he single regressor model is simply he sum of squared residuals from regressing X 1 on a consan. Con dence Inervals To make clear ha a con dence inerval depends on he shape of he esimaor s disribuion consider a con dence inerval for he esimaor of he regression error variance. The esimaor of is S 1 n n k U. Noe (n k) S n k From he abulaed values of n a k (n k) S c 95 Sep 1 Sep 1 c (n k) S 1 95 a (n k) S (n c a k) S Thus, 95 percen of he random inervals of he form (n k) S (n k) S c a 95 conain. How should one choose he criical values a and c? Because he n k disribuion is skew, here are wo ways o choose he criical values. The rs way is o choose equal-ailed criical values for which (n k)s (n k)s a c 05. The second way is o selec he criical values o minimize he disance c a. (The wo ways are idenical for a symmeric disribuion.) Inser he picure from page 7 of he noes. To undersand he impor of criical value selecion, consider he following

3 Example. Le n k 18 and s 36. From he abulaed values of 18 we deermine he equal-ailed criical values (n k) S (n k) S wih Also (by grid search) we deermine he criical values ha minimize heir di erence (n k) S (n k) S and wih The logic ha we wish o consruc he shores con dence inerval seems o poin o he second way. Wha are he wo con dence inervals? For he equal-ailed criical values, he con dence inerval is (06 790) wih lengh For he shores disance criical values, he con - dence inerval is (17 913) wih lengh The shorer inerval is obained wih he equalailed criical values! Why? Because he con dence inerval is no a linear funcion of he criical values ( 1 and 1 appear raher han a and c), so he shores criical a c value inerval does no yield he shores con dence inerval. Hypohesis Tesing While inerval esimaion is ofen done wihou a speci c hypohesis in mind, he esimaed inerval is easily used o es a hypohesis. Recall ha for hypohesis esing we rs deermine he null hypohesis and he alernaive in such a way ha rejecion of he null hypohesis (in favor of he alernaive hypohesis) is a conclusion of ineres. Consider he populaion regression model in which he dependen variable is consumpion of a paricular good and he regressor of ineres is income. We wish o esablish if increases in income a ec consumpion of he good, bu we are fairly cerain ha increases in income do no decrease consumpion of he good. The null hypohesis H 0 0 3

4 is esed agains he one-sided alernaive H 1 > 0 Coe cien Esimaor Tes Saisics As we have seen, we examine he sample daa o see if he esimae is large enough ha we may rejec he null hypohesis. How large is large enough o be saisically persuasive? We mus deermine a bound, such ha if he esimae is a leas as large as he bound, he esimae would cause us o rejec he null hypohesis. Again B N (0 1) B where B is he variance of B. Recall ha we begin by deermining a region for which we do no rejec he null hypohesis (for he wo-sided alernaive, ( )). For he one-sided alernaive a es, he region for which we do no rejec he null will be ( 1 c), where c is he larges value we feel is consisen wih he null hypohesis. From abulaed values of N (0 1), an upper bound for is obained from B B Sep 1 Sep Sep 3 (B 165 B ) 95 ( B B ) 95 (B 165 B ) 95 Thus, 95 percen of random inervals of he form (B 165 B 1) conain. In oher words, 95 percen of he random values B 165 B are less han or equal o. Because is very likely o be a leas as large as he lower bound, we are con den ha if he esimaed lower bound is larger han zero hen he null hypohesis is false. 4

5 For a given sample, we have he esimae b. If we replace he random esimaor wih he xed esimae, hen he 95 percen (one-sided) con dence inerval is (b 165 B 1) or he 95 percen lower con dence bound is b 165 B. To es he null hypohesis, we check o see if he lower con dence bound is posiive. If he lower con dence bound is posiive, we rejec he null hypohesis. We do so because we have 95 percen con dence ha he populaion value is larger han he lower con dence bound, so we have 95 percen con dence ha he populaion value is posiive. If he lower con dence bound is no posiive, hen he null value of zero lies in our con dence inerval and so we fail o rejec he null hypohesis. I is also sraighforward o perform hypohesis ess on sums (or di erences) of coe ciens. Consider a es ha wo coe ciens are equal ( 1 ) H agains H The null hypohesis is rejeced if he con dence inerval formed from 195 B 1 B B1 B does no conain 0. Noe B 1 B V (B 1 ) + V (B ) C (B 1 B ) Likelihood Tes Saisics While con dence inervals are one way of esing hypoheses, oher ess are available. For wo-sided alernaives here is also he likelihood-raio es saisic. Consider a es of H 0 0 agains H The likelihood-raio es saisic is consruced by esimaing he model under he null and alernaive hypoheses. Le L be he likelihood funcion wih in he model and le L R be he likelihood funcion wih excluded from he model (ha is, H 0 0 is imposed). The likelihood-raio es saisic LR ln L 1 5

6 where he single degree-of-freedom re ecs he fac ha only one resricion is imposed on he resriced model. If he null hypohesis is correc, hen L R ' L and he es saisic is close o zero. If he null hypohesis is false, hen L R < L and he es saisic will be larger han zero. We rejec he null hypohesis if he esimaed es saisic exceeds he criical value. Finally, one should noe he di erence beween saisical signi cance and economic signi cance. I may be he case ha an esimaed coe cien is saisically signi can, ye he magniude of he erm is so small ha i has lile or no e ec on he dependen variable. Sum of Squares Decomposiion While speci caion of a model is mos ofen based on heoreical grounds, i is imporan o have some measure of he overall adequacy of he model in explaining he movemens in he dependen variable. The baseline for comparison is he model wihou heoreical regressors 0 + U For such a model, he predicion for each value of is he consan 0, where he OLS esimaor of 0 is B 0 1 Y n 1 Any improvemen aribued o heoreical regressors requires ha we explain he variaion Yn. The oal variaion in he sample, ofen ermed he oal sum-of-squares, is Yn To derive an expression for he oal sum-of-squares, we consider he model wih a single regressor. The unexplained sum-of-squares, which is he variaion no explained by he model, is Y ( B 0 B 1 X ) Yn B 1 X Xn Yn B 1 6 Yn X Xn + B 1 X Xn

7 From he de niion of B 1, he las erm cancels wih par of he middle erm, hus Y Yn B 1 Yn X Xn where he las erm on he righ is he variaion explained by he heoreical regressors (ermed he explained sum-of-squares). We have ha he oal sumof-squares is equal o he explained sum-of-squares plus he unexplained sum-ofsquares. One of he mos common measures of model adequacy is he raio of explained variaion o oal variaion, ofen ermed R squared, R Y Because he OLS esimaors minimize he sum of squared residuals, hey maximize R. Tha is, he OLS esimaors are he esimaors ha resul in he highes degree of explained variaion. As indicaed by he above derivaion, 0 R 1. Ye i is possible o have compuer oupu wih a negaive esimae of R. How so? Consider a regression model wih he inercep omied. A correc program calculaes R as R 1 Y which is conained in [0 1]. Ye some programs err by including an inercep in he oal sum of squares calculaion R 1 Y for which i is possible ha Y Y > n Y. Include graph from aached yellow ruled shee numbered page 8 Because 1 + X + + K X K 1 + U will exacly any sample of K observaions of (X ), inclusion of heoreically irrelevan regressors will increase R. Mos regression model esimaes repor 7

8 anoher measure, he adjused R in which he explanaory power of an addiional regressor is raded o agains he loss of one degree-of-freedom R A (Y n K ) ( Yn) A imes i is useful o sudy he signi cance of he proposed populaion regression model. The signi cance of he proposed model is under es wih n 1 H 0 3 K 0 agains H 1 no H 0 where he alernaive hypohesis is ha a leas one coe cien is no zero. If K, hen we have only a single coe cien o es and a naural es saisic is he coe cien esimaor es saisic B 0 S B wih S B h n S X i 1 X We could, equivalenly, use he square of he coe cien esimaor es saisic B n X X S which has an F 1n disribuion. (Fac If he random variable Z has a disribuion wih n degrees-of-freedom, hen Z has an F 1n disribuion.) If K >, hen a single coe cien esimaor is no available (nor or ransformaions as B B 3 0 does no impose he resricion ha boh coe ciens are zero). Raher we noe ha he numeraor of he squared coe cien esimaor es saisic is simply Y, which can be consruced for K >. In deail, because he ed values conain he sample mean, so Y (B1 + B X + + B K X K ) B 1 + B X + + B KXK KX k1 B k X k Xk 8

9 Hence he coe cien esimaor es saisic is (Y ) K 1 S If he null hypohesis is rue, hen he numeraor of he es saisic will be close o zero, so we rejec he null hypohesis if he es saisic exceeds he criical value from he F K 1n K disribuion. As one would surmise, he es saisic can be wrien in erms of R as (Y K 1 ) n K S K 1 n K K 1 (U ) n K R K 1 1 R The es saisic is an increasing funcion of R. redicion Y Y A goal of regression esimaion is predicion of he dependen variable. Consider a model wih K. Given a value of he regressor (eiher from he sample or ou of he sample) X, he corresponding predicion of he dependen variable is Y B 1 + B X. To be useful, he poin predicion mus be accompanied by a measure of uncerainy. The variance of he predicion is V Y V (B1 ) + V (B ) X + C (B 1 B ) X + V (U ) X n X " 1 n + + X n X n + X Xn X # + X! X Xn n X X X Xn n X X + X + Thus a 95 percen con dence inerval for he prediced values is no parallel o he regression line, bu raher increases in widh as he disance from he sample mean 9

10 increases. Noe, if here is more han one regressor, hen he con dence inerval does no necessarily increase as he disance from he sample means increases. This follows because he variance of he predicion conains he covariance beween he regressors, which can be negaive. Noe, ha if one wished o remove he e ec of he idiosyncraic error U, which implies ha we are predicing E ( jx ) raher han jx, he variance is lowered by he amoun (he quaniy V (U ) is eliminaed because EU 0). 10