Multiple Linear Regression

Transcription

1 Multple Lnear Regresson Heteroskedastcty Heteroskedastcty

2 Multple Lnear Regresson: Assumptons Assumpton MLR. (Lnearty n parameters) Assumpton MLR. (Random Samplng from the populaton) We have a random sample: satsfyng the equaton above Assumpton MLR.3 (No perfect Collnearty) In the sample, none of the ndependent varables s a lnear combnaton of the others. Assumpton MLR.4 (Zero Condtonal Mean) ths mples Under asumptons MLR. to MLR.4 we have seen that OLS estmator s Unbased and Consstent Heteroskedastcty

3 Homoskedastcty Assumpton MLR.5 (Homoskedastcty) Wth MLR. through MLR.5 we have derved the varance of the OLS estmators and further concluded that OLS was assmptotcally Normal: Enough to conduct nference as usual =,,,k ( ) ( ) a ˆ seˆ ~ Normal( 0,) If MLR.5 does not hold, that s, f the condtonal varance of u s allowed to vary gven the x s, then the errors are heteroskedastc and results above are NOT vald. Cannot make nference as usual (t-tests, F tests, LM tests) Heteroskedastcty 3

4 Example of Heteroskedastcty Suppose y s wage and x s educaton f(y x) y... E(y x) = 0 + x x x x 3 Heteroskedastcty 4 x

5 Varance wth Heteroskedastcty Now assume For the smple regresson case : ˆ Var ( ˆ ) When = ( x x) σ SST + = x uˆ ( x x) ( x x) ( x x) σ Var(u x SST x can estmate ths, where uˆ u σ,...,x, so, condtonal on the x's :, k ) = where σ SST x = varance as : ( x x) are are the OLS resduals Heteroskedastcty 5

6 Varance wth Heteroskedastcty General case: k regressors For the multple regresson model, a vald (consstent) estmator of rˆ SSR uˆ s the th Var ( ˆ ) are the OLS resduals wth heteroskedastcty s : Var ˆ ( ˆ ) resdual from regressng x = rˆ uˆ SSR on all other ndependent varables s the sum of squared resduals from ths regresson, Heteroskedastcty 6

7 Robust Standard Errors The square root of ths varance can be used as a standard error for nference (Robust Standard error). Wth these standard errors t turns out that: t = ( ) ( ) a ˆ seˆ ~ Normal( 0,) Ths s an Heteroskedastcty-robust t statstc Often, the estmated varance s corrected for degrees of freedom by multplyng by n/(n k ) (rrelevant for large n) Heteroskedastcty 7

8 Robust Standard Errors (cont) Why not use always robust standard errors? In small samples t statstcs usng robust standard errors wll not have a dstrbuton close to the Normal (or t) Wll not deal wth Heteroskedastcty-robust F statstcs Instead, use Heteroskedastcty-robust LM tests Heteroskedastcty 8

9 A Robust LM Statstc Suppose we have a standard model, y = 0 + x + x +... k x k + u and our null hypothess s H 0 : k-q+ = 0,..., k = 0; the number of restrctons s q Frst, we ust run OLS on the restrcted model and save the resduals ŭ Regress each of the excluded varables on all of the ncluded varables (q dfferent regressons) and save each set of resduals ř, ř,, ř q Regress a varable defned to be = on ř ŭ, ř ŭ,, ř q ŭ, wth no ntercept The LM statstc s n SSR, where SSR s the sum of squared resduals from ths fnal regresson, t has a ch-square Dstrbuton wth q degrees of freedom (under the Null) Heteroskedastcty 9

10 Testng for Heteroskedastcty Want to test H 0 : Var(u x, x,, x k ) = σ, whch s equvalent to H 0 : E(u x, x,, x k ) = E(u ) = σ Assume alternatve s lnear relaton between u and x For u = δ 0 + δ x + + δ k x k + v ths means testng H 0 : δ = δ = = δ k = 0 Don t observe the error, but can use resduals from the OLS regresson Heteroskedastcty 0

11 Testng for Heteroskedastcty - Breusch-Pagan Estmate û = δ 0 + δ x + + δ k x k + error by OLS Want to test H 0 : δ = δ = = δ k = 0 Take the R of ths regresson. Wth assumptons MLR. through MLR.4 stll n place we can use an F test or an LM type test: The F statstc s ust the reported F statstc for overall sgnfcance of ths regresson, F = [R /k]/[( R )/(n k )], whch s dstrbuted approxmately as F k, n k under the null Alternatvely, can form the LM statstc LM = nr, whch s approxmately dstrbuted as a χ k under the null (R of the regresson above!, ths s not the typcal LM test!) These tests are usually called the Breusch-Pagan tests for heteroskedastcty Heteroskedastcty

12 The Whte Test The Breusch-Pagan tests wll detect lnear forms of heteroskedastcty The Whte test allows for nonlneartes by usng squares and cross-products of all the x s Estmate û = δ 0 + δ x + + δ k x k + δ k+ x + + δ k x k + δ k+ x x + + δ k+k(k+)/ x k x k- + error by OLS Want to test H 0 : δ = δ = = δ k+k(k+)/ = 0 Take the R of ths regresson Stll use the F or LM statstcs to test whether all the x, x, and x x h are ontly sgnfcant: F = [R /q]/[( R )/(n q )] ~ F q, n k (approx.) under the null and LM= LM = nr ~ χ q (approx.) under the null (q= k+k(k+)/ ) If k s large and n small these approxmatons are poor Heteroskedastcty

13 Alternate form of the Whte test Now, the ftted values from OLS, ŷ, are a functon of all the x s Thus, ŷ wll be a functon of the squares and cross-products and ŷ and ŷ can substtute for all of the x, x, and x x h, so: Regress the squared resduals on ŷ and ŷ (as well as a constant) and use the R to form an F or LM statstc (as for the BP or Whte tests) Only testng restrctons now Heteroskedastcty 3

14 Revew - Heteroskedastcty Relaxng assumpton MLR.5 we have : Var(u x,...,x ) = k σ We were stll able to estmate consstently the varance of the OLS estmators rˆ uˆ s the SSR th are the OLS resduals Vˆ ar ( ˆ ) resdual from regressng x = rˆ uˆ SSR on all other ndependent varables s the sum of squared resduals from ths regresson The mpled (robust) standard errors allow us to use Heteroskedastcty robust t-statstcs: t = a ( ˆ ) se( ˆ ) ~ Normal( 0,), Heteroskedastcty 4

15 Revew - Heteroskedastcty For multple excluson restrctons we used an Heteroskedastcty robust LM test statstc We have also seen Tests for Heteroskedastcty H 0 : Var(u x, x,, x k ) = σ, whch s equvalent to H 0 : E(u x, x,, x k ) = E(u ) = σ Estmate û = δ 0 + δ x + + δ k x k + error by OLS Breusch-Pagan test Want to test H 0 : δ = δ = = δ k = 0 Take R of ths regresson and use: F = [R /k]/[( R )/(n k )]~F k, n k (approx.) under the null or LM = nr ~χ k (approx.) Heteroskedastcty 5

16 Revew - Heteroskedastcty For more general forms of heteroskedastcty use the Whte test : Estmate û = δ 0 + δ x + + δ k x k + δ k+ x + + δ k x k + δ k+ x x + + δ k+k(k+)/ x k x k- + error by OLS Want to test H 0 : δ = δ = = δ k+k(k+)/ = 0 Take the R of ths regresson and use: F = [R /q]/[( R )/(n q )] ~ F q, n k (approx.) under the null or LM= LM = nr ~ χ q (approx.) (q= k+k(k+)/ ) Alternatvely, f k s large estmate û = δ 0 + δ ŷ+ δ ŷ + error by OLS and test δ = δ = 0 Use the F or LM statstcs (ust as above) but consderng degrees of freedom Heteroskedastcty 6

17 Weghted Least Squares We can always estmate robust standard errors for OLS However, f we know somethng about the specfc form of the heteroskedastcty, we can obtan estmators that have a smaller varance than OLS If we know n fact somethng we are able to transform the model nto one that has homoskedastc errors Heteroskedastcty 7

18 Case of known form up to a multplcatve constant Suppose we know that Var(u x) = σ h(x), or Var(u x) = σ h(x )=σ h, say Example: Wage= 0 + Educaton+ Var( u Educaton, Exp., Ten.) = σ Experence+ Educaton 3 Tenure+ u We know that E(u / h x) = 0, because h depends only on x, and Var(u / h x) = σ, because Var(u x) = σ h So, f we dvde the regresson equaton by h we wll get a model where the error s homoskedastc (MLR. to MLR.5 verfed agan) Heteroskedastcty 8

19 Heteroskedastcty 9 Generalzed Least Squares Estmatng the transformed equaton by OLS s an example of generalzed least squares (GLS) GLS wll be BLUE (Best Lnear Unbased Estmator) n ths case The GLS estmator for the partcular case where we dvde the regresson equaton by h s called a weghted least squares (WLS) estmator. Why? wth larger varance are gven a smaller weght Indvduals / ) ˆ... ˆ ˆ ( /, / where, ) ˆ... ˆ / ˆ ( 0 * * * * 0 * k k n k k n h x x y h x x h y y x x h y = = = = =

20 More on WLS We nterpret WLS estmates n the orgnal (not transformed model) but get varances of the WLS estmators n the transformed model WLS s optmal f we know the form of Var(u x ) In most cases, won t know the form of heteroskedastcty Can often estmate the form of heteroskedastcty Example: Wage= Var( u Educaton, Exp., Ten.) = σ where δ Educaton+ and δ are unknown Experence+ exp( δ 0 + δ 3 Tenure+ u Educaton) Heteroskedastcty 0

21 Must estmate the form of Heteroskedastcty: Feasble GLS Frst, we assume a model for heteroskedastcty E.g., Var(u x) =E[u x] =σ exp(δ 0 + δ x + + δ k x k ) >0 Snce we don t know the δ s, must estmate them We can wrte the above model as: u = σ exp(δ 0 + δ x + + δ k x k )v, where E[v x]= Assume further that v s ndependent of x Then ln(u ) = α 0 + δ x + + δ k x k + e g+e where E(e) = 0 and e s ndependent of x Heteroskedastcty

22 Feasble GLS (contnued) ln(u ) = α 0 + δ x + + δ k x k + e where E(e) = 0 and e s ndependent of x Can use û (from OLS) nstead of u, to estmate ths equaton by OLS Then, obtan an estmate of h by ĥ = exp(ĝ ), Fnally, use /ĥ as the weghts n WLS Summary: Run OLS n the orgnal model, save the resduals, û, square them and take logs Regress ln(û ) on all of the ndependent varables (plus constant) and get the ftted values, ĝ Do WLS usng /exp(ĝ) as the weght Heteroskedastcty

23 Notes on GLS OLS s stll unbased and consstent wth heteroskedastcty (as long as MLR. through MLR.4) hold We use GLS ust for effcency (smaller varance of the estmators) If we know the weghts to use n WLS, then GLS s unbased. Otherwse, and assumng that we estmate a correctly specfed for heteroskedastcty, FGLS (whch s a Feasble GLS) s not unbased but s consstent and asymptotcally effcent Remember, wth FGLS we are estmatng the parameters of the orgnal model. Standard errors n the transformed model also refer to standard errors n the orgnal model Can use the t and F tests for nference When dong F tests wth WLS, form the weghts from the unrestrcted model and use those weghts to do WLS on the restrcted model as well as on the unrestrcted model Heteroskedastcty 3