Multilevel Analysis (ver. 1.0)

Transcription

1 Multlevel Analyss (ver. 1.0) Oscar Torres-Reyna Data Consultant

2 Motvaton Use multlevel model whenever your data s grouped (or nested) n more than one category (for example, states, countres, etc). Multlevel models allow: Study effects that vary by entty (or groups) Estmate group level averages Some advantages: Regular regresson gnores the average varaton between enttes. Indvdual regresson may face sample problems and lack of generalzaton

3 Varaton between enttes use bysort school: egen y_meanmean(y) twoway scatter y school, msze(tny) connected y_mean school, connect(l) clwdth(thck) clcolor(black) mcolor(black) msymbol(none), yttle(y) y school Score y_mean 3

4 statsby nter_b[_cons] slope_b[x1], by(school) savng(ols, replace): regress y x1 sort school merge school usng ols Indvdual regressons (no-poolng approach) drop _merge gen yhat_ols nter + slope*x1 sort school x1 separate y, by(school) separate yhat_ols, by(school) twoway connected yhat_ols1-yhat_ols65 x1 lft y x1, clwdth(thck) clcolor(black) legend(off) yttle(y) y Readng test 4

5 Varyng-ntercept model (null). xtmxed y school:, mle nolog y j[ ] α + ε Mxed-effects ML regresson Number of obs 4059 Group var able: school Number of groups 65 Obs per group: mn avg 6.4 max 198 Mean of state level ntercepts Wald ch(0). Log lkelhood Prob > ch. y Coef. Std. Err. z P> z [95% Conf. Interval] _cons Standard devaton at the school level (level ) Random-effects Parameters Estmate Std. Err. [95% Conf. Interval] Standard devaton at the ndvdual level (level ) school: Identty sd(_cons) sd(resdual) LR test vs. lnear regresson: chbar(01) Prob > chbar Intraclass ( sgma _ u) correlaton ( sgma _ u) + ( sgma _ e) sd(_ cons) sd(_ cons) + sd( resdual) _ 0.17 Ho: Random-effects 0 If the nterclass correlaton (IC) approaches 0 then the groupng by countes (or enttes) are of no use (you may as well run a smple regresson). If the IC approaches 1 then there s no varance to explan at the ndvdual level, everybody s the same. An ntraclass correlaton tells you about the correlaton of the observatons (cases) wthn a cluster ( 5

6 Varyng-ntercept model (one level-1 predctor). xtmxed y x1 school:, mle nolog y α + βx + ε j[ ] Mxed-effects ML regresson Number of obs 4059 Group var able: school Number of groups 65 Obs per group: mn avg 6.4 max 198 Mean of state level ntercepts Standard devaton at the school level (level ) Standard devaton at the ndvdual level (level ) Wald ch(1) Log lkelhood Prob > ch x _cons Random-effects Parameters Estmate Std. Err. [95% Conf. Interval] school: Identty y Coef. Std. Err. z P> z [95% Conf. Interval] sd(_cons) sd(resdual) LR test vs. lnear regresson: chbar(01) Prob > chbar Intraclass ( sgma _ u) correlaton ( sgma _ u) + ( sgma _ e) sd(_ cons) sd(_ cons) + sd( resdual) _ 0.14 Ho: Random-effects 0 If the nterclass correlaton (IC) approaches 0 then the groupng by countes (or enttes) are of no use (you may as well run a smple regresson). If the IC approaches 1 then there s no varance to explan at the ndvdual level, everybody s the same. An ntraclass correlaton tells you about the correlaton of the observatons (cases) wthn a cluster ( 6

7 Varyng-ntercept, varyng-coeffcent model y α β x + ε j[ ] + j[ ]. xtmxed y x1 school: x1, mle nolog covarance(unstructure) Mxed-effects ML regresson Number of obs 4059 Group var able: school Number of groups 65 Obs per group: mn avg 6.4 max 198 Mean of state level ntercepts Wald ch(1) Log lkelhood Prob > ch y Coef. Std. Err. z P> z [95% Conf. Interval] x _cons Standard devaton at the school level (level ) Standard devaton at the ndvdual level (level ) Random-effects Parameters Estmate Std. Err. [95% Conf. Interval] school: Unstructured sd(x1) sd(_cons) corr(x1,_cons) sd(resdual) LR test vs. lnear regresson: ch(3) Prob > ch Note: LR test s conservatve and provded only for reference. Ho: Random-effects 0 Intraclass ( sgma _ u) correlaton ( sgma _ u) + ( sgma _ e) sd(_ cons) + sd( x1) sd(_ cons) + sd( x1) + sd( resdual) _

8 Varyng-slope model y α β x + ε + j[ ]. xtmxed y x1 _all: R.x1, mle nolog Mxed-effects ML regresson Number of obs 4059 Group var able: _all Number of groups 1 Obs per group: mn 4059 avg max 4059 Mean of state level ntercepts Wald ch(1) Log lkelhood Prob > ch y Coef. Std. Err. z P> z [95% Conf. Interval] Standard devaton at the school level (level ) x _cons Random-effects Parameters Estmate Std. Err. [95% Conf. Interval] _all: Identty sd(r.x1) Standard devaton at the ndvdual level (level ) sd(resdual) LR test vs. lnear regresson: chbar(01) 0.00 Prob > chbar

9 Postestmaton 9

10 Comparng models usng lkelhood-raton test Use the lkelhood-rato test (lrtest) to compare models ftted by maxmum lkelhood. Ths test compares the log lkelhood (shown n the output) of two models and tests whether they are sgnfcantly dfferent. /*Fttng random ntercepts and storng results*/ quetly xtmxed y x1 school:, mle nolog estmates store r /*Fttng random coeffcents and storng results*/ quetly xtmxed y x1 school: x1, mle nolog covarance(unstructure) estmates store rc /*Runnng the lkelhood-rato test to compare*/ lrtest r rc. lrtest r rc Lkelhood-rato test LR ch() (Assumpton: r nested n rc) Prob > ch Note: LR test s conservatve The null hypothess s that there s no sgnfcant dfference between the two models. If Prob>ch<0.05, then you may reject the null and conclude that there s a statstcally sgnfcant dfference between the models. In the example above we reject the null and conclude that the random coeffcents model provdes a better ft (t has the lowest log lkelhood) 10

11 Varyng-ntercept, varyng-coeffcent model: postestmaton. xtmxed y x1 school: x1, mle nolog covarance(unstructure) varance Mxed-effects ML regresson Number of obs 4059 Group var able: school Number of groups 65 Obs per group: mn avg 6.4 max 198 Mean of state level ntercepts Wald ch(1) Log lkelhood Prob > ch y Coef. Std. Err. z P> z [95% Conf. Interval] x _cons Standard devaton at the school level (level ) Standard devaton at the ndvdual level (level ) Random-effects Parameters Estmate Std. Err. [95% Conf. Interval] school: Unstructured var(x1) var(_cons) cov(x1,_cons) var(resdual) LR test vs. lnear regresson: ch(3) Prob > ch Note: LR test s conservatve and provded only for reference. ( sgma _ u) var(_ cons) + var( x1) Intraclass _ correlaton 0.14 ( sgma _ u) + ( sgma _ e) var(_ cons) + var( x1) + var( resdual)

12 Postestmaton: varance-covarance matrx. xtmxed y x1 school: x1, mle nolog covarance(unstructure) varance Random-effects Parameters Estmate Std. Err. [95% Conf. Interval] school: Unstructured var(x1) var(_cons) cov(x1,_cons) var(resdual) LR test vs. lnear regresson: ch(3) Prob > ch Note: LR test s conservatve and provded only for reference.. estat recovarance Random-effects covarance matrx for level school x1 _cons x _cons Varance-covarance matrx. estat recovarance, correlaton Random-effects correlaton matrx for level school x1 _cons x1 1 _cons The correlaton between the ntercept and x1 shows a close relatonshp between the average of y and x1. 1

13 Postestmaton: estmatng random effects (group-level errors) y x α j[ ] + β j[ ] + ε y α j[ ] + β j[ ] x + uα + uβ + j[ ] ε Fxed-effects Random-effects To estmate the random effects u, use the command predct wth the opton reffects, ths wll gve you the best lnear unbased predctons (BLUPs) of the random effects whch bascally show the amount of varaton for both the ntercept and the estmated beta coeffcent(s). After runnng xtmxed, type predct u*, reffects Two new varables are created u1 BLUP r.e. for school: x /* u β */ u BLUP r.e. for school: _cons --- /* u α */ 13

14 Postestmaton: estmatng random effects (group-level errors) y x1 y x1 + u α + uβ To explore some results type: Fxed-effects Random-effects bysort school: generate groups(_n1) /*_n1 selects the frst case of each group */ lst school u u1 f school<10 & groups. lst school u u1 f school<10 & groups school u u1 Here u and u1 are the group level errors for the ntercept and the slope respectvely. For the frst school the equaton would be: y x ( ) + ( ) x x1 14

15 Postestmaton: estmatng ntercept/slope y x ( ) + ( ) x x 1 To estmate ntercepts and slopes per school type : gen ntercept _b[_cons] + u gen slope _b[x1] + u1 lst school ntercept slope f school<10 & groups Compare the coeffcents for school 1 above. lst school ntercept slope f school<10 & groups school ntercept slope

16 Postestmaton: fttng values Usng ntercept and slope you can estmate yhat, type gen yhat ntercept + (slope*x1) Or, after xtmxed type: predct yhat_ft, ftted lst school yhat yhat_ft f school<10 & groups. lst school yhat yhat_ft f school<10 & groups school yhat yhat_ft

17 You can plot ndvdual regressons, type Postestmaton: ftted values (graph) twoway connected yhat_ft x1 f school<10, connect(l) Ftted values: xb + Zu Readng test 17

18 After xtmxed you can get the resduals by typng: Postestmaton: resduals predct resd, resduals predct resd_std, rstandard /* resduals/sd(resdual) */ A quck check for normalty n the resduals qnorm resd_std Standardzed resduals Inverse Normal 18

19 DSS Onlne Tranng Secton UCLA Resources Prnceton DSS Lbgudes Books/References Useful lnks / Recommended books / References Beyond Fxed Versus Random Effects : A framework for mprovng substantve and statstcal analyss of panel, tme-seres cross-sectonal, and multlevel data / Brandom Bartels Robust Standard Errors for Panel Regressons wth Cross-Sectonal Dependence / Danel Hoechle, An Introducton to Modern Econometrcs Usng Stata/ Chrstopher F. Baum, Stata Press, 006. Data analyss usng regresson and multlevel/herarchcal models / Andrew Gelman, Jennfer Hll. Cambrdge ; New York : Cambrdge Unversty Press, 007. Data Analyss Usng Stata/ Ulrch Kohler, Frauke Kreuter, nd ed., Stata Press, 009. Desgnng Socal Inqury: Scentfc Inference n Qualtatve Research / Gary Kng, Robert O. Keohane, Sdney Verba, Prnceton Unversty Press, Econometrc analyss / Wllam H. Greene. 6th ed., Upper Saddle Rver, N.J. : Prentce Hall, 008. Introducton to econometrcs / James H. Stock, Mark W. Watson. nd ed., Boston: Pearson Addson Wesley, 007. Statstcal Analyss: an nterdscplnary ntroducton to unvarate & multvarate methods / Sam Kachgan, New York : Radus Press, c1986 Statstcs wth Stata (updated for verson 9) / Lawrence Hamlton, Thomson Books/Cole, 006 Unfyng Poltcal Methodology: The Lkelhood Theory of Statstcal Inference / Gary Kng, Cambrdge Unversty Press,