Specifications for this HLM2 run

Transcription

1 One way ANOVA model 1. How much do U.S. high schools vary in their mean mathematics achievement? 2. What is the reliability of each school s sample mean as an estimate of its true population mean? 3. Do schools vary significantly from each other? Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:47:00 Specifications for this HLM2 run Problem Title: One Way ANOVA The data source for this run = HSB.mdm The command file for this run = C:\Documents and Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\One Way ANOVA.hlm Output file name = One Way ANOVA.html The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000 Method of estimation: full maximum likelihood The outcome variable is MATHACH Summary of the model specified Level-1 Model MATHACH ij = β 0j + r ij Level-2 Model β 0j = γ 00 + u 0j Mixed Model MATHACH ij = γ 00 + u 0j + r ij Final Results - Iteration 4

2 Iterations stopped due to small change in likelihood function σ 2 = of σ 2 = τ INTRCPT1,β of τ INTRCPT1,β Random level-1 coefficient Reliability estimate INTRCPT1,β The value of the log-likelihood function at iteration 4 = E+004 Final estimation of fixed effects: INTRCPT2, γ <0.001 γ 00 : Mean math ach (nothing is predicting it at either level) and associated SE. You can use this to get a mean and confidence interval around that mean. Final estimation of fixed effects (with robust standard s) INTRCPT2, γ <0.001 Final estimation of variance components Random Effect Variance Deviation Component χ 2 INTRCPT1, u <0.001 level-1, r

3 u 0: Variance in mean MATHACH between schools. The variance between schools is significant, χ 2 (159) = Schools vary significantly from each other. r ij : Variance in means within school. The total variance around mean math ach is level-1 (within-school) variance + level-2 (between-school) variance. The Intra-class Correlation (ICC) is the proportion of the total variance in math ach that is between groups. This is exactly the same as For the current data, ICC = 8.55/( ) = Eighteen percent of the variance in math achievement is between-schools. Statistics for the current model Deviance = Number of estimated parameters = 3

4 Means as outcome model Predicting the school mean from school SES (MEANSES) 1. Do schools with high MEAN SES also have high math achievement? 2. What amount of between-school variance in math achievement is accounted for by the model with MEAN SES? 3. Do school achievement means vary significantly once MEAN SES is controlled? Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:48:22 Specifications for this HLM2 run Problem Title: Means as Outcome The data source for this run = HSB.mdm The command file for this run = C:\Documents and Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Means as Outcome.hlm Output file name = Means as Outcome.html The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000 Method of estimation: full maximum likelihood The outcome variable is MATHACH Summary of the model specified Level-1 Model MATHACH ij = β 0j + r ij Level-2 Model β 0j = γ 00 + γ 01 *(MEANSES j ) + u 0j Mixed Model MATHACH ij = γ 00 + γ 01 *MEANSES j + u 0j + r ij Final Results - Iteration 6

5 Iterations stopped due to small change in likelihood function σ 2 = of σ 2 = τ INTRCPT1,β of τ INTRCPT1,β Random level-1 coefficient Reliability estimate INTRCPT1,β The value of the log-likelihood function at iteration 6 = E+004 Final estimation of fixed effects: INTRCPT2, γ <0.001 MEANSES, γ <0.001 γ 00 : Mean Math ach when MEANSES = 0. γ 01: Effect on Math Ach of a one unit change in MEANSES Final estimation of fixed effects (with robust standard s) INTRCPT2, γ <0.001 MEANSES, γ <0.001 Final estimation of variance components Variance Random Effect χ Deviation Component 2 INTRCPT1, u <0.001

6 level-1, r Note the reduction in between school variance, u 0, now that we are accounting for some of it with MEANSES. At the between-school level, β 0, or mean MATHACH, is equal to some initial value plus an effect of MEANSES, or in equation form: β 0 = γ 00 + γ 01 MeanSES + u 0 We are accounting for some of the between-school unknown variance, u 0, with MEANSES. To calculate just how much of that unknown variance we are accounting for, we calculate the proportion by which the initial variance was reduced when we added MEANSES. Initial /unknown variance from previous model = 8.55 Unknown variance when MEANSES is in the model = 2.59 Reduction in variance = = 5.96 Proportion reduction in variance = 5.96/8.55 =.697 MEANSES accounted for 69.7% of the variance in mean MATHACH (β 0 ) It is important to be clear and recall that 18% of the TOTAL variance was at the between-school level (see calculation of ICC above). In other words, recall that u 0 accounted for 18% of the total variance. So we are accounting for 69.7% of that 18% with MEANSES. Statistics for the current model Deviance = Number of estimated parameters = 4 Model comparison test χ 2 statistic = Degrees of freedom = 1 = <0.001 The model comparison test (difference between deviance in current model and previous model) indicates that this is a better fitting model than the previous one.

7 Contextual Effects Model (Grand Mean Centered) Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:50:43 Specifications for this HLM2 run Problem Title: Contextual Effects Model (Grand Mean) The data source for this run = HSB.mdm The command file for this run = C:\Documents and Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Contextual Effects Model (Grand Mean).hlm Output file name = Contextual Effects Model (Grand Mean).html The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000 Method of estimation: full maximum likelihood The outcome variable is MATHACH Summary of the model specified Level-1 Model MATHACH ij = β 0j + β 1j *(SES ij ) + r ij Level-2 Model β 0j = γ 00 + γ 01 *(MEANSES j ) + u 0j β 1j = γ 10 Note that we turned off the level-2 variance in the SES effect (the Slope). This is saying that the slope, or the effect of SES, is the same across all the schools. In other words, no matter what school you are in, a one unit change in individuals SES is equal to a 2.19 unit change in MATHACH (see fixed effects) SES has been centered around the grand mean. SES is centered around the mean for the entire sample of data; this means that it is zero at the mean for the entire sample. So the β 0j is now the mean math ach at mean individual SES for the sample (when SES = 0).

8 MEANSES has been centered around the grand mean. Likewise, MEANSES (each school s average SES) has been centered around the mean for the sample. So β 0j is mean math ach at the mean for schools SES. Mixed Model MATHACH ij = γ 00 + γ 01 *MEANSES j + γ 10 *SES ij + u 0j + r ij You can see it more clearly in the mixed model equation. β 0j which is γ 00 = mean Math Ach when MEANSES and SES are equal to 0. Final Results - Iteration 6 Iterations stopped due to small change in likelihood function σ 2 = of σ 2 = τ INTRCPT1,β of τ INTRCPT1,β Random level-1 coefficient Reliability estimate INTRCPT1,β The value of the log-likelihood function at iteration 6 = E+004 Final estimation of fixed effects: INTRCPT2, γ <0.001 MEANSES, γ <0.001 For SES slope, β 1 INTRCPT2, γ <0.001 The contextual effect with grand-mean centering of level-1 variable:

9 Let s take a look at the model. MATHACH = γ 00 + γ 01 MEANSES + γ 10 SES + u0+ r The effect of SES on math achievement is defined by two sources. A source at the individual level meant to represent his or her own individual circumstances (SES: family income, occupation, etc.), and a source at the school level meant to represent the resources of a particular school (MEANSES: mean of SES variable in each particular school). This SES effect is captured jointly by two coefficients in the model: γ 01 and γ 10. Despite that there are two coefficients in the model that capture the effect of SES, there are actually three effects to consider. There is the effect of the moving from a school with one SES to a school with a different SES, which is β b. But this effect is actually composed of the effect of individuals SES on math achievement (β w ) and the effect of schools, on the contexts, effects on math achievement (β c ). The coefficient for the variable at the individual level (γ 10 ), SES, is always going to be the β w effect. When we grand mean center SES, what we capture with γ 01 is the effect of schools mean SES on math achievement, controlling for the effect of individual SES. This coefficient γ 01 quantifies the effect of being in a particular context, namely a particular school with a particular mean SES. Thus, γ 01 IS the contextual effect when the individual level component of that effect is grand-mean centered. Together, both of these coefficients tell us about the effect of SES on math achievement in schools (the cumulative β b effect.) Sometimes, in using a grand mean centered approach, it can be a bit more intuitive to think about the effects of context controlling for the effects of individual. Finally, let s look exactly at what these coefficients tell us. If Kid A and Kid B are in the same school, and Kid A s SES is greater than Kid B s SES by one unit, then Kid A s MATHACH is greater than Kid B s MATHACH by 2.19 (γ 10 ). If Kid A and Kid B have the same individual SES, but Kid A is in a different school that is greater by one unit of MEANSES, then Kid A s MATHACH is 3.67 (γ 01 ) greater than Kid B s MATHACH. Final estimation of fixed effects (with robust standard s) INTRCPT2, γ <0.001 MEANSES, γ <0.001 For SES slope, β 1 INTRCPT2, γ <0.001

10 Final estimation of variance components Random Effect Variance Deviation Component χ 2 INTRCPT1, u <0.001 level-1, r Statistics for the current model Deviance = Number of estimated parameters = 5 Contextual Effects Model (Group Mean Centered) Module: HLM2 (7.00) Date: Jun 15, 2011 Time: 05:51:39 Specifications for this HLM2 run Problem Title: Contextual Effects Model (Group Mean) The data source for this run = HSB.mdm The command file for this run = C:\Documents and Settings\manningma\Dropbox\Teaching\HLM Workshop 2011\HSB\Contextual Effects Model (Group Mean).hlm Output file name = Contextual Effects Model (Group Mean).html The maximum number of level-1 units = 7185 The maximum number of level-2 units = 160 The maximum number of iterations = 1000 Method of estimation: full maximum likelihood The outcome variable is MATHACH Summary of the model specified Level-1 Model MATHACH ij = β 0j + β 1j *(SES ij ) + r ij

11 Level-2 Model β 0j = γ 00 + γ 01 *(MEANSES j ) + u 0j β 1j = γ 10 SES has been centered around the group mean. SES has been centered around the mean for each school. So SES is equal to 0 at the mean for each school. It might not be the same value for each school, but in the model it has the same meaning for each school. MEANSES has been centered around the grand mean. Mixed Model MATHACH ij = γ 00 + γ 01 *MEANSES j + γ 10 *SES ij + u 0j + r ij Final Results - Iteration 6 Iterations stopped due to small change in likelihood function σ 2 = of σ 2 = τ INTRCPT1,β of τ INTRCPT1,β Random level-1 coefficient Reliability estimate INTRCPT1,β The value of the log-likelihood function at iteration 6 = E+004 Final estimation of fixed effects: INTRCPT2, γ <0.001 MEANSES, γ <0.001 For SES slope, β 1 INTRCPT2, γ <0.001

12 The contextual effect with group-mean centering of level-1 variable: The model is exactly the same: MATHACH = γ 00 + γ 01 MEANSES + γ 10 SES + u0+ r The only thing that we have changed is how we choose to center the within-unit variable, SES. We have group mean centered it. When we group mean center, coefficient γ 01 is now the cumulative β b effect. So to get the effect of context, we subtract the individual level effect, γ 10, from γ 01. This underscores the fact that you have to be mindful of your choice of centering when it comes to introducing effects that exist at both the aggregate and the individual levels, and further underscores the care needed in interpreting your coefficients in these kinds of models. Final estimation of fixed effects (with robust standard s) INTRCPT2, γ <0.001 MEANSES, γ <0.001 For SES slope, β 1 INTRCPT2, γ <0.001 Final estimation of variance components Random Effect Variance Deviation Component χ 2 INTRCPT1, u <0.001 level-1, r Statistics for the current model Deviance = Number of estimated parameters = 5