HLM: A Gentle Introduction

Transcription

1 HLM: A Gentle Introduction D. Betsy McCoach, Ph.D. Associate Professor, Educational Psychology Neag School of Education University of Connecticut Other names for HLMs Multilevel Linear Models incorporate higher level predictors into the analysis. Raudenbush and Bryk (2002) call these HLMs Intercepts and slopes as outcomes Trying to model or explain variability in level one intercepts and slopes, based on group level characteristics Also referred to as contextual models, mixed effects models, random effects models, variance components models The overall model is considered to be a linear model. 2 1

2 Clustering Research has consistently demonstrated that people within a particular group or context tend to be more similar to each other in terms of an outcome variable than they are to people in a different group or context. Statistically, we need to use techniques that consider this dependence of the outcome variable between people from the same context Repeated observations share this same phenomena (correlated observations) 3 Nesting Problem Non independence Students nested within classrooms or schools are typically more homogeneous than observations from a non nested tdstudy. This correlation or dependency (ICC or rho=ρ) violates the assumption of independence necessary for traditional statistics. Even mild violations can lead to severe problems with Type I error (yp (typically y inflated). Kish (1965): Cluster effect 2

3 Intra class correlation Tendency for values of a variable within a cluster to be correlated ltdamong themselves relative to the values outside the cluster The average correlation between variables within the same higher level unit will be higher than the average correlation between variables for students from different higherlevel units 5 How big an ICC is problematic? the degree to which the standard error is affected depends d on both ththe ICC and the number of people per cluster 3

4 Design Effect: DEFT d = 1 + ( n 1) ρ j With clustered data, the ICC is positive and the standard error (which assumes SRS) is underestimated. DEFT shows how much the standard error needs to increase as we move from a simple random sample to a cluster sample of size n. Multiply SE by the DEFT 7 DEFT d = 1 + ( n 1) ρ j The bigger the cluster, the bigger the DEFT. The bigger the ICC, the bigger the DEFT. But even small ICC s can have large consequences. The DEFT for an ICC of.05 with a cluster size of 100 is 2.44 = 1 + (100 1)

5 DEFT as a function of Nj (ICC held constant at.15) 6 DEFT Nj 9 DEFT as a function of the ICC (Nj held constant at 25) DEFT ICC 10 5

6 Historical Approaches 1. Disaggregate the data (students from same school get same value for the macrolevel variable) Violates assumption of independence. Inflated type I errors. 2. Aggregate the data (use school means as the outcome) Not measuring the same thing once aggregated Analyzing group data, but formulating conclusions as if were at the individual level! Robinson effect, also called Ecological fallacy 11 HLM Solution Consider effects at both levels simultaneously with a level 1 and a level 2 model! dl! 12 6

7 Multilevel Research A multilevel data structure allows for an understanding of how person level variables (e.g., gender, behavior, attitudes, knowledge, general health) as well as group level variables (e.g., characteristics of classrooms or teachers, neighborhoods, families) can be used to explain an outcome of interest (measured at the lowest level of the analysis) A hierarchical system, with interactions between the individual and group (or context ) characteristics Can also reframe longitudinal models as multilevel, but for today we will stick with organizational models 13 Reasons for using HLM Addresses the unit of analysis question Canresolve aggregationbias issues Enhances precision of estimates over non hierarchical methods More powerful models for individual growth Individual growth curves Allows us to model variability across contextsts Allows us to partition variance across the levels of analysis 7

8 Modeling the Dependency Accounting for homogeneity within groups allows for an understanding of how group level effects could impact the outcome of interest Do group level variables moderate the effect of an independent variable on Y? Conditional Models Cross level effects, aka cross level interactions HLM provides a methodology for connecting the two (or three, etc.) levels together. Two level structure Children, nested within schools S1... S2 S3 S j 8

9 Three level structure When groups of schools belong to certain districts: District 1 District 2... S1... S2 S3 S j District K 25 schools: Yˆ = b + bses 0 1 Public are blue (0) Private are red (1) SECTOR = 0 SECTOR = MAT THACH SES 18 9

10 How are public and private schools different? Yˆ = b + bses 0 1 MATHACH SES SECTOR = 0 SECTOR = 1 Intercept: The level is higher in the private schools Slope: The slope is flatter in private schools. The slope is steeper in public schools. This means that the relationship between SES and math achievement is STRONGER in public schools (where the slope is steeper) and WEAKER in private schools (where slope is flatter). A completely flat slope would indicate NO relationship between SES and math achievement. 19 Organizational Context Ask questions about how effects at level 2 affect the outcome of interest t Graphs illustrate how context the public versus private school distinction can be important to understanding differences in the relationships between SES and Math Achievement 20 10

11 Level 1: HSB 160 level 1 equations, represented by: Y = β ( ) 0 + β1 SES + r ij j j ij ij For each school, the intercept represents the best estimate for prediction of math achievement, when SES is 0 For each school, slope represents the effect of SES on math achievement For each school, the error term r ij represents deviation for each student from the fitted model ( Y Yˆ ) 21 Level 2: HSB 2 level 2 equations One equation models the variability/differences in the 160 intercepts One equation models the variability/differences in the 160 SES slopes β = γ + γ (Sector) + u β = γ + γ (Sector) + u 0 j j 0 j 1j j 1j 22 11

12 Tau Matrix (example: 2x2) T contains variances and covariances for the (randomly varying) intercepts and slopes. T var( u0 j) cov( u0 j, u1j) = cov( u, u ) var( u ) 0 j 1j 1j τ τ = τ10 τ Combining terms Y = γ + γ SES + γ Sector + γ SES Sector ij ij 01 j 11 ij j + u + u SES + r 0 j 1j ij ij Our goals for today how do we develop and interpret this model? 24 12

13 High School and Beyond Data n = 7185 students, nested within J = 160 schools (approx. 45 students each school). Y = mathematics achievement X = student (family) SES W 1 = sector Public school (0) Catholic school (1) 25 Models we will build Our examples (printouts) treat SES as grand mean centered Model 1: One way Random Effects ANOVA model Null model Model 2: Random Coefficients Model (Includes SES at level 1) Model 3: Contextual Model (Includes Sector at level 2) 26 13

14 One way ANOVA with Random Effects Level 1 Level 2 Y β = β + r ij 0 j ij = γ + u 0 j 00 0 j Combined Model Y = γ + u + r ij 00 0 j ij 27 Partitioning variance. 2 VarY = Var ( u + r ) = τ + σ ij 0 j ij 00 The total variability in the dependent variable can be separated into two pieces: that which lies between clusters (τ 00 ) and that which is within clusters σ

15 Intra class correlation Var( u0 j ) τ 00 ICC = = Var ( u + r ) τ + σ 0 j ij 00 The proportion of the variance in the outcome that is between the level 2 units. Proportion of variance explained by the grouping/clustering structure (Hox, 2002) Expected correlation between two randomly chosen units within the same cluster (Hox, 2002) 29 2 Intra class correlation Tendency for values of a variable within a cluster to be correlated ltdamong themselves relative to the values outside the cluster The average correlation between variables within the same higher level unit will be higher than the average correlation between variables for students from different higherlevel units 30 15

16 Random Coefficients Model Level 1 Y = β ( ) 0 + β 1 SES + r β = γ + u Level 2 ij j j ij ij β 0j 00 0j = γ + u 1j 10 1j Combined Y = γ + γ ( X ) + u + u ( X ) + r ij ij 0 j 1 j ij ij 31 Random Coefficients Regression Models Unconditional level-2 model: No predictors at level-2 Both intercepts and slopes are allowed to vary across groups Variance covariance components: intercept variance slope variance intercept slope p covariance Level one variance Person level, r ij 32 16

17 Characterizing Variability Assuming a random coefficients model (no W s, and here with grand mean centering): β 0j = intercept (meanoutcome) a student ofaverage SES for the j th school β 1j = overall effect of SES within the j th school If Y is achievement: Variability in the intercepts reflects variability in achievement across schools holding SES constant at 0 (τ 00 ) Variability in slopes reflects differences in effect of SES across schools (τ qq ) Each school described by (β 0j, β 1j, β 2j, β qj ) 33 Slope and Intercept Variability For random coefficients model (no level 2 predictors) E(β 0j ) = γ 00 and Var(β 0j ) = τ 00 E(β qj ) = γ qj and Var (β qj ) = τ qq Cov (β 0j, β qj ) = τ 0q There are q+1 variances, and as many covariances as there are pairs of effects in the model. A positive covariance means that schools with high means (intercepts) also have positive slopes (for that variable). Can school level effects explain the variability (variances and covariances)? 34 17

18 Tau Matrix (example: 2x2) One predictor at level 1; thus, T contains variances and covariances for the (randomly varying) intercepts and slopes T var( u0 j) cov( u0 j, u1j) = cov( u0 j, u1j) var( u1j) τ τ = τ10 τ Proportion reduction in variance at level 1 Variance explained at level 1 (within schools) )by the set of variables ibl R & B p. 79 pve σ σ 2 2 B F _1= 2 σ B 36 18

19 Intercepts and Slopes as Outcomes The most general form of multilevel model Model lboth ththe intercepts t and slopes with explanatory variables at level 2 Trying to account for (reduce) variance in the intercepts and slopes, based on contextual characteristics Cross level interactions does the effect of a contextual variable moderate the impact of a level 1 variable on the dependent variable? 37 Using context to predict the intercept AND the slope Level 1 Y = β0 + β1 ( SES ) + r ij j j ij ij Level 2 β = γ + γ ( Sector) + u 0 j j 0 j β = γ + γ ( Sector) + u 1j j 1j 38 19

20 Combining terms Y = γ + γ SES + γ Sector + γ SES Sector ij ij 01 j 11 ij j + u + u SES + r 0 j 1j ij ij SECTOR = 0 SECTOR = H MATHACH SES 39 Proportion reduction in variance at level 2 Variance explained in intercept at level 2 (between schools) )by the set of variables ibl R & B p. 74 pve τ τ 00 B 00 F _ β0 j = τ 00B 40 20

21 Proportion reduction in variance at level 2 Variance explained in slope at level 2 (between schools) )by the set of variables ibl R & B p. 74 pve τ τ 11 B 11 F _ β1 j = τ11b 41 Error Terms in Level 2 u 0j and the u qj are now the residuals for each estimated level 1 effect (intercepts, slopes) (i.e., the level 2 models). The variance for these errors represents the variance remaining in the level 1 intercept and slopes after controlling (accounting) for the W s. Not independent within schools, because the level 2 errors are common to every student in a particular school These errors vary across schools, and depend on the level 2 IV s which vary across schools

22 Variance Components Var(u 0j ) = (τ 00 ) Var(u qj ) = (τ qq ) Cov(u 0j, u qj ) = τ 0q For competing models, reducing the variability of these error terms implies that the level 2 (group or context) variables provides some explanation of the dependent variable. Consider choosing the model that has best explanation (reduction) of variance 43 References My favorite books Hox, J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum Associates. O Connell, A. A., & McCoach, D. B. (Eds.) (2008). Multilevel modeling of educational data. Charlotte, NC: Information Age Publishing. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models (2 nd ed.). Thousand Oaks, CA: Sage Publications, Inc. Singer, J. D., & Willet, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford. Snijders, T. A., & Bosker, R. J. (1999). Multilevel analysis. Thousand Oaks, CA: Sage. 22