Illustration (and the use of HLM)

Transcription

1 Illustration (and the use of HLM) Chapter 4 1 Measurement Incorporated HLM Workshop

2 The Illustration Data Now we cover the example. In doing so we does the use of the software HLM. In addition, we will discuss typical questions that can be answered. 2 Measurement Incorporated HLM Workshop

3 The Illustration Data So the data set consists of 7,185 students who are nested in schools (there are 160 schools). Variables that we will use: MATHACH (Math Achievement at student level). CSES (group centered SES at student level). MEANSES (Average School SES, school level). l) SECTOR (Public 0 versus Catholic 1, school level). 3 Measurement Incorporated HLM Workshop

4 Student Label School Label Level-1 Variables Level-2 Variables

5 The Questions 1. How much do U.S. high schools vary in their mean math achievement? 2. Do schools with high MEANSES also have high math achievement? 3. Is the strength of association between student CSES and math achievement similar across schools? s? Or is CSES S a better predictor of student math achievement in some schools than others? 4. How do public and Catholic schools compare in terms of mean math achievement and in terms of the strength of the SES-math achievement relationship, after we control for MEAN SES? 5 Measurement Incorporated HLM Workshop

6 1 st Question 1. How much do U.S. high schools vary in their mean math achievement? Notice that this question only addresses the variance of the observations, We will use a random effects ANOVA, 6 Measurement Incorporated HLM Workshop

7 1 st Question We assume that u 0j has a normal distribution with mean 0 and variance τ 00 School j s mean MATHACH β γ + 0 = γ + u 0 j 00 0 j MATHACH β + ij = 0 j r ij Here all we want to know is the amount of variability of the mean math achievement across the schools 7 Measurement Incorporated HLM Workshop

8 HLM The program does allow one to keep two separate files in the case of a 2 level analysis, SO in our case we could have a file for each examinee and a file for school, You will need to give the ID variable (school) in both files, Here we have it as an SPSS file and so we will just use it. 8 Measurement Incorporated HLM Workshop

9 HLM To begin using HLM open the program and click on file This will pop-up a question about what format of dt data you are using At this point we want to start a new analysis so click on Make a new MDM file 9 Measurement Incorporated HLM Workshop

10 HLM This brings up an option window. For now we only discuss how to do a two level analysis so click on HLM2 10 Measurement Incorporated HLM Workshop

11 HLM Pick variables Pick variables Pick the SPSS file that you are using Pick the SPSS file that you are using 11 Measurement Incorporated HLM Workshop

12 HLM Then save the mdmt file, a control file Add name of data file Finally, click on Make MDM file, completes your set up. Examine basic statistics file Click Done 12 Measurement Incorporated HLM Workshop

13 HLM That will get you to this part where you define your model. 13 Measurement Incorporated HLM Workshop

14 HLM Output The output for HLM is given as a txt file This file can be opened from HLM Here I will just provide some of the main parts 14 Measurement Incorporated HLM Workshop

15 Summary Information This gives us the random components 15 Measurement Incorporated HLM Workshop

16 Fixed Effects Fixed effect for our model (intercept,,γγ 00 ) 16 Measurement Incorporated HLM Workshop

17 Random Effects Same random components as before only now we have SE and significance tests 17 Measurement Incorporated HLM Workshop

18 Estimates Think back to the original question. 1. How much do U.S. high schools vary in their mean math achievement? We have a parameter that measured that for us, which was τ 00 =8.61. Also, the average e achievement ement across all school averages (i.e., the grand mean). γ 00 = Measurement Incorporated HLM Workshop

19 Range of School Averages Given the mean and its variance I can even compute a confidence interval to describe a range that includes 95% of all schools average math achievement. ˆ γ ± 1.96( ˆ τ ) ± 1.96(8.61) 2 = (6.89,18.39) 19 Measurement Incorporated HLM Workshop

20 CI for Grand Mean We can also put a CI around the our estimate of the grand mean using the same equation. Only now we use the standard error of our estimate for γ 00 1 ± ± 1.96(.24) = (12.17,13.11) ) 20 Measurement Incorporated HLM Workshop

21 Interclass Correlation We can also compute the intraclass correlation. ˆ τ 8.61 ˆ τ 00 + σ ρ = = = Measurement Incorporated HLM Workshop

22 Reliability Lastly we can compute the reliability of our estimate of School j s mean. ˆ ( ) ˆ00 λ j = reliability( Y. j) = ( 2 ˆ τ ) 00 + ( σ / n j ) τ 22 Measurement Incorporated HLM Workshop

23 Reliability If we compute this value for all schools and average it we will get a summary of reliability which is the same that is given in the HLM output. 23 Measurement Incorporated HLM Workshop

24 2 nd Question 1. Do schools with high MEANSES also have high math achievement? Focus is how a level-2 variable effects math achievement (average math achievement). Means as outcomes model. 24 Measurement Incorporated HLM Workshop

25 2 nd Question MATHACH β + ij = 0 j r ij β 0 j = γ γ01meanses u0 j 25 Measurement Incorporated HLM Workshop

26 HLM 26 Measurement Incorporated HLM Workshop

27 Model Summary Information 27 Measurement Incorporated HLM Workshop

28 Results (Fixed and Random) The estimate of our effect for MEANSES, γ 01 Hypothesis test Here are our variance This tells us that t there are still components differences across schools and that we do need the random effect

29 Additional Results There are other things we can compute. For example, we can compute the expected range of school means adjusted for MEANSES. 1 1 γˆ 1.96 ˆ ± τ00 = ± = ( ) ( ) ( 9.47,15.83) 29 Measurement Incorporated HLM Workshop

30 Additional Results We can also compute the proportion of variance that can be explained by MEANSES using. τˆ = = ( ANOVA) τˆ ( MEANSES) τˆ00 ( ANOVA) Measurement Incorporated HLM Workshop

31 Additional Results Lastly we can still compute the intraclass correlation as before. Now it is conditional on MEANSES. A measure of dependence within school given that we are now accounting for MEANSES. τˆ 2.64 ρ = 00 = =.06 τˆ +σ Measurement Incorporated HLM Workshop

32 3 rd Question 1. Is the strength of association between student CSES and math achievement similar across schools? Or is CSES a better predictor of a students math achievement in some schools than others? Now the focus is on the effect of a level-1 variable on the dependent variable. Also how this effect various across schools. 32 Measurement Incorporated HLM Workshop

33 3 rd Question That means that we are interested in: MATHACH = β0 + β1 ( CSES ) + r β β = γ + u 0 j 00 = γ + u j j i ij oj 1j 10 1j 33 Measurement Incorporated HLM Workshop

34 HLM 34 Measurement Incorporated HLM Workshop

35 HLM Output Notice now that we have two level-2 random effects we can estimate each variance and their covariance, this is their covariance matrix 35 Measurement Incorporated HLM Workshop

36 HLM Output 36 Measurement Incorporated HLM Workshop

37 Average Regression Line Can determine the average regression line across school based on the level-2 fixed effects. β = γ + u 0 00 β j = γ + u oj 1 j 10 1 j So the average line is: MATHACH = ( CSES) 37 Measurement Incorporated HLM Workshop

38 Variability of Line Can determine variability of the regression lines across schools. Based on variance components (or random effects) τ 00 and τ ± 1.96(8.68) = 6.87, ( ) (0.68) 68) 0.57,3.81 ± 2 = ( ) 38 Measurement Incorporated HLM Workshop

39 Relationships of Coefficients Can interpret the relationship between average math achievement with in a school and the relationship between CSES and MATHACH. Looking at τ Measurement Incorporated HLM Workshop

40 Additional Results Lastly, we could look at the proportion of error, at level 1, that can be explained by CSES. Because we are at level-1 we use σ σ ( ANOVA ) σ ( CSES ) = =.063 σ 2 ( CSES ) Remember that the school MEANSES accounted for 60% 40 Measurement Incorporated HLM Workshop

41 4 th Question 4. How do public and Catholic schools compare in terms of mean math achievement and in terms of the strength of the SES-math achievement relationship, after we control for MEAN SES? 41 Measurement Incorporated HLM Workshop

42 4 th Question Do MEANSES and SECTOR significantly predict the intercept. Do MEANSES and SECTOR significantly predict the slope. How much variation in the intercept and slope is explained by MEANSES and SECTOR. 42 Measurement Incorporated HLM Workshop

43 HLM 43 Measurement Incorporated HLM Workshop

44 HLM Output 44 Measurement Incorporated HLM Workshop

45 Fixed Effects 45 Measurement Incorporated HLM Workshop

46 Random Effects 46 Measurement Incorporated HLM Workshop

47 HLM Output Lastly, I could, as shown on page 85, compute the amount of variability reduced, compared to the random regression line discussed previously. 47 Measurement Incorporated HLM Workshop

48 Summary The purpose this morning was to get you familiar with terminology, notation, and the types of tests and conclusions we can make with HLM. Are there any questions? 48 Measurement Incorporated HLM Workshop