Multilevel Models Project Based at the Institute of Education Headed by Professor Harvey Goldstein Funded by the ESRC originally through ALCD 3 Full T

Transcription

1 MCMC Estimation of Multilevel Models in the MLwiN software package William Browne Multilevel Models project Institute of Education, London August 2001.

2 Multilevel Models Project Based at the Institute of Education Headed by Professor Harvey Goldstein Funded by the ESRC originally through ALCD 3 Full Time Researchers 2 Lecturers associated with project A Network of Project Fellows

3 Aims of Project Modelling complex structures in social science data Establishing forms of model structure Developing methodology to t models Comparing alternative methodologies Programming methodology into MLwiN Disseminating ideas and training to the social science research community

4 MLwiN Software Package Developed from a chain of packages developed by MMP including ML3, MLn Main Programmer : Jon Rasbash Consists of user friendly Windows interface on top of fast estimation engines Interface written in Visual Basic Estimation engines written in Visual C++

5 MLwiN Software Package Over 2,000 (mainly academic) users Forerunners : Estimation by IGLS and RIGLS only (ML, REML and QL). MLwiN : Additionally allows estimation via MCMC methods and bootstrapping. MCMC algorithms and engine developed by William Browne (and David Draper). Uses IGLS/RIGLS for starting values.

6 Bayesian Statistical Modelling Possible methods of modelling (in increasing complexity) Single Purpose e.g. Code that goes with a particular journal article. Restricted Set of Models e.g. MLwiN, BayesX Restricted General Purpose e.g. WinBUGS General purpose e.g. Fortran, C, C++

7 An Example from Education : The Tutorial dataset Dataset of exam results for 4059 pupils in 65 schools Response is a (normalised) total exam score at age 16 Main predictor is score in a reading test at age 11 We will t a random slopes regression model to data

8 Setting up a Model using IGLS The Equations window is used to set up the model and to display the estimates and likelihood for the model. Estimates change from Blue to Green as model converges. Estimation takes 3 iterations and roughly 1 second on a 733Mhz PC. MLwiN also has screens for calculating residuals and predictions from the above model. Model choice available via Deviance/Likelihood ratio tests and Wald tests.

9 Additional Graphical Features for Multilevel Models Graphical highlighting carries across the graphs. Options available to remove higher level units (for example outliers) from the analysis or absorb them into a dummy variable.

10 MCMC Estimation To change estimation method simply involves pressing a button and setting the MCMC settings. Equations window now contains information about prior distributions.

11 Trajectories Plot Window contains traces for all parameters and the deviance statistic for use with the DIC diagnostic. Estimation is via Gibbs sampling for all parameters. DIC diagnostic for this model suggests that the introduction of random slopes greatly improves the model. ->BDIC Bayesian Deviance Information Criterion (DIC) Dbar D(thetabar) pd DIC Variance components model DIC : Dbar D(thetabar) pd DIC

12 MCMC Diagnostics and Summary statistics The following information is available by clicking on the particular trajectories plot, for example for the school level intercept variance parameter, σ 2 u0 : Here we have a kernel density plot of the posterior distribution, several MCMC convergence diagnostics and summary statistics including quantiles to give a non-symmetric credible interval for the variance parameter.

13 Informative Prior Distributions These can be input in the above window and then appear in the equations window and in the kernel density window on the diagnostics screen.

14 Models Currently Available in MLwiN using MCMC estimation Normal response multilevel models using Gibbs sampling Binomial response multilevel models using a hybrid Random Walk Metropolis-Gibbs sampling algorithm Poisson response multilevel models using a hybrid Random Walk Metropolis-Gibbs sampling algorithm Additional Models Available in development version of MLwiN using MCMC Estimation Multivariate normal response multilevel models with or without missing responses using a Gibbs sampling algorithm. Normal response models with heteroskedasticity at the observation level using a Metropolis-Hastings algorithm for the level 1 variance function. Cross-classified and multiple-membership models for all response types. Mixed Normal and probit link Binomial responses using Gibbs sampling and Metropolis-Hastings for the level 1 variance matrix. Multilevel Factor Analysis for continuous responses using Gibbs sampling and Metropolis-Hastings for any correlations between factors. Classical (Bayesian) Measurement error models for errors in predictors.

15 Multivariate Response Model Here is another example from education that has two responses: a written and coursework component of a Science GCSE examination. The dataset consists of some individuals with only one of the two responses. Here we have fitted an intercept and a gender predictor to our 2 level model. On average girls do 6.8 points better than boys on the coursework component but 2.5 points worse on the written component.

16 Missing Data in Multivariate Models The IGLS algorithm handles multivariate models by fitting them as single level models. If you consider the responses as the lowest level (which has no variability) then by clever use of dummy variables the multivariate is a special case of the univariate problem. In this framework missing data is not a problem. In MCMC all missing data have to be considered as parameters that are estimated at each iteration. This means that we can output estimates for the missing data or use the MCMC algorithm to create imputed datasets (see Schafer s work on this) Here we see the mean estimates for the missing data in the far left column and the original response in the far right column. Missing data code here is 1.

17 Complex Level 1 Variation Consider the tutorial dataset again. We could consider fitting a single level linear regression model to this dataset. Then we could make the variance of the response a function of the intake score. This is commonly known as heteroskedasticity or complex level 1 variation. Here we see that the variance in the response has a quadratic relationship with intake score, with the pupils at the extremes of the intake spectrum, exhibiting greater variability in exam results at age 16.

18 Complex Level 1 Variation To fit this model in MLwiN using MCMC methods (Browne, Draper, Goldstein and Rasbash 2001) we need to use Metropolis Hastings steps for the variance parameters. The worse mixing of the Metropolis Hastings method is evident in the traces above. Here the variance for an individual i is defined as σ 2 ei = e0,0 + e1,0 x i + e1,1 x i 2 where x i is the intake score for individual i.

19 Cross-Classified Models Here is a model for an educational dataset from Fife in Scotland, The response is again exam score at age 16 but for this dataset we know both the primary and secondary school of the children. This notation (introduced in Browne, Goldstein and Rasbash 2001) is based on mapping each individual to one or more classification units and is designed to extend to more complicated multilevel structures. Cross-classified models are hard to fit in the IGLS algorithm as it is optimized for nested structures. In MCMC however you simply click on a button to tell MLwiN the data is cross-classified and the algorithm is no more complex.

20 Mixed Response Multilevel Models As discussed in several papers by Chib and co-authors, mixed Binomial and Normal response models can be fitted by treating the Binomial responses as indicators as to whether a (unknown) continuous response is greater or less than a threshold. To fit such models in MCMC we have an additional step in the algorithm that simulates this continuous response from its full conditional. In the variance matrix at the lowest level we constrain the variances for the Binomial responses to equal 1 and so MH sampling is used for the other variance terms.

21 Multilevel Factor Analysis Modelling Combines multilevel modelling and factor analysis by allowing factors at the different levels of a multivariate Normal multilevel model (See Goldstein and Browne 2001). Care must be taken that the model is identifiable. Factors are input via the following factor screen. Note that if factors are correlated they must be defined at the same level. MLwiN treats the responses as the lowest level and then the individuals as level 2.

22 Multilevel Factor Analysis (cont.) The equations window will contain details of the factors and will give estimates of the loadings (when finished). Currently point estimates and standard errors are available from an output screen from which they are sent to columns on the MLwiN worksheet. It is also possible to store the chains for any or all of the factor values, factor loadings and factor variances.

23 Bayesian Measurement Error Modelling MLwiN allows the user to specify measurement errors in predictor variables. The Equations window includes details of the measurement error model and the MCMC procedure successfully recovers the true predictor estimates (See Browne, Goldstein, Woodhouse and Yang 2001).

24 WinBUGS Interface MLwiN allows the current model to be saved into a WinBUGS input file and can also input WinBUGs output traces to get diagnostic plots WinBUGS file contains model definition, starting values and data taken from the MLwiN worksheet # WINBUGS code generated from MLwiN program #----MODEL Definition model { # Level 1 definition for(i in 1:N) { normexam[i] ~ dnorm(mu[i],tau) mu[i]<- beta[1] * cons[i] + beta[2] * standlrt[i] + u2[school[i],1] * cons[i] + u2[school[i],2] * standlrt[i] } # Higher level definitions for (j in 1:n2) { u2[j,1:2] ~ dmnorm(zero2[1:2],tau.u2[1:2,1:2]) } # Priors for fixed effects for (k in 1:2) { beta[k] ~ dflat() } # Priors for random terms tau ~ dgamma(0.001,0.001) sigma2 <- 1/tau for (i in 1:2) {zero2[i] <- 0} tau.u2[1:2,1:2] ~ dwish(r2[1:2, 1:2],2) for (i in 1:2) { for (j in 1:2) { sigma2.u2[i,j]<-inverse(tau.u2[,],i,j) } } }

25 References Browne, WJ, Draper, D, Goldstein, H and Rasbash, J (2001). Bayesian and likelihood methods for fitting multilevel models with complex level-1 variation. To appear in Computational Statistics and Data Analysis. Browne, WJ, Goldstein, H and Rasbash, J. (2001) Multiple membership multiple classification(mmmc) models. To appear in Statistical Modelling. Browne, W. J., Goldstein, H., Woodhouse, G., and Yang, M. (2001). An MCMC algorithm for adjusting for errors in variables in random slopes multilevel models. Appeared in Multilevel Modelling Newsletter. Goldstein, H. and Browne, W. J. (2001). Multilevel factor analysis modelling using Markov Chain Monte Carlo (MCMC) estimation. To appear in Marcoulides and Moustaki (Eds.), Latent Variable and Latent Structure Models. Web Sites - Project web-site. - site where my publications can be downloaded. - site where a teaching version of MLwiN can be downloaded.