Mixed effects modeling

Mixed effects modeling Generalising to the universe with random item and subject selection Davide Crepaldi MoMo Lab, Department of Psychology University of Milano Bicocca, Italy www.davidecrepaldi.net davide.crepaldi1@unimib.it Spring 2013 Davide Crepaldi 1 / 123

Part II Mixed effects models Davide Crepaldi 45 / 123

Outline of part II 3 Point prediction 4 5 Model fitting Model selection Parameter testing 6 Davide Crepaldi 46 / 123

A new way of thinking about problems Una serie di problemi, in rigoroso ordine di importanza Mangiare carne rossa determina un incremento del rischio di cancro all intestino. Un consumo moderato di alcool (2 bicchieri vino la settimana) durante la gravidanza abbassa il QI del nascituro Un consumo moderato di alcool (2 bicchieri vino la settimana) durante la gravidanza abbassa il QI del nascituro di circa 2 punti Davide Crepaldi 47 / 123

The classic approach Populations to be compared Sample data, to be generalized safely to populations Compare means Sample stats distribution in random sampling Davide Crepaldi 48 / 123

Mixed effect modeling How is it that any given datapoint is such? y ij = β 1 x 1 + β 2 x 2 +... + β n x n + Ss i + Ww j + ɛ i j (4) Davide Crepaldi 49 / 123

An example Suppose you have a dataset with three participants s1, s2, and s3 who each saw three words w1, w2, and w3 in a reading experiment. This is a priming experiment, and each word was tested under a short and a long SOA condition with each participant. Davide Crepaldi 50 / 123

An example ####### 500 ms dealer 30-55 ms DEAL 1500 ms press button Davide Crepaldi 51 / 123

Tabulate the data Davide Crepaldi 52 / 123

Tabulate the data Short SOA w1 w2 w3 s1 475 494 490 s2 491 544 526 s3 484 529 539 Davide Crepaldi 53 / 123

Tabulate the data Short SOA w1 w2 w3 s1 475 494 490 s2 491 544 526 s3 484 529 539 Long SOA w1 w2 w3 s1 466 520 502 s2 516 566 577 s3 470 511 528 Davide Crepaldi 54 / 123

Tabulate the data sbj word SOA RT s1 w1 long 466 s1 w2 long 520 s1 w3 long 502 s1 w1 short 475 s1 w2 short 494 s1 w3 short 490... Davide Crepaldi 55 / 123

Tabulate the data sbj word SOA RT Int s1 w1 long 466 522.2 s1 w2 long 520 522.2 s1 w3 long 502 522.2 s1 w1 short 475 522.2 s1 w2 short 494 522.2 s1 w3 short 490 522.2... Davide Crepaldi 56 / 123

Tabulate the data sbj word SOA RT Int SOA s1 w1 long 466 522.2 0 s1 w2 long 520 522.2 0 s1 w3 long 502 522.2 0 s1 w1 short 475 522.2-19 s1 w2 short 494 522.2-19 s1 w3 short 490 522.2-19... Davide Crepaldi 57 / 123

Tabulate the data sbj word SOA RT Int SOA WordInt s1 w1 long 466 522.2 0-28.3 s1 w2 long 520 522.2 0 14.2 s1 w3 long 502 522.2 0 14.1 s1 w1 short 475 522.2-19 -28.3 s1 w2 short 494 522.2-19 14.2 s1 w3 short 490 522.2-19 14.1... Davide Crepaldi 58 / 123

Tabulate the data sbj word SOA RT Int SOA WordInt SubInt s1 w1 long 466 522.2 0-28.3-26.2 s1 w2 long 520 522.2 0 14.2-26.2 s1 w3 long 502 522.2 0 14.1-26.2 s1 w1 short 475 522.2-19 -28.3-26.2 s1 w2 short 494 522.2-19 14.2-26.2 s1 w3 short 490 522.2-19 14.1-26.2... Davide Crepaldi 59 / 123

Tabulate the data sbj word SOA RT Int SOA WordInt SubInt SubSOA s1 w1 long 466 522.2 0-28.3-26.2 0 s1 w2 long 520 522.2 0 14.2-26.2 0 s1 w3 long 502 522.2 0 14.1-26.2 0 s1 w1 short 475 522.2-19 -28.3-26.2 11 s1 w2 short 494 522.2-19 14.2-26.2 11 s1 w3 short 490 522.2-19 14.1-26.2 11... Davide Crepaldi 60 / 123

Tabulate the data sbj word SOA RT Res Int SOA WordInt SubInt SubSOA s1 w1 long 466 522.2 0-28.3-26.2 0-2 s1 w2 long 520 522.2 0 14.2-26.2 0 9.8 s1 w3 long 502 522.2 0 14.1-26.2 0-8.2 s1 w1 short 475 522.2-19 -28.3-26.2 11 15.4 s1 w2 short 494 522.2-19 14.2-26.2 11-8.4 s1 w3 short 490 522.2-19 14.1-26.2 11-11.9... Davide Crepaldi 61 / 123

More than one X sbj word SOA Lan RT Int SOA Lan s1 w1 long L1 466 522.2 0 0... s1 w2 long L1 520 522.2 0 0 s1 w3 long L2 502 522.2 0 8.3 s1 w1 short L1 475 522.2-19 0 s1 w2 short L1 494 522.2-19 0 s1 w3 short L2 490 522.2-19 8.3... Davide Crepaldi 62 / 123

Interaction sbj word SOA Lan SOA Lan RT Int SOA Lan SOA Lan s1 w1 long L1 0 466 522.2 0 0 0... s1 w2 long L1 0 520 522.2 0 0 0 s1 w3 long L2 0 502 522.2 0 8.3 0 s1 w1 short L1 0 475 522.2-19 0 0 s1 w2 short L1 0 494 522.2-19 0 0 s1 w3 short L2 1 490 522.2-19 8.3 7.5... Davide Crepaldi 63 / 123

Continue X sbj word SOA Freq RT Int SOA Freq s1 w1 long.67 466 522.2 0-8.32... s1 w2 long 1.12 520 522.2 0-8.32 s1 w3 long.02 502 522.2 0-8.32 s1 w1 short.67 475 522.2-19 -8.32 s1 w2 short 1.12 494 522.2-19 -8.32 s1 w3 short.67 490 522.2-19 -8.32... Davide Crepaldi 64 / 123

Non linear effects sbj word SOA Freq Freq 2 RT Int SOA Freq Freq 2 s1 w1 long.67.45 466 522.2 0-8.32-1.34... s1 w2 long 1.12 1.25 520 522.2 0-8.32-1.34 s1 w3 long.02.0004 502 522.2 0-8.32-1.34 s1 w1 short.67.45 475 522.2-19 -8.32-1.34 s1 w2 short 1.12 1.25 494 522.2-19 -8.32-1.34 s1 w3 short.02.004 490 522.2-19 -8.32-1.34... Davide Crepaldi 65 / 123

Fixed and random sbj word SOA RT Res Int SOA WordInt SubInt SubSOA s1 w1 long 466 522.2 0-28.3-26.2 0-2 s1 w2 long 520 522.2 0 14.2-26.2 0 9.8 s1 w3 long 502 522.2 0 14.1-26.2 0-8.2 s1 w1 short 475 522.2-19 -28.3-26.2 11 15.4 s1 w2 short 494 522.2-19 14.2-26.2 11-8.4 s1 w3 short 490 522.2-19 14.1-26.2 11-11.9... Davide Crepaldi 66 / 123

Fixed and random sbj word SOA RT Fixed Random Res Int SOA WordInt SubInt SubSOA s1 w1 long 466 522.2 0-28.3-26.2 0-2 s1 w2 long 520 522.2 0 14.2-26.2 0 9.8 s1 w3 long 502 522.2 0 14.1-26.2 0-8.2 s1 w1 short 475 522.2-19 -28.3-26.2 11 15.4 s1 w2 short 494 522.2-19 14.2-26.2 11-8.4 s1 w3 short 490 522.2-19 14.1-26.2 11-11.9... Davide Crepaldi 67 / 123

Fixed and random Fixed effects Effects of interest Level selection NOT random Interest in estimating effect size Random effects Influence Y, but not of interest Level selection IS random Interest in estimating variability Davide Crepaldi 68 / 123

Random intercept and random slope sbj word SOA RT Fixed Random Res Int SOA WordInt SubInt SubSOA s1 w1 long 466 522.2 0-28.3-26.2 0-2 s1 w2 long 520 522.2 0 14.2-26.2 0 9.8 s1 w3 long 502 522.2 0 14.1-26.2 0-8.2 s1 w1 short 475 522.2-19 -28.3-26.2 11 15.4 s1 w2 short 494 522.2-19 14.2-26.2 11-8.4 s1 w3 short 490 522.2-19 14.1-26.2 11-11.9... Davide Crepaldi 69 / 123

Random intercept and random slope Random intercept Allows overall variation in the Y variable, due to specific subject or item features, independently of any X Random slope Allows X specific variation in the Y variable, due to specific subject or item features Davide Crepaldi 70 / 123

An example Suppose you want to know what is the priming effect related to orthography and semantics. Your hypothesis is that it changes with different exposure times for the primes. You also know that RTs change with the frequency of the target words, and that there are trial series effects. What kind of design would you use? What X? What fixed and what random effects? Davide Crepaldi 71 / 123

Another example Suppose you want to know what is the effect of seeing flashed high valence images before performing a pleasantness judgment. You suspect that this effect depends on some personality trait of the judges. What kind of design would you use? What X? What fixed and what random effects? Davide Crepaldi 72 / 123

Model fitting Model selection Parameter testing What s next? y ij = β 1 x 1 + β 2 x 2 +... + β n x n + Ss i + Ww j + ɛ i j Estimate the parameters, given the data Decide which X help and which do not (model refinement) Once we have the best model, decide which parameters differ reliably from zero Davide Crepaldi 73 / 123

ANOVA does all this in one step Model fitting Model selection Parameter testing df SumSq MeanSq F value p SOA 1 535.2 535.2 7.241.115 Residuals 2 147.8 73.9 Davide Crepaldi 74 / 123

Model fitting Model selection Parameter testing Model fitting Find our best guess at β 1, β 2, β 3...... that never appears into ANOVA tables...... but is the only index of how big is any effect. Questions: Is there an effect? can I be sure that that number isn t 0? How big is an effect? how far is that number from 0? Davide Crepaldi 75 / 123

Model selection Model fitting Model selection Parameter testing Need to be: As simple as you can As precise as you can Davide Crepaldi 76 / 123

Model selection Model fitting Model selection Parameter testing Davide Crepaldi 77 / 123

Model fitting Model selection Parameter testing Order and correlation between predictors Effects are always partialized Order matters because of the correlation between predictors Consider blocks of variables, and then remove one by one Davide Crepaldi 78 / 123

Fixed effect table Model fitting Model selection Parameter testing Is RT dependent on SOA? Estimate Std. Error t value Intercept 522.11 21.99 23.74 SOAshort -18.89 8.26-2.29 Davide Crepaldi 79 / 123

Fixed effect table Model fitting Model selection Parameter testing sbj word SOA RT Fixed Random Res Int SOA WordInt SubInt SubSOA s1 w1 long 466 522.2 0-28.3-26.2 0-2 s1 w2 long 520 522.2 0 14.2-26.2 0 9.8 s1 w3 long 502 522.2 0 14.1-26.2 0-8.2 s1 w1 short 475 522.2-19 -28.3-26.2 11 15.4 s1 w2 short 494 522.2-19 14.2-26.2 11-8.4 s1 w3 short 490 522.2-19 14.1-26.2 11-11.9... Davide Crepaldi 80 / 123

Levels and parameters Model fitting Model selection Parameter testing β level A 0 level B -19 Davide Crepaldi 81 / 123

Levels and parameters Model fitting Model selection Parameter testing β level A 0 level B -19 β level A 0 level B -19 level C? Davide Crepaldi 82 / 123

Levels and parameters Model fitting Model selection Parameter testing β level A 0 level B -19 β 1 β 2 level A 0 0 level B -19 0 level C 0 +12 Davide Crepaldi 83 / 123

Levels and parameters Model fitting Model selection Parameter testing β 1 β 2 β 3 level A 0 0 0 level B -19 0 0 level C 0 +12 0 level D 0 0-34 β 1 β 2 β 3 β 4 level A 0 0 0 0 level B -19 0 0 0 level C 0 +12 0 0 level D 0 0-34 0 level E 0 0 0-3 Davide Crepaldi 84 / 123

Reference level Model fitting Model selection Parameter testing β 1 β 2 β 3 level A 0 0 0 level B -19 0 0 level C 0 +12 0 level D 0 0-34 β 1 β 2 β 3 β 4 level A 0 0 0 0 level B -19 0 0 0 level C 0 +12 0 0 level D 0 0-34 0 level E 0 0 0-3 Davide Crepaldi 85 / 123

Reference level Model fitting Model selection Parameter testing The reference level is the one which is not in the table Estimate Std. Error t value Intercept 522.11 21.99 23.74 SOAshort -18.89 8.26-2.29 Davide Crepaldi 86 / 123

Levels and parameters Model fitting Model selection Parameter testing In this experiment five SOAs were used (12, 24, 36, 48, and 59 ms) Estimate Std. Error t value Intercept -1.664.026-65.21 soa24.008.009.90 soa36.007.008.89 soa48.001.007.19 soa59.004.007.62 Davide Crepaldi 87 / 123

Model fitting Model selection Parameter testing A little exercise Suppose you want to test the claim that low taxation makes people happy. You know that taxes are very high in Sweden, high in Italy, medium to high in Germany, moderate in the UK and low in the US. How would you proceed? Suppose you want to test whether grammatical class influence response times in a reading task. You have reasons to believe that nouns are faster than adjectives, which in turns are faster than adverbs, which in turns are faster than verbs. How would you proceed? Davide Crepaldi 88 / 123

Model fitting Model selection Parameter testing Parameters and effects Parameters and whole effects Whole effects relate to an overall increase in goodness of fit; single parameters relate to specific comparisons (roughly comparable to post hoc effects in the classic approach) With more than two levels, care is needed (some parameters may be significant, some others may not: what about the significance of the whole effect?) Significance testing on individual parameters Variance in the estimate distribution isn t very clear in mixed effect models Unclear how many degrees of freedom each test has Bootstrapping (Monte Carlo Markow Chain, mcmc) Davide Crepaldi 89 / 123

Markov chain Monte Carlo Model fitting Model selection Parameter testing Davide Crepaldi 90 / 123

Model fitting Model selection Parameter testing Markov chain Monte Carlo Estimate MCMCmean HPD95lower HPD95upper pmcmc Pr(> t )) Intercept 522.11 521.94 489.87 553.971 0.000 0.000 SOAshort 18.89 18.88 40.46 2.318 0.078 0.016 Davide Crepaldi 91 / 123

Random effect table Model fitting Model selection Parameter testing Groups Name Variance Std.Dev. SOA:sbj (Intercept) 33.752 5.8096 word (Intercept) 610.701 24.7124 sbj (Intercept) 485.296 22.0294 Residual 120.467 10.9757 Davide Crepaldi 92 / 123

Do the mice succeed? group1 group2 lab1 lab2 lab3 lab4 lab5 lab6 mouse1 1 1 1 0 1 1 mouse2 0 0 0 1 1 1 mouse3 0 1 0 1 1 1 mouse4 1 0 0 1 0 1 mouse5 1 1 0 1 1 1 mouse6 0 0 0 0 1 1 Davide Crepaldi 93 / 123

Impossible values Mean and SD in labs 4, 5 and 6 are.83 and.17 respectively Davide Crepaldi 94 / 123

ANOVA assumptions Mean and variance need to be independent Davide Crepaldi 95 / 123

The logit function Y = ln p 1 p Davide Crepaldi 96 / 123

Fixed effect table Is accuracy dependent on SOA? Estimate Std. Error z value Pr(> z )) Intercept 20.56 41.01 0.501 0.616 SOAshort 35.31 54.29 0.650 0.515 Davide Crepaldi 97 / 123

Random effect table Groups Name Variance Std.Dev. SOA:sbj (Intercept).0002 0.01 word (Intercept).0010 32.33 sbj (Intercept).0656 25.61 Davide Crepaldi 98 / 123