Mixed effects modeling

Transcription

1 Mixed effects modeling Generalising to the universe with random item and subject selection Davide Crepaldi MoMo Lab, Department of Psychology University of Milano Bicocca, Italy Spring 2013 Davide Crepaldi 1 / 123

2 Part II Mixed effects models Davide Crepaldi 45 / 123

3 Outline of part II 3 Point prediction 4 5 Model fitting Model selection Parameter testing 6 Davide Crepaldi 46 / 123

4 A new way of thinking about problems Una serie di problemi, in rigoroso ordine di importanza Mangiare carne rossa determina un incremento del rischio di cancro all intestino. Un consumo moderato di alcool (2 bicchieri vino la settimana) durante la gravidanza abbassa il QI del nascituro Un consumo moderato di alcool (2 bicchieri vino la settimana) durante la gravidanza abbassa il QI del nascituro di circa 2 punti Davide Crepaldi 47 / 123

5 The classic approach Populations to be compared Sample data, to be generalized safely to populations Compare means Sample stats distribution in random sampling Davide Crepaldi 48 / 123

6 Mixed effect modeling How is it that any given datapoint is such? y ij = β 1 x 1 + β 2 x β n x n + Ss i + Ww j + ɛ i j (4) Davide Crepaldi 49 / 123

7 An example Suppose you have a dataset with three participants s1, s2, and s3 who each saw three words w1, w2, and w3 in a reading experiment. This is a priming experiment, and each word was tested under a short and a long SOA condition with each participant. Davide Crepaldi 50 / 123

8 An example ####### 500 ms dealer ms DEAL 1500 ms press button Davide Crepaldi 51 / 123

9 Tabulate the data Davide Crepaldi 52 / 123

10 Tabulate the data Short SOA w1 w2 w3 s s s Davide Crepaldi 53 / 123

11 Tabulate the data Short SOA w1 w2 w3 s s s Long SOA w1 w2 w3 s s s Davide Crepaldi 54 / 123

12 Tabulate the data sbj word SOA RT s1 w1 long 466 s1 w2 long 520 s1 w3 long 502 s1 w1 short 475 s1 w2 short 494 s1 w3 short Davide Crepaldi 55 / 123

13 Tabulate the data sbj word SOA RT Int s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 56 / 123

14 Tabulate the data sbj word SOA RT Int SOA s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 57 / 123

15 Tabulate the data sbj word SOA RT Int SOA WordInt s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 58 / 123

16 Tabulate the data sbj word SOA RT Int SOA WordInt SubInt s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 59 / 123

17 Tabulate the data sbj word SOA RT Int SOA WordInt SubInt SubSOA s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 60 / 123

18 Tabulate the data sbj word SOA RT Res Int SOA WordInt SubInt SubSOA s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 61 / 123

19 More than one X sbj word SOA Lan RT Int SOA Lan s1 w1 long L s1 w2 long L s1 w3 long L s1 w1 short L s1 w2 short L s1 w3 short L Davide Crepaldi 62 / 123

20 Interaction sbj word SOA Lan SOA Lan RT Int SOA Lan SOA Lan s1 w1 long L s1 w2 long L s1 w3 long L s1 w1 short L s1 w2 short L s1 w3 short L Davide Crepaldi 63 / 123

21 Continue X sbj word SOA Freq RT Int SOA Freq s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 64 / 123

22 Non linear effects sbj word SOA Freq Freq 2 RT Int SOA Freq Freq 2 s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 65 / 123

23 Fixed and random sbj word SOA RT Res Int SOA WordInt SubInt SubSOA s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 66 / 123

24 Fixed and random sbj word SOA RT Fixed Random Res Int SOA WordInt SubInt SubSOA s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 67 / 123

25 Fixed and random Fixed effects Effects of interest Level selection NOT random Interest in estimating effect size Random effects Influence Y, but not of interest Level selection IS random Interest in estimating variability Davide Crepaldi 68 / 123

26 Random intercept and random slope sbj word SOA RT Fixed Random Res Int SOA WordInt SubInt SubSOA s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 69 / 123

27 Random intercept and random slope Random intercept Allows overall variation in the Y variable, due to specific subject or item features, independently of any X Random slope Allows X specific variation in the Y variable, due to specific subject or item features Davide Crepaldi 70 / 123

28 An example Suppose you want to know what is the priming effect related to orthography and semantics. Your hypothesis is that it changes with different exposure times for the primes. You also know that RTs change with the frequency of the target words, and that there are trial series effects. What kind of design would you use? What X? What fixed and what random effects? Davide Crepaldi 71 / 123

29 Another example Suppose you want to know what is the effect of seeing flashed high valence images before performing a pleasantness judgment. You suspect that this effect depends on some personality trait of the judges. What kind of design would you use? What X? What fixed and what random effects? Davide Crepaldi 72 / 123

30 Model fitting Model selection Parameter testing What s next? y ij = β 1 x 1 + β 2 x β n x n + Ss i + Ww j + ɛ i j Estimate the parameters, given the data Decide which X help and which do not (model refinement) Once we have the best model, decide which parameters differ reliably from zero Davide Crepaldi 73 / 123

31 ANOVA does all this in one step Model fitting Model selection Parameter testing df SumSq MeanSq F value p SOA Residuals Davide Crepaldi 74 / 123

32 Model fitting Model selection Parameter testing Model fitting Find our best guess at β 1, β 2, β that never appears into ANOVA tables but is the only index of how big is any effect. Questions: Is there an effect? can I be sure that that number isn t 0? How big is an effect? how far is that number from 0? Davide Crepaldi 75 / 123

33 Model selection Model fitting Model selection Parameter testing Need to be: As simple as you can As precise as you can Davide Crepaldi 76 / 123

34 Model selection Model fitting Model selection Parameter testing Davide Crepaldi 77 / 123

35 Model fitting Model selection Parameter testing Order and correlation between predictors Effects are always partialized Order matters because of the correlation between predictors Consider blocks of variables, and then remove one by one Davide Crepaldi 78 / 123

36 Fixed effect table Model fitting Model selection Parameter testing Is RT dependent on SOA? Estimate Std. Error t value Intercept SOAshort Davide Crepaldi 79 / 123

37 Fixed effect table Model fitting Model selection Parameter testing sbj word SOA RT Fixed Random Res Int SOA WordInt SubInt SubSOA s1 w1 long s1 w2 long s1 w3 long s1 w1 short s1 w2 short s1 w3 short Davide Crepaldi 80 / 123

38 Levels and parameters Model fitting Model selection Parameter testing β level A 0 level B -19 Davide Crepaldi 81 / 123

39 Levels and parameters Model fitting Model selection Parameter testing β level A 0 level B -19 β level A 0 level B -19 level C? Davide Crepaldi 82 / 123

40 Levels and parameters Model fitting Model selection Parameter testing β level A 0 level B -19 β 1 β 2 level A 0 0 level B level C Davide Crepaldi 83 / 123

41 Levels and parameters Model fitting Model selection Parameter testing β 1 β 2 β 3 level A level B level C level D β 1 β 2 β 3 β 4 level A level B level C level D level E Davide Crepaldi 84 / 123

42 Reference level Model fitting Model selection Parameter testing β 1 β 2 β 3 level A level B level C level D β 1 β 2 β 3 β 4 level A level B level C level D level E Davide Crepaldi 85 / 123

43 Reference level Model fitting Model selection Parameter testing The reference level is the one which is not in the table Estimate Std. Error t value Intercept SOAshort Davide Crepaldi 86 / 123

44 Levels and parameters Model fitting Model selection Parameter testing In this experiment five SOAs were used (12, 24, 36, 48, and 59 ms) Estimate Std. Error t value Intercept soa soa soa soa Davide Crepaldi 87 / 123

45 Model fitting Model selection Parameter testing A little exercise Suppose you want to test the claim that low taxation makes people happy. You know that taxes are very high in Sweden, high in Italy, medium to high in Germany, moderate in the UK and low in the US. How would you proceed? Suppose you want to test whether grammatical class influence response times in a reading task. You have reasons to believe that nouns are faster than adjectives, which in turns are faster than adverbs, which in turns are faster than verbs. How would you proceed? Davide Crepaldi 88 / 123

46 Model fitting Model selection Parameter testing Parameters and effects Parameters and whole effects Whole effects relate to an overall increase in goodness of fit; single parameters relate to specific comparisons (roughly comparable to post hoc effects in the classic approach) With more than two levels, care is needed (some parameters may be significant, some others may not: what about the significance of the whole effect?) Significance testing on individual parameters Variance in the estimate distribution isn t very clear in mixed effect models Unclear how many degrees of freedom each test has Bootstrapping (Monte Carlo Markow Chain, mcmc) Davide Crepaldi 89 / 123

47 Markov chain Monte Carlo Model fitting Model selection Parameter testing Davide Crepaldi 90 / 123

48 Model fitting Model selection Parameter testing Markov chain Monte Carlo Estimate MCMCmean HPD95lower HPD95upper pmcmc Pr(> t )) Intercept SOAshort Davide Crepaldi 91 / 123

49 Random effect table Model fitting Model selection Parameter testing Groups Name Variance Std.Dev. SOA:sbj (Intercept) word (Intercept) sbj (Intercept) Residual Davide Crepaldi 92 / 123

50 Do the mice succeed? group1 group2 lab1 lab2 lab3 lab4 lab5 lab6 mouse mouse mouse mouse mouse mouse Davide Crepaldi 93 / 123

51 Impossible values Mean and SD in labs 4, 5 and 6 are.83 and.17 respectively Davide Crepaldi 94 / 123

52 ANOVA assumptions Mean and variance need to be independent Davide Crepaldi 95 / 123

53 The logit function Y = ln p 1 p Davide Crepaldi 96 / 123

54 Fixed effect table Is accuracy dependent on SOA? Estimate Std. Error z value Pr(> z )) Intercept SOAshort Davide Crepaldi 97 / 123

55 Random effect table Groups Name Variance Std.Dev. SOA:sbj (Intercept) word (Intercept) sbj (Intercept) Davide Crepaldi 98 / 123