8. Multi-Factor Designs Chapter 8. Experimental Design II: Factorial Designs 1
Goals Identify, describe and create multifactor (a.k.a. factorial ) designs Identify and interpret main effects and interaction effects Calculate N for a given factorial design 2
Complexity and Design As experimental designs increase in complexity: More information can be obtained. Care in design becomes ever more important. Designs with multiple factors and levels: Allow detection of interaction effects Allow detection of non-linear effects Involve more complexity around potential sequence effects and equivalent groups problems 3
8.1 Describing Multi-Factor Designs 4
Multi-Factor Designs Have more than one IV (or factor). a.k.a. factorial design Described by a numbering system that gives the number of levels of each IV Examples: 2 2 or 3 4 2 design Also described by factorial matrices 5
Numbering System for Factorial Designs Number of digits = number of IVs: 3 3 or 5 2 means two IVs. 2 2 2 or 3 4 2 means three IVs. Value of each digit = # of levels in each IV: 3 3 means two IVs, each with three levels. 3 4 2 means three IVs with 3, 4 and 2 levels, respectively 6
2 x 2 Factorial Design Drug Therapy Placebo Prozac Psychotherapy None Control Prozac CBT CBT Combined Therapy 7
2 x 3 Factorial Design Drug Therapy Placebo Prozac None Control Prozac Psychotherapy CBT CBT CBT + Prozac EFT EFT EFT + Prozac 8
Going 3D: 2 x 2 x 2 Factorial Design Female Drug Therapy Male Drug Therapy Placebo Prozac Placebo Prozac None CBT Control CBT Prozac Combo Psychotherapy Psychotherapy None CBT Control CBT Prozac Combo 9
2 x 2 x 3 Factorial Design Female Drug Therapy Male Drug Therapy None CBT EFT Placebo Control CBT EFT Prozac Prozac CBT + Prozac EFT + Prozac Psychotherapy Psychotherapy None CBT EFT Placebo Control CBT EFT Prozac Prozac CBT + Prozac EFT + Prozac 10
Female Drug Therapy Male Drug Therapy Placebo Prozac Placebo Prozac Introverts Extroverts None CBT None CBT Extro Extro Control Prozac Extro Extro CBT Combo Drug Therapy Placebo Prozac Intro Intro Control Prozac Intro Intro CBT Combo Psychotherapy Psychotherapy Psychotherapy Psychotherapy None CBT None CBT Extro Extro Control Prozac Extro Extro CBT Combo Drug Therapy Placebo Prozac Intro Intro Control Prozac Intro Intro CBT Combo 11
Levels vs. Conditions Level: One level of one IV. A row or column in the Factorial Matrix. Also, for 3+ IVs, one of the sub-matrices Condition: A particular combination of one level of each IV. One cell in the Factorial Matrix. In single-factor designs: level = condition 12
Placebo Level of Drug Therapy IV Drug Therapy Placebo Prozac Psychotherapy None Control Prozac CBT CBT Combo 13
Prozac Level of Drug Therapy IV Drug Therapy Placebo Prozac Psychotherapy None Control Prozac CBT CBT Combo 14
None Level of Psychotherapy IV Drug Therapy Placebo Prozac Psychotherapy None Control Prozac CBT CBT Combo 15
CBT Level of Psychotherapy IV Drug Therapy Placebo Prozac Psychotherapy None Control Prozac CBT CBT Combo 16
One-factor Designs 2-level Study Time 2 Hours 5 Hours Multilevel 2 Hours Study Time 3 Hours 4 Hours 5 Hours 17
Discussion / Questions Why are the terms level and factor interchangeable in a single-factor design? How many IVs are there in a 3 2 2 design? How many levels of each IV? How many total conditions? 18
8.2 Interpreting Data From Multi-Factor Designs 19
Interpreting Data from Factorial Designs Two types of effects can emerge in multi-factorial designs: Main Effects: When one IV has an effect on its own. That is, the mean for some pair of levels of the IV differ significantly from one another. Interaction Effects: When the effect of one IV is different for different levels of another IV. These are NOT mutually exclusive 20
A Simple 2x2 Design Drug Therapy Placebo Prozac Psychotherapy None Control Prozac CBT CBT Combo 21
Main Effect of Psychotherapy Drug Therapy Psychotherapy None CBT Placebo Prozac (Control+ Prozac ) / 2 (CBT + / Combo) 2 We collapse across the levels of all other IVs to evaluate a main effect 22
Main Effect of Drug Therapy Drug Therapy Placebo Prozac Psychotherapy None CBT (Control+ CBT ) /2 (Prozac + Combo) /2 We collapse across the levels of all other IVs to evaluate a main effect 23
Numerical Example Drug Therapy Placebo Prozac Psychotherapy None 12 ± 2 18 ± 1 CBT 17 ± 1 23 ± 3 24
Main Effect of Psychotherapy? Drug Therapy Placebo Prozac Psychotherapy None (12+18)/2 = 15 CBT (17+23)/2 = 20 25
Main Effect of Drug Therapy? Drug Therapy Psychotherapy Placebo None 12+17 2 CBT 14.5 Prozac 18+23 2 20.5 26
Numerical Example Drug Therapy Placebo Prozac µ Psychotherapy None 12 ± 2 18 ± 1 15-6 CBT 17 ± 1 23 ± 3 20-6 µ 14.5 20.5-5 -5 27
Numerical Example Drug Therapy Placebo Prozac µ Psychotherapy None 12 ± 2 18 ± 1 15-6 CBT 17 ± 1 30 ± 3 20-13 µ 14.5 20.5-5 -12 Evidence of Interaction 28
Discussion / Questions In a 3x3x2 design, how many potential main effects are there? How many IVs would you collapse across to evaluate each main effect? 29
Example Multi-Factorial Multi-factorial experiments manipulate several IVs to see if their effects interact Example Question: Does gender interact with psychotherapy in affecting depression? Two IVs: Experiment Gender. 2 Levels = male; female Psychotherapy. 2 levels: control (none); experimental (therapy) One DV: Depression (measure = BDI) 30
!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+' 31
Another 2-Factor Design, 3 Levels Per Factor Arousal Low Med High Easy Low Easy Med Easy High Easy Task Difficulty Average Low Average Med Average High Average Hard Low Hard Med Hard High Hard 32
Another 2-Factor Design, 3 Levels Per Factor Arousal Low Med High µ ΔLM ΔMH ΔLH Easy 40 40 40 40 0 0 0 Task Difficulty Avrge 15 30 15 20 15-15 0 Hard 8 5 2 5-3 -3-6 µ 21 25 19 ΔEA -25-10 -25 ΔAH -7-25 -13 ΔEH -32-35 -38 33
Results: 3x3 Design!"#$%#&'()"* '!" &!" %!" $!" #!"!" ()*" +,-./0" 1.23" +#%,-'.* 4567" 89,:52," 15:-" 34
3x3 Results: Main Effects, No Interaction Arousal Low Med High µ ΔL ΔM ΔLH Task Difficult y Easy 30 40 50 40 10 10 20 Avrge 15 25 35 25 10 10 20 Hard 6 16 26 16 10 10 20 µ 17 27 37 ΔEA -15-15 -15 ΔAH -9-9 -9 ΔEH -24-24 -24 35
3x3 Results: 2 Main Effects, No Interaction!"#$%#&'()"* (!" '!" &!" %!" $!" #!"!" )*+",-./01" 2/34" +#%,-'.* 5678" 9:-;63-" 26;." 36
Interpreting Data from Factorial Designs If one IV has an effect--that is, there s a significant effect of going from one level of that IV to another, while ignoring ( collapsing across ) all other IVs--then that IV is said to produce a main effect. If the effect of one IV differs depending on the level of another IV, there s an interaction. 37
!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '!" ()*+,)-"""""""""".*)"+/0,1234" 5/0,123"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+' 38
The Importance of Interactions Interpretation of interaction fx overrides interpretation of main fx Example: What s most important in these results: Main effect of gender? Main effect of therapy? Interaction of the two?!"#$%&#'()*)+' &!" %!" $!" #!"!" '()*+(,"""""""""" -)("*./+0123" 4./+012",-)./$"'()*)+' If the gender factor is ignored, the therapy seems to simply be effective for all people. But this is not true. It is effective for females only. 39
X-Way Interactions When there are 2 IVs, a 2-way interaction is possible,with 3 IVs, may have a 3-way interaction, etc. 3-way interaction means the 2-way interaction changes depending on a 3rd variable. 40
!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+'!" #!" $!" %!" &!" '()*+(,"""""""""" -)("*./+0123" 4./+012"!"#$%&#'()*)+',-)./$"'()*)+' Introverts Extroverts Introverts Extroverts Introverts Extroverts 41
Discussion / Questions 42
8.3 Mixed Multi-Factor Designs 43
Review: Between-Subjects Design All Participants (N = 20) Condition 1 (n = 10) Condition 2 (n = 10) 44
All Participants (N=10) Review: Within-Subjects Design Level 1 (N = 10) Level 2 (N = 10) 45
Within, Between & Mixed Multi-Factor Designs With multiple factors/ivs, one can mix different kinds of variables (within/between; subject/manipulated, etc.) If all IVs are within-subjects then the design is fully within If all IVs are between-subjects then the design is fully between Otherwise, it s a mixed design 46
All Participants (N = 20) 2x2 Fully Between Subjects Design Condition A1B1 (n=5) Condition A1B2 (n=5) Condition A2B1 (n=5) Condition A2B2 (n=5) 47
All Participants (n = 20) 2x2 Fully Within Subjects Design Note that orders are not shown, there would be 24 for a fully-counterbalanced design! Condition A1B1 (n = 20) Condition A1B2 (n = 20) Condition A2B1 (n = 20) Condition A2B2 (n = 20) 48
2x2 Mixed Design Level B1 (10) All Participants (20) Level B2 (10) A1B1 (10) A1B2 (10) A2B1 (10) A2B2 (10) 49
Fully Within-Subjects Factorial Design a.k.a., Repeated-measures factorial design. All subjects are run through all conditions (i.e., all cells of the factorial matrix). Same advantages/disadvantages as single-factor repeated measures design 50
Example Experiment 1: Fully Within-Subjects Question: Is face recognition more impaired by inversion than object recognition? Method Subjects are 20 undergraduates Materials are pictures of 25 famous faces and 25 common objects, either inverted or not. (So 100 images in all). 51
Example Experiment 1 Design: 2x2 Fully within-subjects factorial, with factors being Type of Image (Face or Object) and View (upright or inverted). Procedure: All 20 subjects are shown all 100 images several times in random order and asked to identify each as quickly as possible. Repeated-measures factorial design. DV is reaction time to name picture. 52
Image Type Face Object Upright View Inverted 53
Example Experiment 1 Expected results: RT will be higher for inverted images than upright ones (main effect). But this effect will be greater for faces (interaction). Implications: Implies that there s something different about how people process faces as compared to objects 54
Possible Results!"#$%&'%()*"+,% )!!" (!!" '!!" &!!" %!!" $!!" #!!"!" 67849" :;<4809" *+,-./0" 1234,045" -./)010*%23/"$.#./4$% 55
Fully Between-Subjects Factorial Designs Each subject run through only one condition (i.e., one cell of the factorial matrix) If all IVs are subject variables, you have a Nonequivalent groups factorial design If all IVs are manipulated, decide how equivalent groups are formed: Random assignment: Independent groups factorial design Matching: Matched groups factorial design 56
Example Experiment 2: Fully Between-Subjects Question: Same as before, are faces more affected by inversion than objects? Method Subjects are 80 undergraduates (note higher N than within-ss design). Materials: Same as before, 25 pictures of faces, 25 pictures of objects, shown both upright and inverted. 57
Example Experiment 2 Design 2 2 fully between-subjects factorial design. Assign subjects randomly to one of four groups of 20. Independent groups factorial design. Procedure: Each group sees 25 pictures (upright faces, inverted face, upright objects, or inverted objects). 58
Image Type Face Object Upright View Inverted 59
Discussion / Questions 60
Mixed Factorial Designs At least one IV within-subjects and one between-subjects. Subjects run through all levels of some IVs, but only single level of other IVs. That is, each subject goes through one row or column of the factorial matrix. Random assignment, matching, counterbalancing can all be used. 61
Example Experiment 3: Mixed Factorial Design Question: Is face recognition more impaired by inversion than object recognition? Method Subjects are 40 undergraduates (note higher N than fully within, but lower than fully between). Materials are pictures of 25 famous faces and 25 objects, either inverted or not. 62
Example Experiment 3 Design: 2x2 Mixed factorial with factors being Type of Image (face or object, within) and View (upright or inverted, between) Procedure: 20 subjects are shown the 50 inverted images (25 faces and 25 objects), while 20 other subjects are shown the 50 upright images (25 faces, 25 objects). 63
Image Type Face Object Upright View Inverted 64
PxE Factorial Designs Person by Environment Variety of fully-between or mixed factorial design At least one subject IV (person) and at least one manipulated IV ( environment ) 65
Example Experiment 4: PxE Design Question: Does the effect of assigned study style interact with preferred study style? Method Person IV: Ss assigned to groups based on preferred study style: Crammers or Distributers. This is a subject IV Enviro IV: Half of subjects in each above group are assigned to study by cramming or by distributing study. This is manipulated 66
Possible Results Preferred Style (subject) Crammer Distributer Assigned Style (manipulated) Cramming 65 65 Distributing 80 90 67
Possible Results!"#$%&'" (""# '"# &"# %"# $"#!"# )*+,,5*1# 0-12*-3425*1# )*+,,-./# 0-12*-342-./# ()*+,-,."/$*'0)%)*10" Assigned Study Style 68
Interpreting Results From PxE Designs Cannot draw causal links for the subject variables, can draw causal links for the manipulated ( environment ) variable. So a causal link can be established for assigned style but not preferred style. Cannot draw causal links for interaction effects. 69
Example 2x3x2 Study Caspi et al., 2007, PNAS, 104 (47), 18860-18865 70
How Many Participants? If I need 50 participants per cell in a 2 2 factorial design, what is the total N? What if the design is fully within? What if the design is mixed? Answer the same questions for a 3 2 3 design with 10 participants per cell. 71
Analyzing Data From Multi- Factor Designs As for multi-level designs, multi-factor designs are generally analyzed via ANOVA procedures: Pre-tests for normality and other assumptions 2-way (or X-way) ANOVA/MANOVA/ANCOVA... Post-hoc tests to examine effects in greater detail Planned comparison techniques may also be involved Note that there are no well-established techniques for dealing with multi-factor ordinal-scale data 72
Discussion / Questions 73
8.4 Summary: Design Complexity 74
Single-Factor, 2-Level Experimental Designs Can t detect non-linear effects. Can t detect interactions. Involve only simple counter-balancing or simple equivalent groups problems. 75
Single-Factor, Multilevel Designs Can detect non-linear effects Can t detect interactions May involve relatively complex counterbalancing or equivalent groups problems 76
Multi-Factor Designs Multi-factor Designs Can detect interactions and main effects Can detect non-linear effects where IVs have 3 levels May involve both complex counter-balancing and equivalent groups problems. 77
Conclusion: Experimental Design Experiments and quasi-experiments are just one way of doing research True experiments (not quasi) allow conclusions about causality Next we will turn to observational research, which is simpler in some ways 78