Chapter. Three-Way ANOVA CONCEPTUAL FOUNDATION. A Simple Three-Way Example. 688 Chapter 22 Three-Way ANOVA

Transcription

1 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA Three-Way ANOVA 22 Chapter A CONCEPTUAL FOUNDATION 688 You will need to use the following from previous chapters: Symbols k: Number of independent groups in a one-way ANOVA c: Number of levels (i.e., conditions) of an RM factor n: Number of subjects in each cell of a factorial ANOVA N T : Total number of observations in an experiment Formulas Formula 16.2: SS inter (by subtraction) also Formulas 16.3, 16.4, 16.5 Formula 14.3: SS bet or one of its components Concepts Advantages and disadvantages of the RM ANOVA SS components of the one-way RM ANOVA SS components of the two-way ANOVA Interaction of factors in a two-way ANOVA So far I have covered two types of two-way factorial ANOVAs: two-way independent (Chapter 14) and the mixed design ANOVA (Chapter 16). There is only one more simple two-way ANOVA to describe: the two-way repeated measures design. [There are other two-way designs, such as those including randomeffects or nested factors, but they are not commonly used see Hays (1994) for a description of some of these.] Just as the one-way RM ANOVA can be described in terms of a two-way independent-groups ANOVA, the two-way RM ANOVA can be described in terms of a three-way independent-groups ANOVA. This gives me a reason to describe the latter design next. Of course, the threeway factorial ANOVA is interesting in its own right, and its frequent use in the psychological literature makes it an important topic to cover, anyway. I will deal with the three-way independent-groups ANOVA and the two-way RM ANOVA in this section and the two types of three-way mixed designs in Section B. Computationally, the three-way ANOVA adds nothing new to the procedure you learned for the two-way; the same basic formulas are used a greater number of times to extract a greater number of SS components from SS total (eight SSs for the three-way as compared with four for the two-way). However, anytime you include three factors, you can have a three-way interaction, and that is something that can get quite complicated, as you will see. To give you a manageable view of the complexities that may arise when dealing with three factors, I ll start with a description of the simplest case: the ANOVA. A Simple Three-Way Example At the end of Section B in Chapter 14, I reported the results of a published study, which was based on a 2 2 ANOVA. In that study one factor contrasted subjects who had an alcohol-dependent parent with those who did not. I ll call this the alcohol factor and its two levels, at risk (of codependency) and control. The other factor (the experimenter factor) also had two levels; in one level subjects were told that the experimenter was an exploitive person, and in the other level the experimenter was described as a nurturing person. All of the subjects were women. If we imagine that the experiment was replicated using equal-sized groups of men and women, the original

2 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 689 Section A Conceptual Foundation 689 two-way design becomes a three-way design with gender as the third factor. We will assume that all eight cells of the design contain the same number of subjects. As in the case of the two-way ANOVA, unbalanced threeway designs can be difficult to deal with both computationally and conceptually and therefore will not be discussed in this chapter (see Chapter 18, section A). The cell means for a three-factor experiment are often displayed in published articles in the form of a table, such as Table Nurturing Exploitive Row Mean Control: Men Women Mean Table 22.1 At risk: Men Women Mean Column mean Graphing Three Factors The easiest way to see the effects of this experiment is to graph the cell means. However, putting all of the cell means on a single graph would not be an easy way to look at the three-way interaction. It is better to use two graphs side by side, as shown in Figure With a two-way design one has to decide which factor is to be placed along the horizontal axis, leaving the other to be represented by different lines on the graph. With a three-way design one chooses both the factor to be placed along the horizontal axis and the factor to be represented by different lines, leaving the third factor to be represented by different graphs. These decisions result in six different ways that the cell means of a three-way design can be presented. Let us look again at Figure The graph for the women shows the twoway interaction you would expect from the study on which it is based. The graph for the men shows the same kind of interaction, but to a considerably lesser extent (the lines for the men are closer to being parallel). This difference 80 Women At risk 80 Men Figure 22.1 Graph of Cell Means for Data in Table At risk Control 20 Control 20 0 Nurturing Exploitive 0 Nurturing Exploitive

3 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA in amount of two-way interaction for men and women constitutes a three-way interaction. If the two graphs had looked exactly the same, the F ratio for the three-way interaction would have been zero. However, that is not a necessary condition. A main effect of gender could raise the lines on one graph relative to the other without contributing to a three-way interaction. Moreover, an interaction of gender with the experimenter factor could rotate the lines on one graph relative to the other, again without contributing to the three-way interaction. As long as the difference in slopes (i.e., the amount of two-way interaction) is the same in both graphs, the three-way interaction will be zero. Simple Interaction Effects A three-way interaction can be defined in terms of simple effects in a way that is analogous to the definition of a two-way interaction. A two-way interaction is a difference in the simple main effects of one of the variables as you change levels of the other variable (if you look at just the graph of the women in Figure 22.1, each line is a simple main effect). In Figure 22.1 each of the two graphs can be considered a simple effect of the three-way design more specifically, a simple interaction effect. Each graph depicts the two-way interaction of alcohol and experimenter at one level of the gender factor. The three-way interaction can be defined as the difference between these two simple interaction effects. If the simple interaction effects differ significantly, the three-way interaction will be significant. Of course, it doesn t matter which of the three variables is chosen as the one whose different levels are represented as different graphs if the three-way interaction is statistically significant, there will be significant differences in the simple interaction effects in each case. Varieties of Three-way Interactions Just as there are many patterns of cell means that lead to two-way interactions (e.g., one line is flat while the other goes up or down, the two lines go in opposite directions, or the lines go in the same direction but with different slopes), there are even more distinct patterns in a three-way design. Perhaps the simplest is when all of the means are about the same, except for one, which is distinctly different. For instance, in our present example the results might have shown no effect for the men (all cell means about 40), no difference for the control women (both means about 40), and a mean of 40 for at-risk women exposed to the nice experimenter. Then, if the mean for atrisk women with the exploitive experimenter were well above 40, there would be a strong three-way interaction. This is a situation in which all three variables must be at the right level simultaneously to see the effect in this variation of our example the subject must be female and raised by an alcohol-dependent parent and exposed to the exploitive experimenter to attain a high score. Not only might the three-way interaction be significant, but one cell mean might be significantly different from all of the other cell means, making an even stronger case that all three variables must be combined properly to see any effect (if you were sure that this pattern were going to occur, you could test a contrast comparing the average of seven cell means to the one you expect to be different and not bother with the ANOVA at all). More often the results are not so clear-cut, but there is one cell mean that is considerably higher than the others (as in Figure 22.1). This kind of pattern is analogous to the ordinal interaction in the two-way case and tends to cause all of the effects to be significant. On the other hand, a three-way interaction could arise because the two-way interaction reverses its pattern when changing levels of the third variable (e.g., imagine that in Figure 22.1

4 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 691 Section A Conceptual Foundation 691 the labels of the two lines were reversed for the graph of men but not for the women). This is analogous to the disordinal interaction in the two-way case. Or, the two-way interaction could be strong at one level of the third variable and much weaker (or nonexistent) at another level. Of course, there are many other possible variations. And consider how much more complicated the three-way interaction can get when each factor has more than two levels (we will deal with a greater number of levels in Section B). Fortunately, three-way (between-subjects) ANOVAs with many levels for each factor are not common. One reason is a practical one: the number of subjects required. Even a design as simple as a has 24 cells (to find the number of cells, you just multiply the numbers of levels). If you want to have at least 5 subjects per cell, 120 subjects are required. This is not an impractical study, but you can see how quickly the addition of more levels would result in a required sample size that could be prohibitive. Main Effects In addition to the three-way interaction there are three main effects to look at, one for each factor. To look at the gender main effect, for instance, just take the average of the scores for all of the men and compare it to the average of all of the women. If you have the cell means handy and the design is balanced, you can average all of the cell means involving men and then all of the cell means involving women. In Table 22.1, you can average the four cell means for the men (40, 28, 36, 48) to get 38 (alternatively, you could use the row means in the extreme right column and average 34 and 42 to get the same result). The average for the women (30, 22, 40, 88) is 45. The means for the other main effects have already been included in Table Looking at the bottom row you can see that the mean for the nurturing experimenter is 36.5 as compared to 46.5 for the exploitive one. In the extreme right column you ll find that the mean for the control subjects is 30, as compared to 53 for the at-risk subjects. Two-Way Interactions in Three-Way ANOVAs Further complicating the three-way ANOVA is that, in addition to the threeway interaction and the three main effects, there are three two-way interactions to consider. In terms of our example there are the gender by experimenter, gender by alcohol, and experimenter by alcohol interactions. We will look at the last of these first. Before graphing a two-way interaction in a three-factor design, you have to collapse (i.e., average) your scores over the variable that is not involved in the two-way interaction. To graph the alcohol by experimenter (A B) interaction you need to average the men with the women for each combination of alcohol and experimenter levels (i.e., each cell of the A B matrix). These means have also been included in Table The graph of these cell means is shown in Figure If you compare this overall two-way interaction with the two-way interactions for the men and women separately (see Figure 22.1), you will see that the overall interaction looks like an average of the two separate interactions; the amount of interaction seen in Figure 22.2 is midway between the amount of interaction for the men and that amount for the women. Does it make sense to average the interactions for the two genders into one overall interaction? It does if they are not very different. How different is too different? The size of the three-way interaction tells us how different these two two-way interactions are. A statistically significant three-way interaction suggests that we should be cautious in interpreting any of the two-way interactions. Just as a significant two-way interaction tells us to look carefully at, and possible test, the

5 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA Figure 22.2 Graph of Cell Means in Table 22.1 after Averaging Across Gender Average of Men and Women At risk Control 0 Nurturing Exploitive simple main effects (rather than the overall main effects), a significant threeway interaction suggests that we focus on the simple interaction effects the two-way interactions at each level of the third variable (which of the three independent variables is treated as the third variable is a matter of convenience). Even if the three-way interaction falls somewhat short of significance, I would recommend caution in interpreting the two-way interactions and the main effects, as well, whenever the simple interaction effects look completely different and, perhaps, show opposite patterns. So far I have been focusing on the two-way interaction of alcohol and experimenter in our example, but this choice is somewhat arbitrary. The two genders are populations that we are likely to have theories about, so it is often meaningful to compare them. However, I can just as easily graph the three-way interaction using alcohol as the third factor, as I have done in Figure 22.3a. To graph the overall two-way interaction of gender and experimenter, you can go back to Table 22.1 and average across the alcohol factor. For instance, the mean for men in the nurturing condition is found by averaging the mean for control group men in the nurturing condition (40) with Figure 22.3a Control At Risk Graph of Cell Means in Table 22.1 Using the Alcohol Factor to Distinguish the Panels Women Men Men Women Nurturing Exploitive 0 Nurturing Exploitive

6 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 693 Section A Conceptual Foundation Average of Control and at Risk Women Men Figure 22.3b Graph of Cell Means in Table 22.1 after Averaging Across the Alcohol Factor 0 Nurturing Exploitive the mean for at-risk men in the nurturing condition (36), which is 38. The overall two-way interaction of gender and experimenter is shown in Figure 22.3b. Note that once again the two-way interaction is a compromise. (Actually, the two two-way interactions are not as different as they look; in both cases the slope of the line for the women is more positive or at least less negative). For completeness, I have graphed the three-way interaction using experimenter as the third variable, and the overall two-way interaction of gender and alcohol in Figures 22.4a and 22.4b. An Example of a Disordinal Three-Way Interaction In the three-factor example I have been describing, it looks like all three main effects and all three two-way interactions, as well as the three-way interaction, could easily be statistically significant. However, it is important to note that in a balanced design all seven of these effects are independent; the seven F ratios do share the same error term (i.e., denominator), but the sizes of the numerators are entirely independent. It is quite possible to have Nurturing Exploitive Figure 22.4a Women Graph of Cell Means in Table 22.1 Using the Experimenter Factor to Distinguish the Panels Women Men Men Control At risk 0 Control At risk

7 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA Figure 22.4b Average of Nurturing and Exploitive Graph of Cell Means in Table 22.1 after Averaging Across the Experimenter Factor Women Men 0 Control At risk a large three-way interaction while all of the other effects are quite small. By changing the means only for the men in our example, I will illustrate a large, disordinal interaction that obliterates two of the two-way interactions and two of the main effects. You can see in Figure 22.5a that this new three-way interaction is caused by a reversal of the alcohol by experimenter interaction from one gender to the other. In Figure 22.5b, you can see that the overall interaction of alcohol by gender is now zero (the lines are parallel); the gender by experimenter interaction is also zero (not shown). On the other hand, the large gender by alcohol interaction very nearly obliterates the main effects of both gender and alcohol (see Figure 22.5c). The main effect of experimenter is, however, large, as can be seen in Figure 22.5b. An Example in which the Three-Way Interaction Equals Zero Finally, I will change the means for the men once more to create an example in which the three-way interaction is zero, even though the graphs for the Figure 22.5a Women Men Rearranging the Cell Means of Table 22.1 to Depict a Disordinal 3-Way Interaction At risk Control Control At risk 0 Nurturing Expoitive 0 Nurturing Expoitive

8 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 695 Section A Conceptual Foundation Average of men and women At risk Control Figure 22.5b Regraphing Figure 22.5a after Averaging Across Gender 0 Nurturing Exploitive Average of Nurturing and Exploitive Figure 22.5c Men Women Regraphing Figure 22.5a after Averaging Across the Experimenter Factor Control At risk two genders do not look the same. In Figure 22.6, I created the means for the men by starting out with the women s means and subtracting 10 from each (this creates a main effect of gender); then I added 30 only to the men s means that involved the nurturing condition. The latter change creates a two-way interaction between experimenter and gender, but because it affects both the men/nurturing means equally, it does not produce any threeway interaction. One way to see that the three-way interaction is zero in Figure 22.6 is to subtract the slopes of the two lines for each gender. For the women the slope of the at-risk line is positive: = 48. The slope of the control line is negative: = 8. The difference of the slopes is 48 ( 8) = 56. If we do the same for the men, we get slopes of 18 and 38, whose difference is also 56. You may recall that a 2 2 interaction has only one df, and can be summarized by a single number, L, that forms the basis of a simple linear contrast. The same is true for a interaction or any higher-order interaction in which all of the factors have two levels. Of course, quantifying a three-way interaction gets considerably more complicated when the factors have more than two levels, but it is safe to say that if the two (or more) graphs are exactly the same, there will be no three-way interaction (they will continue to be identical, even if a different factor is chosen to distinguish the

9 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA Figure 22.6 Women Men Rearranging the Cell Means of Table 22.1 to Depict a Zero Amount of Three-Way Interaction At risk At risk Control Control 0 Nurturing Expoitive 0 Nurturing Expoitive graphs). Bear in mind, however, that even if the graphs do not look the same, the three-way interaction will be zero if the amount of two-way interaction is the same for every graph. Calculating the Three-Way ANOVA Calculating a three-way independent-groups ANOVA is a simple extension of the method for a two-way independent-groups ANOVA, using the same basic formulas. In particular, there is really nothing new about calculating MS W (the error term for all the F ratios); it is just the ordinary average of the cell variances when the design is balanced. (It is hard to imagine that anyone would calculate an unbalanced three-way ANOVA with a calculator rather than a computer, so I will not consider that possibility. The analysis of unbalanced designs is described in general in Chapter 18, Section A). Rather than give you all of the cell standard deviations or variances for the example in Table 22.1, I ll just tell you that SS W equals 6,400; later I ll divide this by df W to obtain MS W. (If you had all of the raw scores, you would also have the option of obtaining SS W by calculating SS total and subtracting SS between-cells as defined in the following.) Main Effects The calculation of the main effects is also the same as in the two-way ANOVA; the SS for a main effect is just the biased variance of the relevant group means multiplied by the total N. Let us say that each of the eight cells in our example contains five subjects, so N T equals 40. Then the SS for the experimenter factor (SS exper ) is 40 times the biased variance of 36.5 and 46.5 (the nurturing and exploitive means from Table 22.1), which equals 40(25) = 1000 (the shortcut for finding the biased variance of two numbers is to take the square of the difference between them and then divide by 4). Similarly, SS alcohol = 40(132.25) = 5290, and SS gender = 40(12.25) = 490. The Two-Way Interactions When calculating the two-way ANOVA, the SS for the two-way interaction is found by subtraction; it is the amount of the SS between-cells that is left after sub-

10 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 697 Section A Conceptual Foundation 697 tracting the SSs for the main effects. Similarly, the three-way interaction SS is the amount left over after subtracting the SSs for the main effects and the SSs for all the two-way interactions from the overall SS between-cells. However, finding the SSs for the two-way interactions in a three-way design gets a little tricky. In addition to the overall SS between-cells, we must also calculate some intermediate two-way SS between terms. To keep track of these I will have to introduce some new subscripts. The overall SS between-cells is based on the variance of all the cell means, so no factors are collapsed, or averaged over. Representing gender as G, alcohol as A, and experimenter as E, the overall SS between-cells will be written as SS GAE. We will also need to calculate an SS between after averaging over gender. This is based on the four means (included in Table 22.1) I used to graph the alcohol by experimenter interaction and will be represented by SS AE. Because the design is balanced, you can take the simple average of the appropriate male cell mean and female cell mean in each case. Note that SS AE is not the SS for the alcohol by experimenter interaction because it also includes the main effects of those two factors. In similar fashion, we need to find SS GA from the means you get after averaging over the experimenter factor and SS GE by averaging over the alcohol factor. Once we have calculated these four SS between terms, all of the SSs we need for the three-way ANOVA can be found by subtraction. Let s begin with the calculation of SS GAE ; the biased variance of the eight cell means is , so SS GAE = 40(366.75) = 14,670. The means for SS AE are 35, 25, 38, 68, and their biased variance equals , so SS AE = 10,290. SS GA is based on the following means: 34, 26, 42, 64, so SS GA = 40(200.75) = 8,030. Finally, SS GE, based on means of 38, 38, 35, 55, equals 2,490. Next we find the SSs for each two-way interaction: SS A E = SS AE SS alcohol SS exper = 10,290 5,290 1,000 = 4,000 SS G A = SS GA SS gender SS alcohol = 8, ,290 = 2,250 SS G E = SS GE SS gender SS exper = 2, ,000 = 1,000 Finally, the SS for the three-way interaction (SS G A E ) equals SS GAE SS A E SS G A SS G E SS gender SS alcohol SS exper = 14,670 4,000 2,250 1, ,290 1,000 = 640 Formulas for the General Case It is traditional to assign the letters A, B, and, C to the three independent variables in the general case; variables D, E, and so forth, can then be added to represent a four-way, five-way, or higher ANOVA. I ll assume that the following components have already been calculated using Formula 14.3 applied to the appropriate means: SS A, SS B, SS C, SS AB, SS AC, SS BC, SS ABC. In addition, I ll assume that SS W has also been calculated, either by averaging the cell variances and multiplying by df W or by subtracting SS ABC from SS total. The remaining SS components are found by Formula 22.1: a. SS A B = SS AB SS A SS B Formula 22.1 b. SS A C = SS AC SS A SS C c. SS B C = SS BC SS B SS C d. SS A B C = SS ABC SS A B SS B C SS A C SS A SS B SS C At the end of the analysis, SS total (whether or not it has been calculated separately) has been divided into eight components: SS A, SS B, SS C, the four interactions listed in Formula 22.1, and SS W. Each of these is divided by its corresponding df to form a variance estimate, MS. Using a to represent the

11 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA number of levels of the A factor, b for the B factor, c for the C factor, and n for the number of subjects in each cell, the formulas for the df components are as follows: a. df A = a 1 Formula 22.2 b. df B = b 1 c. df C = c 1 d. df A B = (a 1)(b 1) e. df A C = (a 1)(c 1) f. df B C = (b 1)(c 1) g. df A B C = (a 1)(b 1)(c 1) h. df W = abc (n 1) Completing the Analysis for the Example Because each factor in the example has only two levels, all of the numerator df s are equal to 1, which means that all of the MS terms are equal to their corresponding SS terms except, of course, for the error term. The df for the error term (i.e., df W ) equals the number of cells (abc) times one less than the number of subjects per cell (this gives the same value as N T minus the number of cells); in this case df W = 8(4) = 32. MS W = SS W /df W ; therefore, MS W = 6400/32 = 200. (Reminder: I gave the value of SS W to you to reduce the amount of calculation.) Now we can complete the three-way ANOVA by calculating all of the possible F ratios and testing each for statistical significance: MS gender 490 F gender = = = 2.45 MSW 200 MS alcohol 5,290 F alcohol = = = MSW 200 MS exper 1,000 F exper = = = 5 MSW 200 MS A E 4,000 F A E = = = 20 MSW 200 MS G A 2,250 F G A = = = MSW 200 MS G E 1000 F G E = = = 5 MSW 200 MS G A E 640 F G A E = = = 3.2 MSW 200 Because the df happens to be 1 for all of the numerator terms, the critical F for all seven tests is F.05 (1,32), which is equal (approximately) to Except for the main effect of gender, and the three-way interaction, all of the F ratios exceed the critical value (4.15) and are therefore significant at the.05 level. Follow-Up Tests for the Three-Way ANOVA Decisions concerning follow-up comparisons for a factorial ANOVA are made in a top-down fashion. First, one checks the highest-order interaction

12 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 699 Section A Conceptual Foundation 699 for significance; in a three-way ANOVA it is the three-way interaction. (Twoway interactions are the simplest possible interactions and are called firstorder interactions; three-way interactions are known as second-order interactions, etc.) If the highest interaction is significant, the post hoc tests focus on the various simple effects or interaction contrasts, followed by appropriate cell-to-cell comparisons. In a three-way ANOVA in which the three-way interaction is not significant, as in the present example, attention turns to the three two-way interactions. Although all of the two-way interactions are significant in our example, the alcohol by experimenter interaction is the easiest to interpret because it replicates previous results. It would be appropriate to follow up the significant alcohol by experimenter interaction with four t tests (e.g., one of the relevant t tests would determine whether at-risk subjects differ significantly from controls in the exploitive condition). Given the disordinal nature of the interaction (see Figure 22.2), it is likely that the main effects would simply be ignored. A similar approach would be taken to the two other significant two-way interactions. Thus, all three main effects would be regarded with caution. Note that because all of the factors are dichotomous, there would be no follow-up tests to perform on significant main effects, even if none of the interactions were significant. With more than two levels for some or all of the factors, it becomes possible to test partial interactions, and significant main effects for factors not involved in significant interactions can be followed by pairwise or complex comparisons, as described in Chapter 14, Section C. I will illustrate some of the complex planned and post hoc comparisons for the threeway design in Section B. Types of Three-Way Designs Cases involving significant three-way interactions and factors with more than two levels will be considered in the context of mixed designs in Section B. However, before we turn to mixed designs, let us look at some of the typical situations in which three-way designs with no repeated measures arise. One situation involves three experimental manipulations for which repeated measures are not feasible. For instance, subjects perform a repetitive task in one of two conditions: They are told that their performance is being measured or that it is not. In each condition half of the subjects are told that performance on the task is related to intelligence, and the other half are told that it is not. Finally, within each of the four groups just described, half the subjects are treated respectfully and half are treated rudely. The work output of each subject can then be analyzed by a ANOVA. Another possibility involves three grouping variables, each of which involves selecting subjects whose group is already determined. For instance, a group of people who exercise regularly and an equal-sized group of those who don t are divided into those high and those relatively low on self-esteem (by a median split). If there are equal numbers of men and women in each of the four cells, we have a balanced design. More commonly one or two of the variables involve experimental manipulations and two or one involve grouping variables. The example calculated earlier in this section involved two grouping variables (gender and having an alcohol-dependent parent or not) and one experimental variable (nurturing vs. exploitive experimenter). To devise an interesting example with two experimental manipulations and one grouping variable, start with two experimental factors that are expected to interact (e.g., one factor is whether or not the subjects are told that performance on the experimental task is related to intelligence, and the other factor is whether or not the group of subjects run together will know

13 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA each other s final scores). Then, add a grouping variable by comparing subjects who are either high or low on self-esteem, need for achievement, or some other relevant aspect of personality. If the two-way interaction differs significantly between the two groups of subjects, the three-way interaction will be significant. The Two-Way RM ANOVA One added benefit of learning how to calculate a three-way ANOVA is that you now know how to calculate a two-way ANOVA in which both factors involve repeated measures. In Chapter 15, I showed you that the SS components of a one-way RM design are calculated as though the design were a two-way independent-groups ANOVA with no within-cell variability. Similarly, a two-way RM ANOVA is calculated just as shown in the preceding for the three-way independent-groups ANOVA, with the following modifications: (1) One of the three factors is the subjects factor each subject represents a different level of the subjects factor, (2) the main effect of subjects is not tested, and there is no MS W error term, (3) each of the two main effects that is tested uses the interaction of that factor with the subjects factor as the error term, and (4) the interaction of the two factors of interest is tested by using as the error term the interaction of all three factors (i.e., including the subjects factor). If one RM factor is labeled Q and the other factor, R, and we use S to represent the subjects factor, the equations for the three F ratios can be written as follows: MS Q MS R MS Q R F Q =, F R = F Q R = MSQ S MSR S MXQ R S Higher-Order ANOVA This text will not cover factorial designs of higher order than the three-way ANOVA. Although higher-order ANOVAs can be difficult to interpret, no new principles are introduced. The four-way ANOVA produces 15 different F ratios to test: four main effects, 6 two-way interactions, 4 three-way interactions, and 1 four-way interaction. Testing each of these 15 effects at the.05 level raises serious concerns about the increased risk of Type I errors. Usually, all of the F ratios are not tested; specific hypotheses should guide the selection of particular effects to test. Of course, the potential for an inflated rate of Type I errors only increases as factors are added. In general, an N-way ANOVA produces 2 N 1 F ratios that can be tested for significance. In the next section I will delve into more complex varieties of the threeway ANOVA in particular those that include repeated measures on one or two of the factors. A SUMMARY 1. To display the cell means of a three-way factorial design, it is convenient to create two-way graphs for each level of the third variable and place these graphs side by side (you have to decide which of the three variables will distinguish the graphs and which of the two remaining variables will be placed along the X axis of each graph). Each two-way graph depicts a simple interaction effect; if the simple interaction effects are significantly different from each other, the three-way interaction will be significant. 2. Three-way interactions can occur in a variety of ways. The interaction of two of the factors can be strong at one level of the third factor and close

14 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 701 Section A Conceptual Foundation 701 to zero at a different level (or even stronger at a different level). The direction of the two-way interaction can reverse from one level of the third variable to another. Also, a three-way interaction can arise when all of the cell means are similar except for one. 3. The main effects of the three-way ANOVA are based on the means at each level of one of the factors, averaging across the other two. A twoway interaction is the average of the separate two-way interactions (simple interaction effects) at each level of the third factor. A two-way interaction is based on a two-way table of means created by averaging across the third factor. 4. The error term for the three-way ANOVA, MS W, is a simple extension of the error term for a two-way ANOVA; in a balanced design, it is the simple average of all of the cell variances. All of the SS between components are found by Formula 14.3, or by subtraction using Formula There are seven F ratios that can be tested for significance: the three main effects, three two-way interactions, and the three-way interaction. 5. Averaging simple interaction effects together to create a two-way interaction is reasonable only if these effects do not differ significantly. If they do differ, follow-up tests usually focus on the simple interaction effects themselves or particular 2 2 interaction contrasts. If the threeway interaction is not significant, but a two-way interaction is, the significant two-way interaction is explored as in a two-way ANOVA with simple main effects or interaction contrasts. Also, when the three-way interaction is not significant, any significant main effect can be followed up in the usual way if that variable is not involved in a significant twoway interaction. 6. All three factors in a three-way ANOVA can be grouping variables (i.e., based on intact groups), but this is rare. It is more common to have just one grouping variable and compare the interaction of two experimental factors among various subgroups of the population. Of course, all three factors can involve experimental manipulations. 7. The two-way ANOVA in which both factors involve repeated measures is analyzed as a three-way ANOVA, with the different subjects serving as the levels of the third factor. The error term for each RM factor is the interaction of that factor with the subject factor; the error term for the interaction of the two RM factors is the three-way interaction. 8. In an N-way factorial ANOVA, there are 2 N 1 F ratios that can be tested. The two-way interaction is called a first-order interaction, the three-way is a second-order interaction, and so forth. EXERCISES 1. Imagine an experiment in which each subject is required to use his or her memories to create one emotion: either happiness, sadness, anger, or fear. Within each emotion group, half of the subjects participate in a relaxation exercise just before the emotion condition, and half do not. Finally, half the subjects in each emotion/relaxation condition are run in a dark, sound-proof chamber, and the other half are run in a normally lit room. The dependent variable is the subject s systolic blood pressure when the subject signals that the emotion is fully present. The design is balanced, with a total of 128 subjects. The results of the three-way ANOVA for this hypothetical experiment are as follows: SS emotion = 223.1, SS relax = 64.4, SS dark = 31.6, SS emo rel = 167.3, SS emo dark = 51.5; SS rel dark = 127.3, and SS emo rel dark = The total sum of squares is 2,344. a. Calculate the seven F ratios, and test each for significance.

15 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA b. Calculate partial eta squared for each of the three main effects (use Formula 14.9). Are any of these effects at least moderate in size? 2. In this exercise there are 20 subjects in each cell of a design. The levels of the first factor (location) are urban, suburban, and rural. The levels of the second factor are no siblings, one or two siblings, and more than two siblings. The third factor has only two levels: presently married and not presently married. The dependent variable is the number of close friends that each subject reports having. The cell means are as follows: Urban Suburban Rural No Siblings Married Not Married or 2 Siblings Married Not Married or more Siblings Married Not Married a. Given that SS W equals 1,094, complete the three-way ANOVA, and present your results in a summary table. b. Draw a graph of the means for Location Number of Siblings (averaging across marital status). Describe the nature of the interaction. c. Using the means from part b, test the simple effect of number of siblings at each location. 3. Seventy-two patients with agoraphobia are randomly assigned to one of four drug conditions: SSRI (e.g., Prozac), tricyclic antidepressant (e.g., Elavil), antianxiety (e.g., Xanax), or a placebo (offered as a new drug for agoraphobia). Within each drug condition, a third of the patients are randomly assigned to each of three types of psychotherapy: psychodynamic, cognitive/behavioral, and group. The subjects are assigned so that half the subjects in each drug/therapy group are also depressed, and half are not. After 6 months of treatment, the severity of agoraphobia is measured for each subject (30 is the maximum possible phobia score); the cell means (n = 3) are as follows: a. Given that SS W equals 131, complete the three-way ANOVA, and present your results in a summary table. SSRI Tricyclic Antianxiety Placebo Psychodynamic Not Depressed Depressed Cog/Behav Not Depressed Depressed Group Not Depressed Depressed b. Draw a graph of the cell means, with separate panels for depressed and not depressed. Describe the nature of the therapy drug interaction in each panel. Does there appear to be a three-way interaction? Explain. c. Given your results in part a, describe a set of follow-up tests that would be justifiable. d. Optional: Test the interaction contrast that results from deleting Group therapy and the SSRI and placebo conditions from the analysis (extend the techniques of Chapter 13, Section B, and Chapter 14, Section C). 4. An industrial psychologist is studying the relation between motivation and productivity. Subjects are told to perform as many repetitions of a given clerical task as they can in a 1-hour period. The dependent variable is the number of tasks correctly performed. Sixteen subjects participated in the experiment for credit toward a requirement of their introductory psychology course (credit group). Another 16 subjects were recruited from other classes and paid $10 for the hour (money group). All subjects performed a small set of similar clerical tasks as practice before the main study; in each group (credit or money) half the subjects (selected randomly) were told they had performed unusually well on the practice trials (positive feedback), and half were told they had performed poorly (negative feedback). Finally, within each of the four groups created by the manipulations just described, half of the subjects (at random) were told that performing the tasks quickly and accurately was correlated with other important job skills (self motivation), whereas the other half were told that good performance would help the experiment (other motivation). The data appear in the following table:

16 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 703 Section B Basic Statistical Procedures 703 CREDIT SUBJECTS PAID SUBJECTS Positive Negative Positive Negative Feedback Feedback Feedback Feedback Self Other a. Perform a three-way ANOVA on the data. Test all seven F ratios for significance, and present your results in a summary table. b. Use graphs of the cell means to help you describe the pattern underlying each effect that was significant in part a. c. Based on the results in part a, what post hoc tests would be justified? 5. Imagine that subjects are matched in blocks of three based on height, weight, and other physical characteristics; six blocks are formed in this way. Then the subjects in each block are randomly assigned to three different weight-loss programs. Subjects are measured before the diet, at the end of the diet program, 3 months later, and 6 months later. The results of the two-way RM ANOVA for this hypothetical experiment are given in terms of the SS components, as follows: SS diet = 403.1, SS time = 316.8, SS diet time = 52, SS diet S = 295.7, SS time S = 174.1, and SS diet time S = 230. a. Calculate the three F ratios, and test each for significance. b. Find the conservatively adjusted critical F for each test. Will any of your conclusions be affected if you do not assume that sphericity exists in the population? 6. A psychologist wants to know how both the affective valence (happy vs. sad vs. neutral) and the imageability (low, medium, high) of words affect their recall. A list of 90 words is prepared with 10 words from each combination of factors (e.g., happy, low imagery: promotion; sad, high imagery: cemetery) randomly mixed together. The number of words recalled in each category by each of the six subjects in the study is given in the following table: SAD NEUTRAL HAPPY Subject No. Low Medium High Low Medium High Low Medium High a. Perform a two-way RM ANOVA on the data. Test the three F ratios for significance, and present your results in a summary table. b. Find the conservatively adjusted critical F for each test. Will any of your conclusions be affected if you do not assume that sphericity exists in the population? c. Draw a graph of the cell means, and describe any trend toward an interaction that you can see. d. Based on the variables in this exercise, and the results in part a, what post hoc tests would be justified and meaningful? An important way in which one three-factor design can differ from another is the number of factors that involve repeated measures (or matching). The design in which none of the factors involve repeated measures was covered in Section A. The design in which all three factors are RM factors will not be covered in this text; however, the three-way RM design is a straightforward extension of the two-way RM design described at the end of Section A. This section will focus on three-way designs with either one or two RM factors (i.e., mixed designs), and it will also elaborate on the general principles of dealing with three-way ANOVAs, as introduced in Section A, and consider B BASIC STATISTICAL PROCEDIRES

17 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA the complexities of interactions and post hoc tests when the factors have more than two levels each. One RM Factor I will begin with a three-factor design in which there are repeated measures on only one of the factors. The ANOVA for this design is not much more complicated than the two-way mixed ANOVA described in the previous chapter for instance, there are only two different error terms. Such designs arise frequently in psychological research. One simple way to arrive at such a design is to start with a two-way ANOVA with no repeated measures. For instance, patients with two different types of anxiety disorders (generalized anxiety vs. specific phobias) are treated with two different forms of psychotherapy (psychodynamic vs. behavioral). The third factor is added by measuring the patients anxiety at several points in time (e.g., beginning of therapy, end of therapy, several months after therapy has stopped); I will refer to this factor simply as time. To illustrate the analysis of this type of design I will take the two-way ANOVA from Section B of Chapter 14 and add time as an RM factor. You may recall that that example involved four levels of sleep deprivation and three levels of stimulation. Performance was measured only once after 4 days in the sleep lab. Now imagine that performance on the simulated truck driving task is measured three times: after 2, 4, and 6 days in the sleep lab. The raw data for the three-factor study are given in Table 22.2, along with the various means we will need to graph and analyze the results; note that the data for Day 4 are identical to the data for the corresponding two-way ANOVA in Chapter 14. To see what we may expect from the results of a threeway ANOVA on these data, the cell means have been graphed so that we can look at the sleep by stimulation interaction at each time period (see Figure 22.7). You can see from Figure 22.7 that the sleep stimulation interaction, which was not quite significant for Day 4 alone (see Chapter 14, section B), increases over time, perhaps enough so as to produce a three-way interaction. We can also see that the main effects of stimulation and sleep, significant at Day 4, are likely to be significant in the three-way analysis. The general decrease in scores from Day 2 to Day 4 to Day 6 is also likely to yield a significant main effect for time. Without regraphing the data, it is hard to see whether the interactions of time with either sleep or stimulation are large or small. However, because these interactions are less interesting in the context of this experiment, I won t bother to present the two other possible sets of graphs. To present general formulas for analyzing the kind of experiment shown in Table 22.2, I will adopt the following notation. The two between-subject factors will be labeled A and B. Of course, it is arbitrary which factor is called A and which B; in this example the sleep deprivation factor will be A, and the stimulation factor will be B. The lowercase letters a and b will stand for the number of levels of their corresponding factors in this case, 4 and 3, respectively. The within-subject factor will be labeled R, and its number of levels, c, to be consistent with previous chapters. Let us begin with the simplest SS components: SS total, and the SSs for the numerators of each main effect. SS total is based on the total number of observations, N T, which for any balanced three-way factorial ANOVA is equal to abcn, where n is the number of different subjects in each cell of the A B table. So, N T = = 180. The biased variance obtained by entering all 180 scores is , so SS total = = 7, SS A is based

18 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 705 Table 22.2 PLACEBO MOTIVATION CAFFEINE Subject Subject Subject Row Day 2 Day 4 Day 6 Means Day 2 Day 4 Day 6 Means Day 2 Day 4 Day 6 Means Means None AB means Jet Lag AB means Interrupt AB means Total AB means Column means

19 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page Chapter 22 Three-Way ANOVA Figure Graph of the Cell Means in Table Motivation Caffeine Day 2 15 Placebo None Jet-Lag Interrupt Total Caffeine Motivation Placebo None Jet-Lag Interrupt Total 30 Day Caffeine Day 6 Motivation 10 7 Placebo 0 None Jet-Lag Interrupt Total on the means for the four sleep deprivation levels, which can be found in the rightmost column of the table, labeled row means. SS B is based on the means for the three stimulation levels, which are found where the bottom row of the table (Column Means), intersects the columns labeled Subject Means (these are averaged over the three days, as well as the sleep levels). The means for the three different days are not in the table but can be found by averaging the three Column Means for Day 2, the three for Day 4, and similarly for Day 6. The SSs for the main effects are as follows: SS A =σ 2 (25.58, 23.51, 17.35, 16.53) 180 = = 2, SS B =σ 2 (17.77, 21.38, 23.08) 180 = = SS R =σ 2 (23.63, 21.22, 17.38) = = 1,192.0 As in Section A, we will need the SS based on the cell means, SS ABR, and the SSs for each two-way table of means: SS AB, SS AR, and SS BR. In addition, because one factor has repeated measures we will also need to find the means for each subject (averaging their scores for Day 2, Day 4, and Day 6) and the SS based on those means, SS between-subjects.

20 Cohen_Chapter22.j.qxd 8/23/02 11:56 M Page 707 Section B Basic Statistical Procedures 707 The cell means we need for SS ABR are given in Table 22.2, under Day 2, Day 4, and Day 6, in each of the rows labeled AB Means; there are 36 of them (a b c). The biased variance of these cell means is , so SS ABR = = 5, The means for SS AB are found by averaging across the 3 days for each combination of sleep and stimulation levels and are found in the rows for AB Means under Subject Means. The biased variance of these 12 (i.e., a b) means equals , so SS AB = 3,974. The nine means for SS BR are the column means of Table 22.2, except for the columns labeled Subject Means. SS BR =σ 2 (20.3, 19.0, 14.0, 25.5, 21.0, 17.65, 25.1, 23.65, 20.5) 180 = 2, Unfortunately, there was no convenient place in Table 22.2 to put the means for SS AR. They are found by averaging the (AB) means for each day and level of sleep deprivation over the three stimulation levels. SS AR =σ 2 (27.13, 25.6, 24, 25.6, 24.47, 20.47, 21.27, 18.07, 12.73, 20.53, 16.73, 12.33) 180 = 4, Finally, we need to calculate SS between-subjects for the 60 (a b n) subject means found in Table 22.2 under Subject Means (ignoring the entries in the rows labeled AB Means and Column Means, of course). SS between-subjects = = 5, Now we can get the rest of the SS components we need by subtraction. The SSs for the two-way interactions are found just as in Section A from Formula 22.1a, b, and c (except that factor C has been changed to R): SS A B = SS AB SS A SS B SS A R = SS AR SS A SS R SS B R = SS BR SS B SS R Plugging in the SSs for the present example, we get SS A B = 3,974 2, = SS A R = 4, , ,192 = SS B R = 2, ,192 = The three-way interaction is found by subtracting from SS ABR the SSs for three two-way interactions and the three main effects (Formula 22.1d). SS A B R = SS ABR SS A B SS A R SS B R SS A SS B SS R SS A B R = 5, , = As in the two-way mixed design there are two different error terms. One of the error terms involves subject-to-subject variability within each group or, in the case of the present design, within each cell formed by the two between-group factors. This is the error component you have come to know as SS W, and I will continue to call it that. The total variability from one subject to another (averaging across the RM factor) is represented by a term we have already calculated: SS between-subjects, or SS bet-s, for short. In the one-way RM ANOVA this source of variability was called the subjects factor (SS sub ), or the main effect of subjects, and because it did not play a useful role, we ignored it. In the mixed design of the previous chapter it was simply divided between SS groups and SS W. Now that we have two between-group factors, that source of variability can be divided into four components, as follows: SS bet-s = SS A + SS B + SS A B + SS W This relation can be expressed more simply as SS bet-s = SS AB + SS W The error portion, SS W, is found most easily by subtraction: SS W = SS bet-s SS AB Formula 22.3