Analysis of Variance: General Concepts

Transcription

1 Research Skills for Psychology Majors: Everything You Need to Know to Get Started Analysis of Variance: General Concepts This chapter is designed to present the most basic ideas in analysis of variance in a non-statistical manner. Its intent is to communicate the general idea of the analysis and provide enough information to begin to read research result sections that report ANOVA analyses. Analysis of Variance is a general-purpose statistical procedure that is used to analize a wide range of research designs and to investigate many complex problems. In this chapter we will only discuss the original, basic use of ANOVA: analysis of experiments that include more than two groups. When ANOVA is used in this simple sense, it follows directly from a still simpler procedure, the t-test. The t-test compares two groups, either in a between-subjects design (different subjects in the groups) or a repeated-measures design (same subjects assessed twice). ANOVA can be thought of as an extension of the t-test to situations in which there are more than two groups (one-way design) or where there is more than one independent variable (factorial design). These situations are the most common in research, so ANOVA is used far more frequently than t-tests. Variance is Analyzed The name analysis of variance is more representative of what the analysis is about than t-test because we are in fact focusing on analyzing variances. The conceptual model for ANOVA follows the familiar pattern first introduced in the Inferential Statistics chapter: a ratio is formed between the differences in the means of the groups and the error variance. In the same way that a variance (or standard deviation) can be calculated from a set of data, a variance can be calculated from a set of means. So the differences among the means is thought of as their variance: higher variance among the means indicates that there are more differences (which is good, right?). The variance among the group means is called the between-groups variance. A Deeper Truth Actually, the t-test is a special case of ANOVA. ANOVA is the real thing. A Still Deeper Truth Actually, ANOVA is a simplification of very complex correlations. Correlation is the real thing. The ratio, then, is between-groups variance divided by error variance. A larger ratio indicates that the differences between the groups are greater than the error or noise going on inside the groups. If this ratio, the F statistic, is large enough given the size of the sample, we can reject the null hypothesis. The whole story in ANOVA is figuring out how to calculate (and understand) these two types of variance W. K. Gabrenya Jr. Version: 1.0

2 Page 2 A Visual Example Here is an example of a one-way, between-groups design that would be analyzed using ANOVA. Four groups of participants are randomly sampled from four majors on campus. We will not identify the majors for the sake of interdepartmental harmony, but the identity of Group 4 is clear. Each sample includes five students. They are each administered the Wechsler Adult Intelligence Scale (WAIS-III) to obtain a measure if IQ. IQs have a mean of 100 in the population as a whole. Our question: which major is smarter? The following table presents the raw data (IQ scores), the means within each group, the standard deviation within each group, and the variance. The variance is simply the SD squared, a more useful number for certain aspects of the calcultions. It is normal that some of the SDs are larger than others. The gray bars below the scale represent the range of the IQs in each major, which is one indication of the within-group variability. (A wider range often produces a higher SD.) In the last column, the mean of the means (grand mean), the standard deviation of the means, and the variance of the means are presented. Data: 80, 85, 90, 95, 100 Group 1 Group 2 Group 3 Group 4 Grand Mean 90, 93, 96, 99, , 100, 103, 106, , 110, 115, 120, 125 Mean: Std. Dev.: Variance: Grand Mean (the group means could go here) Gp Group 4 Group 2 Group 3 What s the null hypothesis? The null condition is that there is no difference between the population means: H 0 : µ 1 = µ 2 = µ 3 = µ 4, where µ is mu, the population mean. Our task is to determine if the sample means, presented in the table above, are sufficiently different from each other compared to the error variance within the groups, to reject the null hypothesis. Of course the sample size will also affect the outcome because larger samples allow for better tests of the null hypothesis. In the language of ANOVA, we will look at the ratio of the between-group variance to the within-group (error) variance. Between-groups variance F = Error Variance Within Groups

3 Page 3 In the example, we have included the individual data for group 1 as circles inside the group 1 gray bar. The SD of group 1, 7.9, is computed from these 5 values. Recall that the SD is the variability of the individual data based on how distant each one is from the group mean (90.0). In other words, it is a measure of the extent to which the five students sampled for that major are not exactly of the same intelligence. The student in group 2 are more similar to each other and produce a SD of 4.7. The overall error variance for the sample is computed by combining these four SDs (see sidebar). Calculating the Variances Within-Groups (Error) Variance: The overall amount of error variance is the combined variances of the four groups. Combining the variances from several groups together is called pooling, so the resulting combined variance is termed the pooled variance. Averaging the variances in this study produces a pooled error variance of =42.5. Between-Groups Variance: Calculation of the between-groups variance is not as intuitive as the whithin-groups variance. Conceptually, it seems that the SD of the four group means would be a good measure. (The SD of the means is 10.7.) However, the actual between-groups SD is 24.0, so the between-groups variance is 24 2 = 577. The between-groups variability is computed in the same way, but we look at how much the group means vary from the grand mean (the mean of the means). The higher this variability, the more the means differ from each other and the more the null hyothesis looks rejectable. (See sidebar.) Finally, the ANOVA The ANOVA focuses on the ratio of the between group variance to the withingroups variance. SPSS produces an ANOVA source table to report the result of the analysis.this table is called a source table because it identifies the sources of variability in the data. As explained above, there are two kinds of variability, variability between group means, and variability within groups (error variance). The source table provides information about these two sources. The column numbers have been added for our use. Column 3, reflecting the number of groups and the sample size, is discussed in the sidebar. Column 4 presents the variance associated with the mean differences (between groups) and within-group error. These numbers are discussed in the Variances sidebar. Column 5 is the ratio of these two values, the F statistic. Column 6 presents the p-value (see Inferential Statistics chapter) of the F statistic based on the sample size. Because our normal criterion for rejecting the null hypothesis is p<.05, this p value is very good (good = low), and ANOVA Source Table Between Groups Within Groups Sum of Squares df Mean Square F Sig Total Degrees of Freedom in ANOVA All statistics, such as F, t, and chi-square, are evaluated in the context of the sample size: larger samples allow lower statistical values to reach the magic.05 level of confidence. The sample size is expressed in terms of degrees of freedom (df). Your statistics class has more to say about df. In a t-test, the df is the sample size minus 2 (N-2). In ANOVA, we use two df values. The df-error is based on the sample size: df e = (n g -1), where n g is the size of each of the group samples 16 = (5-1) + (5-1) + (5-1) + (5-1) ANOVA also requires a df for the number of groups: df bg = g-1, where g is the number of groups The F statistic is always presented along with these df values, e.g., F(3,19)=13.6, p<.0001.

4 Page 4 we can reject the null hypothesis. What has been rejected? By rejecting the null hypothesis, we conclude that the four means are not equal in the population, that is, all majors are not created equal. However, what it does not tell us is exactly which major is smarter than which other major. Is Group 4 smarter than Group 2, or just smarter than the hapless Group 1? Just eyeballing the means is not good enough: we need to know if the differences between particular pairs of means are really significantly different. How is this done? One way is to perform t-tests between pairs of means (there are several other ways as well). Using SPSS to Calculate One-Way ANOVA A one-way ANOVA is an analysis in which there is only one independent variable, as the preceding example. This is the simplest kind of ANOVA, and SPSS dedicates a procedure purely to it. (See menu screen illustration.) The dialog window in which the details of the analysis are entered is quite simple. In the dialog illustration, the dependent variable (IQ) and the independent variable (group) have been entered. In the Options... dialog you can ask for descriptive statistics and a rather sorry looking graph of the means. Syntax: ONEWAY iq BY group /STATISTICS DESCRIPTIVES /PLOT MEANS /MISSING ANALYSIS. The principal output of the procedure is the source table shown above. In a paper, the approprate way to report the results of an ANOVA is a variation of: A one-way between-groups ANOVA revealed a significant effect of major, F(3,16) = 13.6, p <.05. Note that the ANOVA used two types of degrees of freedom: the between-groups df and the error df.

5 Page 5 Factorial ANOVA If indeed the truth lies in the interactions, then we need to perform more complicated studies that include more than one IV. Factorial designs of this kind were introduced in the research designs chapter. For example, in the study presented above, we might want to know if gender is related to IQ. The obvious design would be a 4x2 between-subjects factorial: four majors crossed with gender. In the table below, the 40 students are indicated by S 1...S 40 in the 8 cells of the factorial design. Group 1 Group 2 Group 3 Group 4 Male S 1, S 2, S 3, S 4, S 5 S 11, S 12, S 13, S 14, S 15 S 21, S 22, S 23, S 24, S 25 S 31, S 32, S 33, S 34, S 35 Female S 6, S 7, S 8, S 9, S 10 S 16, S 17, S 18, S 19, S 20 S 26, S 27, S 28, S 29, S 30 S 36, S 37, S 38, S 39, S 40 The mathematics of a factorial ANOVA are more complicated than those of the one-way ANOVA, but the principles are the same. The ANOVA compares the variability due to between-groups differences to the amount of error variance in the sample. However, in this two-way factorial, we need to look at three types of between-groups variability: the variability between the majors, the variability between the genders, and the interaction effect variability. A ratio (F statistic) of between-subjects variability to error variance is calculated for each of these three types of between-groups variability. How many null hypotheses are there? SPSS and Factorial ANOVA The simple one-way ANOVA procedure cannot be used. Instead, factorial ANOVAs are produced by the SPSS GLM procedure. GLM mean general linear model. You will study the GLM in your second year of graduate-level statistics. GLM is a very powerful and flexible procedure that was only introduced to SPSS in the 1980s. Because it is powerful and flexible, it can be configured in many ways and has a large number of options. Univariate refers to the fact that you will be analyzing one dependent variable at a time. The IQ across majors study presented previously was enhanced by adding gender as a second independent variable to serve as an example of a factorial ANOVA. The analysis dialog box shown here has been configured to run this 4x2 ANOVA. Use the Fixed Factors box for the IVs. Ignore the boxes below that until you get to graduate school. You can specify in detail which means tables you would like to see display in the output by clicking on Options. What Other Goodies are in this Menu? Multivariate ANOVA (MANOVA) allows you to analyze several DVs simultaneously, in a single set. Repeated Measures ANOVA analyzes the repeated measures designs introduced in the research designs chapter.

6 Page 6 Double-clicking on the items in the left-side box moves them to the right side Display Means for box. In this case, moving group to the Display Means box produces a means table that includes just the main effect of group. The group*gender item displays a 4x2 table of means from which you can see if there is an interaction effect. Syntax: UNIANOVA iq BY group gender /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /EMMEANS = TABLES(group*gender) /CRITERIA = ALPHA(.05) /DESIGN = group gender group*gender. The Output The source table in a factorial ANOVA expands on that of the one-way ANOVA. Two additional sources are reported: the second IV, and the interaction effect. (See Tests of Between-Subjects Effects table.) The only rows of importance in this source table are those indicating the effects in the factorial model: GROUP, GENDER, and GROUP*GENDER. The F statistics in this type of source table are calculated by dividing a factor s Mean Square by the Mean Square of the Error row. Mean Square is another way of saying variance. Hence, F for the Group factor is: MS group /MS error = 477.2/ = These results show that the Group and Gender main effects are significant at a very low p value. SPSS will not print all of the significant digits of a very small p value. For Group, the actual p value is , but no one cares because it is so far below.05. The Group X Gender interaction is not significant because the p value is so large (p =.567). In a paper, there are several forms for reporting the results of a factorial ANOVA: A 4 (major) x 2 (gender) between-groups ANOVA revealed

7 Page 7 significant main effects of major, F(3,12) = 33.7, p <.05, and gender, F(1, 12) = 33.9, p <.05. The interaction effect did not approach significance, F < 1. or, if the interaction had been stronger: A 4 (major) X 2 (gender) between-groups ANOVA revealed significant main effects of major, F(3,12) = 33.7, p <.05, and gender, F(1, 12) = 33.9, p <.05. However, these main effects must be interpreted within the significant Major X Gender interaction, F(3,12) = 8.5, p <.05. Note that the ANOVA used two types of degrees of freedom: the between-groups df and the error df. Digging Deeper Overall, do major and gender help us know what students IQs are? Said another way, do major and gender predict IQ? The Corrected Model row in the source table answers this general question: yes. The idea of a model was introduced in an early chapter. Here, the model is expressed mathematically: IQ = ƒ (major, gender) The Corrected Model essentially combines all the predictors of IQ (Group, Gender, and their interaction) to see if, as a whole, they predict the dependent variable. (Hint: add the df.) Of course, we usually don t care about the whole model, but rather only about its component parts, the individual IVs. The Intercept row in the table is Tests of Between-Subjects Effects Dependent Variable: IQ Source Corrected Model Type III Sum of Squares Reprise of Still Deeper Truth The intercept reveals a clue to the ridiculous conspiracy theory that ANOVA is just a lot of correlations. Do you remember the equation for a line from algebra? In statistics we call this a regression line, and write the equation as y is the dependent variable, IQ x is sort of the independent variables, major, gender, and the interaction, all rolled into one (sort of) y = a + bx + e a is the y-intercept of the line b is the slope of the line e is the error variance In the ANOVA table, the intercept F-test is testing if the y-intercept (a) is different than zero. In a certain sense, the corrected model F-test is testing whether the slope (b) is different than zero. When the slope is different than zero, the independent variables (x) affect the dependent variable (y). In the manner of a correlation, a (b) near 1.0 and a low (e) gives us a correlation scattergram with a long, skinny oval. (i.e., a good correlation). Error variance (e) is analogous to the fatness of the oval. df Mean Square F Sig Intercept GROUP GENDER GROUP * GENDER Error Total Corrected Total R Squared =.929 (Adjusted R Squared =.888) y-intercept (a) y (DV) Line, slope (b) x Skinny oval and slope near 1.0 indicates high correlation coefficient (IV)

8 Page 8 not usually important. It compares the grand mean (101.0) to zero. Because 101 is so far from zero, the F is enormous. (But see the sidebar for its deeper meaning.) What s Next? A lot more...