Introduction to Analysis of Variance (ANOVA) Limitations of the t-test

Transcription

1 Introduction to Analysis of Variance (ANOVA) The Structural Model, The Summary Table, and the One- Way ANOVA Limitations of the t-test Although the t-test is commonly used, it has limitations Can only test differences between groups High school class? College year? Can examine ONLY the effects of 1 IV on 1 DV Limited to single group OR repeated measures designs 1

2 Limitations of the t-test Testing differences between group means IV: Gender (Male & Female) IV: High-school class (First-year, Sophomore, Junior, & Senior) Using the t-test, we must either collapse categories or not run the analysis Limitations of the t-test 1 Independent Variable Gender differences in depression IV: Gender (Male & Female) DV: Level of depression (BDI score) Independent Variables Gender and social support on depression IV 1 : Gender (Male & Female) IV : Social support (High, Medium, & Low) DV: Level of depression (BDI score)

3 Limitations of the t-test or more Independent Variables Simultaneously examine the impact of or more IVs on a single DV Examine how the effects of or more IVs COMBINE to affect a single DV Limitations of the t-test Single time point OR repeated measures designs 1 group at time points = repeated measures groups at 1 time point = independent groups Single time point AND repeated measures designs or more groups at or more time points 3

4 The Analysis of Variance (ANOVA) The ANOVA can test hypotheses that the t-test cannot Probably the most commonly abused statistical test Many varieties of ANOVA One-Way (between subjects) Factorial ANOVA (between or within subjects) Repeated Measures (within subjects) Mixed-Model (between & within subjects) One-Way ANOVA Varieties of ANOVA 1 continuous Dependent Variable 1 Independent Variable consisting of or more categorical groups The one-way ANOVA with groups is equivalent to the independent groups t-test 4

5 Varieties of ANOVA Factorial ANOVA 1 continuous Dependent Variable or more Independent Variables consisting of or more categorical groups for each IV IVs = Two-Way Factorial ANOVA 3 IVs = Three-Way Factorial ANOVA We call these factorial designs because EACH level of each IV is paired with EVERY level of ALL other IVs x Contingency Table Level 1 IV Level IV 1 Level 1 Level DATA DATA Col 1 DATA DATA Col Row 1 Row Note: Each Level 1 of IV 1 is paired with BOTH Level 1 and Level of IV 5

6 x Contingency Table Social Support High Low Gender Male DATA DATA Row 1 Female DATA DATA Row Col 1 Col Note: Each Level 1 of IV 1 is paired with BOTH Level 1 and Level of IV Repeated Measures ANOVA Varieties of ANOVA Time points = IV The DV is assessed at EACH time point 6

7 Mixed-Model ANOVA Varieties of ANOVA 1 continuous Dependent Variable 1 or more Independent Variables consisting of or more categorical groups (between) 1 Independent Variable consisting of or more categorical time points (within) The DV is assessed at EACH time point ANOVA models we will consider One-Way ANOVA Two- and Three-way Factorial ANOVA Repeated measures ANOVA Mixed-model ANOVA ANOVA 7

8 Choosing the Best Test The Underlying Model A statistical model by example: Assume: the average 18 year old human being weighs approximately 138 pounds Men, on average, weigh 1 pounds more than the average human weight Women, on average, weigh 10 pounds less than the average human weight 8

9 The Underlying Model For any given human being, I can break weight down into 3 components: Average weight for all individual 138 lbs Average weight for each group Men: +1 lbs Women: - 10 lbs The individual s unique difference The Underlying Model Male weight Weight = 138 lbs + 1 lbs + uniqueness Female weight Weight = 138 lbs 10 lbs + uniqueness If you understand this process, you understand the basic theory behind the ANOVA 9

10 Partitioning Variance The idea behind the ANOVA test is to divide or separate (partition) variance observed in the data into categories of what we CAN and what we CANNOT explain Variance Explained Unexplained The Structural Model Mathematically, we partition the total variance of our data using the structural form of the ANOVA model X ij = µ + τ j +ε ij The structural model translates as follows: The score for any single individual is equal to the sum of the population mean plus the mean of the group plus the individual s unique contribution 10

11 For our weight example: The Structural Model µ = population weight = 138 lbs τ = group difference in weight = 1 or 10 lbs ε = unique contribution of an individual s score µ& τ can be explained ε cannot be explained Uniqueness Oftentimes, we value our uniqueness In statistics, unique variance is BAD Since we can t explain unique variance, we call it error Thus, the ANOVA seeks to examine the relative proportion of explainable variance in our data to the unexplainable variance 11

12 Assumptions of the ANOVA Owing to the mathematical construction of the ANOVA, the underlying assumptions of the test are very important Homogeneity of variance Normality Independence of Observations The Null Hypothesis Homogeneity of Variance Homogeneity of variance refers to the variance for each group being equal to the variance of every other group Really, we mean that the variance of each group is equal to the variance of the error for the total analysis σ 1 = σ = σ 3 = σ j = σ e 1

13 Homogeneity of Variance Heterogeneity of variance is another BAD thing Heterogeneous variances can greatly influence the results you obtain, making it either more or less likely that you will reject H 0 Visual inspection of variances Tests of homogeneity of variance Normality The ANOVA procedure assumes that scores are normally distributed More accurately, it assumes that ERRORS are normally distributed Random sampling and random assignment Lacking normality, consider mathematical transformations Logarithmic & square root transformations 13

14 Independence of Observations Simple: The scores for 1 group are not dependent on the scores from another group Don t share subjects between groups If violated What is wrong with your experimental design? Are you using the appropriate test? The Null Hypothesis Less an assumption and more a theoretical point: H 0 : µ 1 = µ = µ 3 = µ 4 = µ 5 This is almost ALWAYS the basic form of your null hypothesis 14

15 Calculating the One-Way ANOVA In order to calculate the One-Way ANOVA statistic, we need to complete a number of intermediate steps Because there are several intermediate steps, we keep track of our progress with something called a summary table The Summary Table Source df Sum of Squares (SS) Mean Square (MS) F 15

16 One-Way ANOVA: Partitioning Variance The idea behind the ANOVA test is to divide or separate (partition) variance observed in the data into categories of what we CAN and what we CANNOT explain Variance The Summary Table Source df (k-1) Sum of Squares (SS) Mean Square (MS) F k(n-1) (N-1) Note: df + df =df 16

17 Note: x j Sum of Squares = The treatment group mean x.. = The grand mean (mean of all scores) X ij = Each individual score The Summary Table Source df (k-1) Sum of Squares (SS) SS n( X X ) = j.. Mean Square (MS) F k(n-1) SS = SS SS (N-1) SS = ( X ij X.. ) 17

18 The Summary Table Source df (k-1) Sum of Squares (SS) SS n( X X ) = j.. Mean Square (MS) SS MS = df F k(n-1) SS = SS SS SS MS = df (N-1) SS = ( X ij X.. ) The Summary Table Source df (k-1) Sum of Squares (SS) SS n( X X ) = j.. Mean Square (MS) SS MS = df F MS F = MS k(n-1) SS = SS SS SS MS = df (N-1) SS = ( X ij X.. ) 18

19 Example: Anorexia Nervosa 3 Group Tx Control IPT CBT Example: Anorexia Nervosa 3 Group Tx Change in Weight Descriptives Control IPT CBT 95% Confidence Interval for Mean N Mean Std. Deviation Std. Lower Bound Upper Bound

20 Example: Anorexia Nervosa 3 Group Tx Test of Homogeneity of Variances Change in Weight Levene Statistic df1 df Sig H 0 : σ 1 = σ = σ 3 H 1 : σ 1 σ σ 3 Fail to reject H 0... Example: Anorexia Nervosa 3 Group Tx Change in Weight ANOVA Between Groups Within Groups Sum of Squares df Mean Square F Sig F(,34) =.54, p <.05 0

21 Example: Anorexia Nervosa Group Tx Control CBT Example: Anorexia Nervosa Group Tx Descriptives Change in Weight Control CBT 95% Confidence Interval for Mean N Mean Std. Deviation Std. Lower Bound Upper Bound Minimum Maximum

22 Example: Anorexia Nervosa Group Tx Test of Homogeneity of Variances Change in Weight Levene Statistic df1 df Sig H 0 : σ 1 = σ = σ 3 H 1 : σ 1 σ σ 3 Fail to reject H 0 Example: Anorexia Nervosa Group Tx Change in Weight ANOVA Between Groups Within Groups Sum of Squares df Mean Square F Sig F(1,4) = 36.44, p <.05

23 Example: Anorexia Nervosa Group Tx Independent Groups t-test Group Statistics Experimental Group Change in Weight Control CBT One-way ANOVA N Mean Std. Deviation Std. Mean Descriptives Change in Weight Control CBT 95% Confidence Interval for Mean N Mean Std. Deviation Std. Lower Bound Upper Bound Example: Anorexia Nervosa Group Tx Independent groups t-test Independent Samples Test Levene's Test for Equality of Variances t-test for Equality of Means Change in Weight Equal variances assumed Equal variances not assumed One-way ANOVA F Sig. t df Sig. (-tailed) Mean Difference Std. Difference ANOVA Change in Weight Sum of Squares df Mean Square F Sig. Between Groups Within Groups

24 Example: Anorexia Nervosa Group Tx Shouldn t the results of the F- and t-tests be equal if the tests are equivalent? F(1,4) = 36.44, p <.05 t(4) = -6.04, p <.05 t = F 4