Hypothesis Testing for Three or More Means. Sometimes we wish to compare the means of three or more independent populations rather than just two

Transcription

1 Hypothesis Testing for Three or More Means Sometimes we wish to compare the means of three or more independent populations rather than just two The two-sample t test can be extended to accommodate this situation This extension of the t test is known as the one-way analysis of variance Example: A study was conducted to follow 3 populations of overweight females for a period of 1 year Members of the first group decreased their energy intake by dieting, but did not participate in an exercise program 1

2 The second group exercised regularly but did not change their eating habits The third group changed neither diet nor level of physical activity At the end of 1 year, total change in body weight was measured for samples of individuals from each population Is there any difference in mean change in body weight among the 3 groups of women? Represent the 3 underlying population means by µ 1, µ 2, and µ 3 respectively We want to test the null hypothesis that the 3 means are equal, or H 0 : µ 1 = µ 2 = µ 3 2

3 The alternative hypothesis is that at least one of the population means differs from one of the others Why not just test all possible pairs of sample means using the two-sample t test? For 3 groups, we would need ( ) 3 2 = 3 tests This becomes more complicated as the number of samples increases A more serious problem is that this procedure is likely to lead to an incorrect conclusion Suppose that the null hypothesis is true and µ 1 = µ 2 = µ 3 3

4 Assuming that the 3 tests are independent and that each one is conducted at the 0.05 level of significance, P(fail to reject in all 3) = (1 0.05) 3 = Therefore, P(reject in at least 1) = = = P(type I error) This is much larger than 0.05! We would like the overall probability of committing a type I error to be some fixed value α 4

5 In general, we are interested in comparing the means of k independent and normally distributed populations H 0 : µ 1 = µ 2 = = µ k We must assume that the underlying population variances are all identical (σ 1 = σ 2 = = σ) As its name implies, one-way analysis of variance (ANOVA) focuses on estimates of variability When working with several different populations, two measures of variance can be calculated: (1) the variation of the individual values around their population means, and (2) the variation of the population means around the overall mean 5

6 If the variability within each of the k populations is small relative to the variability among the means, this implies that the population means are in fact different Therefore, we need to answer the following question: Does the variability in the data come mainly from the variation within groups, or is it mostly a result of the variation between groups? If most of the variability is between groups, this implies that the means are different from each other 6

7 Before specifying how to perform the test, we must first define some notation Select a random sample of size n i from the ith group (i =1, 2, 3 so sample sizes are n 1,n 2,n 3 ) Represent the jth observation in the ith group by x ij x i = n i j=1 x ij/n i is the sample mean for the observations in the ith group ( x 1, x 2, x 3 ) x is the sample mean for the observations in all groups combined x = 3i=1 n i j=1 x ij n 1 + n 2 + n 3 = n 1 x 1 + n 2 x 2 + n 3 x 3 n 1 + n 2 + n 3 7

8 x is sometimes called the grand mean or the overall mean To examine variability in the data, the deviation of an individual observation from the overall mean can be written x ij x = (x ij x)+( x i x i ) = (x ij x i )+( x i x) (x ij x i ) is the deviation of an individual observation in the ith group from the mean of that group and is a measure of within group variation ( x i x) is the deviation of the mean of the ith group from the overall mean and is a measure of between group variation 8

9 Square both sides of the equation and sum over all observations 3 n i i=1 j=1 (x ij x) 2 = 3 n i i=1 j= (x ij x i ) 2 n i i=1 j=1 3 n i i=1 j=1 2(x ij x i )( x i x) ( x i x) 2 It can be shown that 3 n i i=1 j=1 and therefore 2(x ij x i )( x i x) =0, 3 n i i=1 j=1 (x ij x) 2 = 3 n i i=1 j=1 3 + (x ij x i ) 2 n i i=1 j=1 ( x i x) 2 9

10 The term on the left is called the total sum of squares (SS T ) The first term on the right is the within groups sum of squares (SS W ) The second term on the right is the between groups sum of squares (SS B ) SS T =SS W +SS B Return to the original question is the variability between groups large relative to the variability within groups? We first find a measure of the within groups variability 10

11 Note that SS W = = 3 n i i=1 j=1 3 i=1 (x ij x i ) 2 (n i 1)s 2 = (n 1 1)s1 2 +(n 2 1)s2 2 +(n 3 1)s3 2 i If n = n 1 + n 2 + n 3, then a measure of variability is MS W = (n 1 1)s1 2+(n 2 1)s2 2+(n 3 1)s3 2 n 1 + n 2 + n 3 3 = SS W n 3 This is called the within groups mean square 11

12 Note that this expression is an extension of the pooled estimate of the common variance σ 2 used for the two-sample t test If the underlying variances are not the same, ANOVA should not be used However, the procedure is relatively insensitive to small departures from this assumption Now find a measure of the between groups variability 12

13 First note that SS B = = 3 n i i=1 j=1 3 i=1 ( x i x) 2 n i ( x i x) 2 = n 1 ( x 1 x) 2 + n 2 ( x 2 x) 2 + n 3 ( x 3 x) 2 A measure of variability is MS B = n 1( x 1 x) 2 +n 2 ( x 2 x) 2 +n 3 ( x 3 x) = SS B 3 1 This is called the between groups mean square 13

14 To test the null hypothesis that the population means are all the same, use the test statistic F = MS B MS W = SS B/(k 1) SS W /(n k) k represents the number of groups Under H 0, this statistic has an F distribution with k 1 and n k df H 0 is rejected if the value of this statistic is large, or, in particular, if F >F k 1,n k,1 α (critical value method) The p-value is the area under the curve F k 1,n k that lies to the right of F (p-value method) 14

15 Now return to the study investigating 3 groups of overweight females The first group dieted but did not exercise, the second group exercised but did not diet, and the third group neither dieted nor exercised At the end of one year, total change in body weight (kg) was measured for each individual Group n i x i s i 1 Diet Exercise Neither Note that k = 3 and n = 131 The sample sizes for the 3 groups are not the same 15

16 The overall sample mean is x = n 1 x 1 + n 2 x 2 + n 3 x 3 n 1 + n 2 + n 3 = 42( 7.2) + 47( 4.0) + 42(0.6) = The estimate of the within groups variability is MS W = SS W n k = (n 1 1)s1 2+(n 2 1)s2 2+(n 3 1)s3 2 n 1 + n 2 + n 3 3 = (41)(3.7)2 +(46)(3.9) 2 +(41)(3.7) = =

17 The estimate of the between groups variability is MS B = SS B k 1 = n 1( x 1 x) 2 +n 2 ( x 2 x) 2 +n 3 ( x 3 x) = 42( 3.65)2 +47( 0.45) 2 +42(4.15) = = We are evaluating the null hypothesis H 0 : µ 1 = µ 2 = µ 3 at the 0.05 level of significance 17

18 The test statistic is F = MS B MS W = = 45.4 For an F distribution with 3 1 = 2 and = 128 df, the area to the right of 45.4 is less than Since p<α, we reject the null hypothesis and conclude that the population mean change in body weight differs for at least two of the three groups of women 18

19 The results of a one-way analysis of variance are often displayed in the form of an ANOVA table All calculations are summarized Source of Variation SS df MS F p Between <0.001 Within Total More generally, the table takes the form Source of Variation SS df MS F p Between SS B k 1 MS B MS B /MS W Within SS W n k MS W Total SS T 19

20 What happens when the null hypothesis is rejected? We conclude that the population means are not all equal to each other, but we cannot be more specific than this We often want to conduct additional tests to determine where the differences lie Many different techniques for multiple comparisons exist they usually involve testing each pair of means individually while controlling the overall level of significance α 20

21 The choice of a particular procedure depends on a number of factors, including the type of comparisons to be made, whether the comparisons were specified before looking at the data or after, and whether all of the samples contain the same number of observations The simplest and perhaps most widely used multiple comparisons procedure is the Bonferroni correction To compare a total of k different means while ensuring that the overall significance level does not exceed α, use α = ( α k 2) as the significance level for each individual test 21

22 For example, to make all possible comparisons among a total of 5 means with an overall significance level of 0.05, set the significance level of each test at α = 0.05 ( 5 2) = = Use a version of the two-sample t test (equal variances) for each individual comparison To test the null hypothesis H 0 : µ i = µ j, calculate the test statistic t ij = ( x i x j ) 0 MSW [(1/n i )+(1/n j )] 22

23 Recall that MS W is an extension of the pooled estimate of the variance that uses data from all k samples, not just the two being tested Under H 0,t ij has a t distribution with n k degrees of freedom Example: Conduct tests of the null hypotheses µ 1 = µ 2, µ 1 = µ 3, and µ 2 = µ 3 for the three groups of overweight females Since there are a total of 3 hypothesis tests, each one must be conducted at the α = = level of significance 23

24 The three test statistics are t 1 vs 2 = ( 7.2 ( 4.0)) 0 (14.24)[(1/42) + (1/47)] = 3.99 t 1 vs 3 = ( ) 0 (14.24)[(1/42) + (1/42)] = 9.47 t 2 vs 3 = ( ) 0 (14.24)[(1/47) + (1/42)] = 5.74 Each of these test statistics has a t distribution with = 128 df 24

25 For t 128, each of these values cuts off an area of less than in the lower tail of the distribution Therefore, p<0.001 for each test, which is less than α = Each of the three null hypotheses is rejected We conclude that mean change in body weight is greatest (largest reduction) for overweight women who diet, followed by those who exercise The women who make no change in their lifestyle have the smallest reduction (and may actually gain a little weight) 25

26 A disadvantage of the Bonferroni correction is that it tends to be overly conservative and may fail to detect a difference that actually exists However, there are a number of other tests that could be used instead Note: Multiple comparisons issues can also arise in a non-anova situation in which large numbers of tests are being performed One option is to specify one primary hypothesis and several secondary ones Use caution when data-dredging significant p-values could indicate potential associations, or questions for future research 26

27 Keep in mind that one-way ANOVA depends upon a number of assumptions: The k random samples are drawn from independent populations The variances of the populations are identical The underlying data are approximately normally distributed Other options are available if these assumptions are not satisfied If the data are not normally distributed, the measurements could be transformed Alternatively, a test with less stringent assumptions could be used If the samples are not independent, repeated measures ANOVA might be appropriate 27