Stat 101. Unit 11: Analysis of Variance. Textbook Chapter 12

Transcription

1 Stat 101 Unit 11: Analysis of Variance Textbook Chapter 1

2 Unit 11 Outline Analysis of Variance (ANOVA) General format and ANOVA s F-test Link to binary regression Contrast testing Multiple comparisons (& the Bonferroni correction) Two-way ANOVA (and Multi-way ANOVA)

3 Recall the Jumping Rats Example As always, first visualize the data: Groups 1 No jumping 30 cm jump 3 60 cm jump Means & SD s We d like to do a t-test, but there s no formula for 3 groups Solution: Analysis of Variance (ANOVA) 3

4 Analysis of Variance (ANOVA) ANOVA extends hypotheses that we have seen already With one population: H 0 : μ = μ 0 With two populations: H 0 : μ 1 = μ Now consider inferences for g = 3 or more populations (technically, g > ): H 0 : μ 1 = μ =... = μ g Analysis technique for this situation is ANOVA 4

5 Main concept of ANOVA With I populations (groups), there are two types of variability in the data: (A) Variation of individual values around their group means ( groups (variability within (B) Variation of group means around the overall mean ( groups (variability between Main concept: If (A) is small relative to (B), this implies the group means are different. ANOVA determines whether variability in data is mainly from variation within groups or variation between groups. That is, does labelling the data into these groups lead to more than chance differences in the group means Think Boxplots split by groups 5 5

6 Within group variability relatively large, adds noise, obscures group differences Within group variability relatively small, group differences not obscured by noise 6 6

7 One-way ANOVA the model The one-way ANOVA model for a quantitative variable X is x ij = μ i + ε ij for i = 1,,..., g groups (subpopulations)for a specific factor and j = 1,,..., n i individuals sampled in each group ( σ The ε ij are assumed to be ~ N(0, Model parameters are μ 1, μ,..., μ g and σ (one common SD!) Model can be written as (just like regression!) DATA = MODEL + RESIDUAL Note: we used the symbol x here to represent the measured variable, but we could have chosen y instead (to mimic regression s response variable) 7 7

8 Model for one-way ANOVA with g = 3 groups ( σ Residuals ε ij are assumed to be ~ N(0, So only difference among the groups are differing means 8 8

9 One-way ANOVA the data SRS s from each of the I populations (groups): Sample size n 1 n n g Observations x 1j x j x gj for j = 1,,, n i Sample means x1 x xg Sample SD s s 1 s s g Practical rule for examining SD s in ANOVA (checking assumption of a common σ.( If largest s is less than twice the smallest s, the ANOVA results will be approximately correct. Check to see if the ratio holds: s s largest smallest 9 9

10 Variance within groups Since we assumed equal σ s for all g groups, all sample SD s should be estimating the common σ Thus we combine them in a pooled estimate (same idea as for ( the -sample pooled t-test which we skipped Pooled estimate of σ (the variance within groups) is s W ( n 1 1) s SSError dferror 1 ( n MSE 1) s n The subscript W refers to this within groups estimate g... ( n 1) s Also known as the mean square within groups (MSW) and mean square error (MSE) g g g i1 ( n i n 1) s g i

11 Variance between groups If the null hypothesis, H 0 : μ 1 = μ =... = μ g, is true then examining individual sample means is as if we are sampling g times from the same population, with mean μ and standard deviation σ Recall from the sampling distribution of sample means: s so under H 0, σ can be estimated by B n 1 ( x1 x) n( x x)... ng ( xg x) i1 g 1 SSGroups dfgroups MSG The subscript B (on s B ) refers to this between groups estimate Also known as the mean square between groups (MSB or MSG) or mean square of the model (MSM) Only a valid estimate of σ if H 0 : true, otherwise inflated x g ~ n N, i ( x g i 1 n x)

12 Concept behind the test If H 0 : μ 1 = μ =... = μ g is true then s W and s B both estimate σ and should be of similar magnitude If H 0 : is not true, the between groups estimate of σ will, in general, be larger than the within groups estimate of σ Therefore a test of H 0 : μ 1 = μ =... = μ g can be based on a comparison (ratio) of the between groups and within groups estimates of σ Examine this ratio of variance estimates as an F test F s s B W MS between groups MS within groups MSGroups MSError 1 1

13 One-way ANOVA: F-test To test H 0 : μ 1 = μ =... = μ g H A : not all μ i are equal, use the F statistic F MSG MSE SSG SSE / / df df G E This test statistic has an F distribution with g 1 and N g degrees of freedom when H 0 is true Reject H 0 : for large values of F; if F > F* α, g 1, N - g or the corresponding p-value is less than α

14 Analysis of variance (ANOVA) As for simple and multiple linear regression, for one-way ANOVA the total sum of squares is decomposed by: SST = SSG + SSE The degrees of freedom are now partitioned DFT = DFG + DFE (n - 1) = (g - 1) + (n - g) The mean squares (MS) are formed the same way MS SS df Sum of Squares degrees of freedom 14 14

15 One-way ANOVA table Source SS DF MS F Groups (Between) s B SSG g 1 = SSG/DFG F = MSG/MSE Error s W (Within) SSE n g = SSE/DFE Total SST n 1 SST/DFT The F statistic tests if there is a difference among any of the g population means MSE is still our estimate of σ : the variance of the residuals or the average variance of the observations from their group means (sometime called the pooled variance estimate). 15

16 Solution: H 0 : μ 1 = μ = μ 3 H A : at least one μ i is different g SSG ni i1 10( ) 10( ) 10( ) g SSE ( ni i1 ( x x) MSG i 1) s i MSE SSG / df ( G SSE / df /(3 1) ) ( E ) ( F = / = F* = Since our F > F*, we reject H 0. The rat s have differing mean bone densities in the different treatment groups /( ) )

17 ANOVA Results from SPSS In SPSS: Analyze Compare Means One-Way ANOVA 17 17

18 Unit 11 Outline Analysis of Variance (ANOVA) General format and ANOVA s F-test Link to binary regression Contrast testing Multiple comparisons (& the Bonferroni correction) Two-way ANOVA (and Multi-way ANOVA) 18 18

19 The link between ANOVA and Regression ANOVA treats the groups as `nominal variables, i.e., variables whose coding gives the groups a name but no numerical meaning A naive application of regression here might use the codes no jump = 1, low jump =, high jump = 3 in a single predictor, but this is mathematically incorrect! A correct way to apply regression here is to create (g 1) binary variables (variables coded (0 or 1), sometimes called dummy variables) that recreate the groups: 19 19

20 A few thoughts about ANOVA ANOVA is easy to calculate However, regression is more general because additional continuous variables can be added ANOVA generalizes to several categorical predictor variables/factors (Ch 13), in two-way and higher ANOVA. Note: the term ANOVA is thrown around a lot. It can refer to the ANOVA setting/procedure where you are looking at a quantitative variable across multiple groups (or treatments). It can also refer to the ANOVA table (that is calculated under both regression and ANOVA settings). 0 0

22 Steps in a Complete ANOVA Procedure 1. Examine the data, checking assumptions. If possible, formulate in advance some working (alternative) hypotheses about how the population group means might differ. 3. Check the evidence against the global null hypothesis of no differences among the groups by calculating the F-test 4. If the F-test is significant (that is, if it leads to a rejection of the null hypothesis of equal population group means): Test the individual hypotheses specified at the second step If there was not enough information to formulate working hypotheses, test all pairwise comparisons of means, adjusting for multiple comparisons (example also coming)

23 Contrasts After the omnibus F test has shown overall significance investigate other comparisons of groups using contrasts can be performed A contrast is a linear combination of μ i s i a i ( i ) where a 0 The corresponding contrast of sample means is (ψ is the Greek psi ). In this rats example, what might be an interesting comparison of the 3 groups involved (no jump, low jump, high jump)? For example, consider the bone density study. To compare control (group 1) versus treatment (groups and 3) use. ψ = μ 1 (μ + μ 3 )/ = (1)μ 1 +(- 1 / )μ + (- 1 / )μ 3 To compare levels of jumping (group versus group 3) use ψ = μ μ 3 = (0)μ 1 +(1)μ + ( 1)μ c a i x ) ( i

24 Contrasts testing hypotheses To test the hypothesis H versus 0 : 0 H A : 0 we use a t-test much like the pooled t-test: t s p a x i i a n i i This t-test has a t distribution with degrees of freedom associated with s p under H 0 (test can be 1-sided or -sided) 4 4

25 Example bone density Analysis of variance for bone density s p Reject H 0 : μ 1 = μ = μ 3 if our calculated F > F* 0.05,df=,7 = 3.35 Omnibus test: P = 0.00, reject H 0 and conclude there is a difference in bone density among the three groups. Where does that difference lie? Let s look at the comparisons (via contrasts): (Group 1) versus (Groups and 3) (Group ) versus (Group 3) 5 5

26 Example bone density Does jumping (any level) affect bone density? Set-up hypotheses: H H 0 A : : Control Control Calculate t-test t s p a i ( x 0.5( i a n i i lowjump lowjump highjump highjump Calculate the p-value: df = N I = 30 3 = 7. p-value = *P(t<-.93) < (0.005) = So reject H 0 and conclude any kind of jumping improves bone density over no jumping at all ) ) (1)601.1 ( 0.5)61.5 ( 0.5) ( 0.5) 10 ( 0.5)

27 Example bone density Does the level of jumping affect bone density? Look at group versus group 3 using the contrast H H 0 A : (0) : (0) Control Control (1) (1) lowjump lowjump ( 1) ( 1) highjump highjump 0 0 t s p a i i x i a n i (0)601.1 (1)61.5 ( 1) (1) 10 ( 1) 10 So reject H 0 (since our p-value < 0.0 < α) and conclude higher jumping affects bone density differently than moderate jumping (in fact, it makes bones denser). 7 7

29 The multiple comparisons problem To test H 0 : μ 1 = μ =... = μ g, why not simply conduct multiple twosample t-tests? For example, with g = 9 there are 36 possible pair-wise t-tests. What is the probability of rejecting a true H 0 at least once? P(Type I error) = 1 ( ) = 0.84 This inflated Type I error is due to multiple comparisons: we have looked at multiple tests at once (each with α = 0.05), and thus it will lead to significant results that are not truly there (simply by chance). In general, we would like the probability of a type I error to be some fixed value α (e.g., α = 0.05) This is accomplished using the overall ANOVA F-test What happens when we start using multiple contrasts? If we don t have any contrasts pre-specified, we can just look at all the pairwise two-sample t-tests, but adjust α. 9 9

30 The Bonferroni correction A solution to the multiple comparisons problem is the adjustment of α levels using the Bonferroni correction This correction is a conservative solution Suppose we wish to perform all possible pairs of comparisons among g groups: There are g g!!( g )! g( g 1) such comparisons The Bonferroni correction - to protect the overall level of α we must perform each individual test at level α * * g 30 30

31 Example - Bonferroni correction Suppose we wish to perform pair-wise comparisons among 3 groups but still maintain an overall α = 0.05 If g = 3, there are 3 3!!(1)! possible comparisons: (Group 1 versus ), (1 versus 3), and ( versus 3) The Bonferroni correction says that if we want an overall α level of < 0.05, then we do each of the 3 tests at the α* = 0.05 / 3 = level Thus, with each test at level α* = , this Bonferroni correction gives an overall α <

32 Bonferroni Correction in SPSS Then the pairwise comparisons can be made when you run the one-way ANOVA. Just click on the Post Hoc button, and select Bonferroni : * x1 x tdf 7sex ( 13.4, 36.04) x 11.4 Note: these confidence intervals are wider than those from the regression (or if you did them separately based on the t-stats for comparing two means). Here they use (gotten from an online calculator): * t /, df

33 Using Multiple Comparisons to your Advantage (bad use of statistics!!!) 33

35 Extension to Two-way ANOVA (and beyond!) We can expand ANOVA to include an additional factor (a second grouping variable) call it two-way ANOVA For example, we could predict text messaging based on class year (with g = 4 groups) and sex (with g = groups), and even a third variable like house (g = 1?) or a fourth like concentration (with g =???). Algebraically, things get a little more complicated, but we won t focus on that in this course Having multiple grouping variables (just like predictor variables) also allows us to include interaction terms as well Easy to perform in SPSS!!! Analyze General Linear Model Univariate 35 35

36 Two-way ANOVA in SPSS The example we will use is trying to predict the ln_text messages by Harvard College students based on their class year and sex. In SPSS: Analyze General Linear Model Univariate Click on the Model button, select Custom, and drag the grouping variables over into the model. Click Continue then OK

37 Two-way ANOVA in SPSS (with interaction) Here are the results in SPSS for this sample if n = 169 students What should be the 3 different sets of degrees of freedoms here? The Corrected Model row tests for the model overall, and the gender and classyear rows tests for each variable separately. Note: you can ignore the Intercept and Total rows, and always compare to the Corrected Total row. SS s will not add up

38 Two-way ANOVA in SPSS (with interaction) We can also consider the interaction between the two grouping variables here (sex and class year) What would the interaction term represent? The effect of class year on # text messages sent may be different for men than women Equivalent to saying that the effect of women on # text messages sent may be different in the 4 class years Maybe men s average decreases over the 4 years, but women s average stays the same What should be the degrees of freedom for the interaction between these two variables? (g 1-1)*(g -1) = (4-1)*(-1) =

39 Two-way ANOVA in SPSS (with interaction) Here are the results in SPSS for this sample if we include the interaction between the grouping variables as well (which is sometimes called the full factorial ANOVA model this is the default in SPSS)

40 ANOVA: Main Points Simple idea behind ANOVA: significant difference among or more groups if variability between groups (differences among the group means) is significantly larger than variability within groups (differences from mean within each group) [F-test]. If there is evidence of difference among groups, then an a priori hypothesis can be tested via a contrast t-test, or some care must be taken in searching for the group pairs that are significantly different (using Bonferroni correction). ANOVA can be expanded to include multiple grouping variables (Multi-way ANOVA). Algebra is hard, but SPSS does the work for us. Interactions can then be considered