Multiple/Post Hoc Group Comparisons in ANOVA

Transcription

1 Multple/Pot Hoc Group Comparon n ANOVA Note: We may ut go over th quckly n cla. The key thng to undertand that, when tryng to dentfy where dfference are between group, there are dfferent way of adutng the probablty etmate to reflect the fact that multple comparon are beng made. Introducton. In a one-way ANOVA, the F tattc tet whether the treatment effect are all equal,.e. that there are no dfference among the mean of the J group. A gnfcant F value ndcate that there are dfference n the mean, but t doe not tell you where thoe dfference are, e.g. group 1 mean mght be dfferent than group mean but not dfferent from group 3 mean. To olate where the dfference are, you could do a ere of parwe T-tet. The problem wth th that the gnfcance level can be mleadng. For example, f you have 7 group, there wll be 1 parwe comparon of mean; f ung the.05 level of gnfcance, you would expect at leat one tattcally gnfcant dfference even f no dfference ext. Therefore, varou method have been developed for dong multple comparon of group mean. In SPSS, one way to accomplh th va the ue of the /POSTHOC parameter on the Oneway command. We ll preent the SPSS output and then explan what the dfferent part mean. ONEWAY core BY program /STATISTICS DESCRIPTIVES /MISSING ANALYSIS /POSTHOC = BONFERRONI SIDAK SCHEFFE ALPHA(.05. Oneway Decrptve SCORE Total 95% Confdence Interval for Mean N Mean Std. Devaton Std. Error Lower Bound Upper Bound Mnmum Maxmum ANOVA SCORE Between Group Wthn Group Total Sum of Square df Mean Square F Sg Multple/Pot Hoc Group Comparon n Anova - Page 1

2 Pot Hoc Tet Multple Comparon Dependent Varable: SCORE Bonferron Sdak Scheffe (I PROGRAM (J PROGRAM *. The mean dfference gnfcant at the.05 level. Mean Dfference 95% Confdence Interval (I-J Std. Error Sg. Lower Bound Upper Bound 00* * * * * * * * * * * * * * * * * * * * * * * * * * Multple/Pot Hoc Group Comparon n Anova - Page

3 We have een the decrptve tattc and the ANOVA table before, o we wll focu on the Pothoc comparon table. Mean dfference. Th column gve the dfference n the mean of the group. For example, group 1 mean 11.8, group mean 8.8, o the dfference 3. An aterk by the value ndcate whether the dfference tattcally gnfcant gven the method of multple comparon beng ued. (More on method below. Standard error. In a One-way Anova, the tandard error of the dfference between the two mean of group and = 1 1 MSE * + N N (Recall that MSE another name for MS Wthn. In th partcular example, the group ze are all the ame, whch why the reported tandard error are all the ame, but th wll not be true when group ze dffer. In th example, ˆ ˆ * µ = MSE + =.6* + = 1.04 = µ N N 5 5 Sg. Th column gve you the gnfcance of the dfference under the multple comparon method beng ued. To undertand th, we need explan each of the 4 method beng ued and what ther ratonale.. tand for Leat Sgnfcant Dfference t tet. Th tet doe not control the overall probablty of reectng the hypothee that ome par of mean are dfferent, whle n fact they are equal,.e. t doen t matter f you are comparng 1 par of mean or a 100, no adutment made for the number of comparon. The formula ˆ µ ˆ µ = Th tattc ha a T dtrbuton wth N-J d.f. where J = number of group. So, for example, the value for the comparon of group 1 and ˆ µ ˆ µ = = =.94175, d.f = 16 Multple/Pot Hoc Group Comparon n Anova - Page 3

4 The -taled probablty of gettng a t value th large or larger n magntude f the null true only ,.e. there le than 1 chance n a hundred that ther could be no dfference n group mean and the ample would produce a dfference n mean that th large. Alternatvely, you can ut quare the tattc; the reultng value ha an F dtrbuton wth d.f. 1, N-J,.e. ˆ ˆ µ µ = ˆ µ ˆ µ = = 8.654, d.f = 1,16 The name derve from the fact that you determne what the mallet dfference between mean that would be tattcally gnfcant. If the actual dfference greater than that, then you regard the reult a tattcally gnfcant. In th cae, note that f we are dong a twotaled tet ung the.01 level of gnfcance, the crtcal value for a t wth 16 d.f..91. (For an F wth d.f. 1, 16, the crtcal value 8.53 For the.05 level (whch what we told ONEWAY to ue the crtcal value.1, hence there an * by the value of 3 n the mean dfference column. Note that make no adutment for the fact that multple comparon are beng made. In th cae, there are 6 poble parwe comparon; hence the odd that at leat one of them would be gnfcant at the.05 level (even f there are no dfference actually much greater than.05,.e. f you do enough comparon, ut by chance ome wll how up a gnfcant. The remanng method offer dfferent way of adutng the gnfcance level to compenate for th. Bonferron. The Bonferron adutment the mplet. It bacally multple each of the gnfcance level from the tet by the number of tet performed,.e. J*(J-1/. If th value greater than 1, then a gnfcance level of 1 ued. So, for example, the tet report that the dfference between group 1 and gnfcant at the level. The Bonferron adutment multple th by 6 (the number of parwe comparon when there are 4 group and report a gnfcance level of 6 * = Note that th greater than.05, o the dfference between group 1 and not condered gnfcant (hence no * n the mean dfference column. For group 1 veru 3, report that the dfference only gnfcant at the.7 level. Snce 6 *.7 greater than 1, the Bonferron adutment report a gnfcance level of 1. If you compare the gnfcance level of and Bonferron, you ll ee that Bonferron alway 6 tme larger than, or ele 1,.e. Bonferron = Mnmum(6*, 1. An addtonal mplcaton of th that reult have to be gnfcant at the.05/6 = level n order to be gnfcant at the.05 level under Bonferron. Smlarly, f we had 7 group and hence 1 parwe comparon, the tet would have to be gnfcant at the.05/1 =.0038 level to be gnfcant after the Bonferron adutment. Multple/Pot Hoc Group Comparon n Anova - Page 4

5 Sdak. Whle mple, the Bonferron adutment actually overcompenate for the fact that multple comparon are beng made, e.g. f you do 1 tet, the probablty NOT 1.05 that at leat one of them wll be gnfcant at the.05 level; rather, t =.659. The Sdak adutment compute the level of gnfcance a gnfcance So, for example, for the group 1 veru group comparon, the Sdak gnfcance gnfcance = = Th a lttle more gnfcant than what Bonferron came up wth but tll more than.05, o the dfference between group 1 and not condered gnfcant. For group 1 veru 3, gnfcance = =.9997 Scheffe. The Scheffe tet take a omewhat dfferent approach. The Scheffe tet compute an F tattc wth d.f. = J-1, N-J. Scheffe = /(J 1. So, for group 1 veru group, the Scheffe value 8.654/3 = An F value of.8847 wth d.f. = 3, 16 gnfcant at the.068 level. (For an F wth d.f. 3, 16, the tet tattc ha to be 3.01 or larger to be gnfcant at the.05 level. Agan, Scheffe ay the group 1 veru group dfference not gnfcant at the.05 level. Confdence Interval. I won t go nto the detal of how the confdence nterval are computed. But, note that, f 0 fall wthn the confdence nterval, you hould NOT reect the null hypothe that there no dfference n the mean. Other Comment. I m not ure that t make a whole lot of dfference whch of the adutment method you ue. But, nce all 3 of thee wll how up n the lterature, you hould undertand the general dea that thee are method degned to reduce our overall chance of falely reectng each hypothe to α rather than lettng t ncreae wth each addtonal tet. The flp de that the adutment method ncreae the lkelhood we wll tck wth the null when we hould reect t. For example, uppoe there were 7 group and each parwe dfference wa gnfcant at the.04 level. It extremely unlkely that you would get o many gnfcant dfference by chance and your overall F value n your ANOVA would be hghly gnfcant. Nonethele, f you made the Bonferron adutment, each would now be gnfcant at the.84 level and hence none of the dfference would be condered gnfcant. Smlar adutment can be done n other context, e.g. n a correlaton matrx, ome correlaton can be gnfcant ut by chance; o, you ll ometme ee Bonferron or other adutment beng made. I ve never been aked to make any uch adutment n my work! Indeed, t get complcated to do o once you get beyond a one-way Anova framework. But, uch adutment are probably much more common n other feld of tudy. Multple/Pot Hoc Group Comparon n Anova - Page 5