Power Analysis and Sample Size Estimation in Multivariate Analysis James H. Steiger Psychology 312

Transcription

1 Power Anlysis nd Smple Size Estimtion in Multivrite Anlysis Jmes H. Steiger Psychology 31 Textbooks never tret it in detil, nd often do not tret it t ll. (Morrison nd Anderson, the two clssic references, do not even hve n index item for power. ) In prctice, power nd smple size clcultion is linked to n pproch to confidence intervl estimtion tht I cll noncentrlity intervl estimtion. However, most references on power nd smple size clcultion do not discuss the ccompnying intervl estimtion procedures. An exception (besides my own work) is the work of Ken Kelley nd his collegues t Notre Dme. In the pges tht follow, I ll present bsic notes on techniques covered in the course, with the exception of cnonicl correltion. We begin with detiled nlysis of the simplest specil cse, ANOVA nd t-tests, becuse mny of the concepts developed there re employed in the other procedures.

2 Bsic Concepts vi t-test nd 1-Wy ANOVA Stndrdized Effect Size One Smple t E s µ µ 0 = σ -Smple t E s µ 1 µ = σ Generl Distribution One Smple t t λ λ n 1,, = nes -Smple t, nn 1 t n 1+ n, λ λ = E s n1+ n Intervl Estimtion Estimte E s rther thn simply test null hypothesis. See Steiger nd Fouldi (1997) for detils. See the MBESS pckge in R nd its documenttion for routines. Equivlence Testing A simple pproch is to see if the entire 1 α confidence intervl for E s fits within zone of trivility. Precision Plnning versus Power Plnning Choose smple size so tht the stndrd error of E s is sufficiently smll. Alterntively, the AIPE (Accurcy in Prmeter Estimtion) pproch of Ken Kelley is to pln smple size so tht the expected width of the confidence intervl is sufficiently smll.

3 Fctoril ANOVA 1-Wy Stndrdized Effect Size. We seek multi-smple nlogue of severl lterntive mesures. With levels of the A fctor, Cohen s f is E s. There re f j α j µ µ j= 1 σ j= 1 σ = = The distribution of the F sttistic is noncentrl F 1, n ( 1), λ, with noncentrlity prmeter λ given by α j λ = n j= 1 σ = nf = Ntot f One my clculte power nd smple size for given f directly from the noncentrl F distribution in R. This is utomted very nicely in the progrm Gpower 3. This reltionship my lso be turned round to generte confidence intervl for λ, nd ultimtely, for f. This clcultion my be performed using the routines in the R pckge MBESS. -Wy nd Beyond Except for degrees of freedom, little chnges. For min effect or interction θ, the noncentrlity prmeter my be clculted directly s λ = θ Ntot θ f For detiled tretment of confidence intervl estimtion in the context of ANOVA nd regression, consult the book chpter by Steiger nd Fouldi (1997), nd the journl rticle by Steiger (004).

4 Multiple Regression with Fixed Regressors Test tht ρ = 0 With smple size of n, number of predictors k, the F sttistic is F kn, k, λ R / k = ( 1 R ) / ( n k 1) This is distributed s noncentrl F with noncentrlity prmeter λ = n ρ 1 ρ Hence, it is rther strightforwrd to clculte power for given ρ, kn,. In ny given sitution, one my plot power s function of n for given ρ nd k, nd then compute required smple size by inverting the function plot. This is nicely utomted in numerous progrms, including the freewre progrm Gpower 3. This sme pproch cn lso be used to perform power nd smple size nlysis for tests of n dditionl predictor or group of predictors. For detils, see Ful, et l (007), p. 181.

5 Multiple Regression with Rndom Regressors The non-null distribution in the cse of rndom regressors is much more complicted thn for the fixed regressors cse. As consequence, most people relied on fixed regressor clcultions s n pproximtion. In 199, Steiger nd Fouldi produced R, the first progrm tht could clculte the exct distribution of R. The progrm performed full rnge of power nd smple size clcultions for tests tht ρ = c, where c need not be zero. In ddition, the progrm produced n exct confidence intervl on ρ. The power nd smple size clcultions re vilble in Gpower 3 nd MBESS, nd the confidence intervl clcultions cn be performed by MBESS.

6 Structurl Eqution Modeling, Fctor Anlysis, Confirmtory Fctor Anlysis Fitting model by mximum likelihood involves minimizing the mximum likelihood criterion F( SM, ( θ )). The stndrd test sttistic is the chi-squre sttistic ( N 1) F( SM, ( θ )). Steiger, Shpiro, nd Browne (1985) showed tht this sttistic hs pproximtely noncentrl chi-squre distribution with noncentrlity prmeter λ = ( N 1) F * * where F is the popultion discrepncy function, i.e., the vlue of the discrepncy function tht would be obtined if the popultion covrince mtrix Σ were vilble nd nlyzed by mximum likelihood. Steiger nd Lind (1980) proposed the RMSEA s n index of popultion bdness of fit. (See hndout on Fit Indices in SEM t the course website.) This index is RMSEA = * F ν * where ν is the degrees of freedom for the model. Work by Browne (1977) hd shown tht F is closely pproximted by sum of squred orthogonlized model errors, much the sme s the squred Mhlnobis distnce in form, i.e., F * 1 e' Γ where e is vector of discrepncies between the elements of Σ nd the model s pproximtion of them, nd Γ is the covrince mtrix of the elements of S. So the RMSEA is essentilly root-men-squre-error of pproximtion of the model to the dt. This mens tht the noncentrlity prmeter my be clculted s e λ = ( n = 1) ν RMSEA McCllum, Browne, nd Sugwr (1996) suggested forml hypothesis test of trget vlues of the RMSEA. The trditionl test is, of course, test tht the popultion RMSEA = 0. Steiger (1990) fvored confidence intervl bsed pproch, centering on wht McCllum, Browne, nd Sugwr termed test of not-close-fit, corresponding to the stndrd pproch in biosttistics bioequivlence testing. McCllum, Browne, nd Sugwr produced smple size tbles. Steiger (1999) included power nd smple size clcultor in the progrm Sttistic Power Anlysis. Ken Kelley includes routines to perform clcultions on these procedures in the R pckge MBESS.

7 Hotelling s T The Squred Mhlnobis Distnce The Ψ index: Ψ= k 1 = µ µ 1 Σ µ µ 1 ( )' ( ) One Smple 1 = x µ 0 S x µ 0 = T n( )' ( ) n ˆ F k 1, n k 1, λ = n k+ 1 T ( n 1)( k 1) Two Smple λ = n = nkψ T nn nn = ( x1 x)' Σ ( x1 x) = n1+ n n1+ n nn nn λ = = kψ n + n n + n ˆ

8 Binry Logistic Regression GPower 3 gives power clcultion for single predictor, in terms of the null hypothesized probbility of the response = 1 given X = 1. One simply specifies the odds rtio nd smple size, nd the progrm clcultes power. Or, lterntively, one specifies power nd the odds rtio, nd the progrm computes smple size. GPower 3 lso computes power nd smple size for testing n dditionl predictor. One must specify the ρ for the other predictors, which is of course lrgely bsed on guesswork. Theory underlying the tests in GPower3 is given in Hsieh (1989) nd discussed in detil in Hosmer nd Lemeshow, Applied Logistic Regression, p

9 MANOVA Power nd smple size nlysis in MANOVA requires specifiction of numerous prmeters tht you re unlikely to know. It is not for the fint of hert, but n pproch is implemented in Gpower 3 nd discussed in the tutoril, vilble for downlod online. Ful, et l. (007) give thorough discussion of the theory behind the methods employed in GPower3. An lternte, somewht simpler pproch tht Ful et l. clim is slightly less ccurte is given by Muller et l (199). Their pproch is very generl.