Table of standard normal deviates for Z ß...31

Transcription

1 Statistical Power Ad Sample Size Calculatios... 1 Whe Do You Need Statistical Power Calculatios, Ad Why?... 1 Preparatio For The Questio What Is Statistical Power?... 1 Statistical Hypothesis Testig... 1 Whe H o Is True Ad You Reject It, You Make A Type I Error.... Whe H o is False Ad You Fail To Reject It, You Make A Type II Error.... The Defiitio Of Statistical Power... Calculatig Statistical Power... 3 How Do We Measure Effect Size?... 3 Cohe's Rules Of Thumb For Effect Size... 3 Calculatig Cohe s d... 4 Calculatig Cohe s d from a t test... 4 Covetios Ad Decisios About Statistical Power... 5 Cosiderig Statistical Power Whe Reviewig Scietific Research... 6 Statistical Power Aalysis I Miitab... 7 Summary: Factors That Ifluece Power... 7 Footote... 8 Usig Miitab To Calculate Power Ad Miimum Sample Size... 8 Example 1: Statistical power of a t-test o scores i groups... 8 Example : Required sample size for a give power: group compariso... 9 Example 3: Example of calculatig power for a oe-way ANOVA...10 Power ad Sample Size Calculatios for Other Desigs ad Tests...11 Sample Size Equatios Backgroud Theory...13 Determiig The Necessary Sample Size For Estimatig A Sigle Populatio Mea Or A Sigle Populatio Total With A Specified Level Of Precisio...14 Table Of Stadard Normal Deviates (Z α ) for Various Cofidece Levels...15 Example:...16 Example:...17 Determiig The Necessary Sample Size For Detectig Differeces Betwee Two Meas With Temporary Samplig Uits Table of stadard ormal deviates for Z α...18 Table of stadard ormal deviates for Z ß...18 Example:...18 Example:...0 Determiig The Necessary Sample Size For Detectig Differeces Betwee Two Meas Whe Usig Paired Or Permaet Samplig Uits....1 Table of stadard ormal deviates for Z α... Table of stadard ormal deviates for Z ß... Example...3 Example...4 Example...5 Determiig The Necessary Sample Size For Estimatig A Sigle Populatio Proportio With A Specified Level Of Precisio....6 Table of stadard ormal deviates (Z α ) for various cofidece levels...7 Example:...7 Example:...8 Determiig The Necessary Sample Size For Detectig Differeces Betwee Two Proportios With Temporary Samplig Uits Table of stadard ormal deviates for Z α...30 Table of stadard ormal deviates for Z ß...31 Example:...31 Example:...33 Caveat...34 Bibliography...34 Sample Size Calculatios i Cliical Research by Shei-Chug Chow, Ju Shao ad Hasheg Wag 34 Sample size estimatio: How may idividuals should be studied? By Eg Joh...35 i

2 Power ad Sample Size Estimatio i Research by Ajeeye Fracis...35 Sample size determiatio by Dell RB, Hollera S, Ramakrisha R...35 Statistical power ad estimatio of the umber of required subjects for a study based o the t-test: A surgeo's primer by Livigsto EH, Cassidy L...36 Sample Size Correctio Table for Sigle Parameter Estimates...37 ii

3 Statistical Power Ad Sample Size Calculatios Whe Do You Need Statistical Power Calculatios, Ad Why? People embarkig o scietific projects i ew fields eed statistical power aalysis i order to desig their studies - particularly to decide o how may cases are eeded. A prospective power aalysis is used before collectig data, to cosider desig sesitivity - that is, the ability to detect what you are lookig for at a time whe you ca do somethig about it. For example, you ca icrease the desig sesitivity by icreasig the sample size, or by takig measures to decrease the error variace, e.g., by cotrollig extraeous variables. Thus, prospective power aalysis is what you use whe you are plaig your ow project. Readers ad reviewers of the scietific literature may eed to draw o retrospective power aalysis, i order to kow whether the studies they are iterpretig were well eough desiged - especially if these report failures to reach statistical sigificace. For example, suppose you read a paper about a study i which the authors had coducted a experimet ad the data aalysis did ot reveal ay statistically sigificat results. You would eed to kow whether that study had ever had a chace of comig up with a sigificat result - e.g., whether the ivestigators gathered a big eough sample. To do this you would eed to estimate whether the study had sufficiet statistical power. You ca use Miitab to perform both prospective ad retrospective power studies. Preparatio For The Questio What Is Statistical Power? The aswer to this questio depeds o your havig a clear uderstadig of the followig techical terms: the ull hypothesis, ( H o ), sigificace level, α, Type I error, Type II error. If you are very usure about these, please refer to your ow statistics otes ad to your usual statistics textbook - but let's briefly review these cocepts as a prelimiary to uderstadig statistical power: Statistical Hypothesis Testig Whe you perform a statistical hypothesis test, there are four possible outcomes. These outcomes deped o: 1

4 whether the ull hypothesis (H o ) is true or false, ad whether you decide either to reject, or else to retai, provisioal belief i H o. These outcomes are summarised i the followig table: Decisio H o is really true i.e., there is really o effect to fid H o is really false i.e., there really is a effect to be foud Retai H o correct decisio: prob = 1 - α Type II error: prob = β Reject H o Type I error: prob = α correct decisio: prob = 1 - β Whe H o Is True Ad You Reject It, You Make A Type I Error. (Traslatio: whe there really is o effect, but the statistical test comes out sigificat by chace, you make a Type I error.) Whe H o is true, the probability of makig a Type I error is called alpha (α). This probability is the sigificace level associated with your statistical test. Whe H o is False Ad You Fail To Reject It, You Make A Type II Error. (Traslatio: Whe, i the populatio, there really is a effect, but your statistical test comes out osigificat, due to iadequate power ad/or bad luck with samplig error, you make a Type II error.) Whe H o is false, (so that there really is a effect there waitig to be foud) the probability of makig a Type II error is called beta (β). The Defiitio Of Statistical Power Statistical power is the probability of ot missig a effect, due to samplig error, whe there really is a effect there to be foud. I techical terms: power is the probability (prob = 1 - β) of correctly rejectig H o whe it really is false.

5 Calculatig Statistical Power Power depeds o: 1. the sample size(s),. the level of statistical sigificace required, ad (here's the tricky bit!) o 3. the miimum size of effect that it is reasoable to expect. How Do We Measure Effect Size? For a compariso of two groups, e.g., a experimetal group with a cotrol group, the measure of effect size will probably be Cohe's d. This hady measure is defied as the differece betwee the meas for the two groups, divided by a estimate of the stadard deviatio i the populatio - ofte we use the average of the stadard deviatios of the samples as a rough guide for the latter. The reaso why the issue of effect size is tricky is that, all too ofte i Psychology, we do't kow how big a effect we should expect. Oe of the good thigs about the recet developmet of power aalysis is that it throws us back to thikig about the appropriate psychological theory, which ought to tell us how big a effect to expect. I Physics, people do't just predict that oe variable will have a statistically sigificat effect o aother variable: they develop theories that predict how big the effect should be. It's good for us to be forced to thik how big the effect should be. But, as Psychology is so much more complex tha Physics, we ofte caot do much more tha guess at expected effect sizes. Ofte we ed up sayig Well, we ca oly afford to test so may subjects, so we will probably oly be able to pick up a effect if it is big. - So, we make some guess at what would be a big effect size. Eve this is still likely to be useful, however, if the study is ot powerful eough to pick up eve a big effect, it is very ulikely to pick up a small oe. Cohe (199) gives useful rules of thumb about what to regard as a big, medium or small effect. Cohe's Rules Of Thumb For Effect Size Effect size Correlatio Differece betwee meas coefficiet Small effect r = 0.1 d = 0. stadard deviatios Medium effect r = 0.3 d = 0.5 stadard deviatios Large effect r = 0.5 d = 0.8 stadard deviatios 3

6 The two commoest effect-size measures are Pearso's r ad Cohe's d. Cohe's tutorial paper gives similar rules of thumb for differeces i proportios, partial correlatios ad ANOVA desigs. Cohe, J., (199). A Power Primer. Psychological Bulleti 11: Cohe, J., (1977). Statistical power aalysis for the behavioural scieces. Sa Diego, CA: Academic Press. Calculatig Cohe s d Notatio x1 x d s Pooled d s x Cohe s d effect size Mea Stadard deviatio Subscript refers to the two coditios beig compared Notatio s Pooled 1s s s Stadard deviatio Sample size Subscript refers to the two coditios beig compared Calculatig Cohe s d from a t test Notatio d t d t Cohe s d effect size t statistic Subscript refers to the two coditios beig compared If the two sample sizes are approximately equal this becomes t d where 1. If stadard errors rather tha stadard deviatios are available the 4

7 Notatio s SE s SE Stadard deviatio Stadard error Number of subjects Covetios Ad Decisios About Statistical Power For statistical sigificace, there is a covetio that we will usually accept a 1 i 0 risk of makig a Type I error. Thus, we usually start thikig of fidigs as beig statistically sigificat if they attai a sigificace level of 0.05 or lower (i.e., risk of 1 i 0). There is a similar rule of thumb for statistical power, but the acceptable risk of makig a Type II error is usually set rather higher. There is a overall implicit cosesus that Type I errors, that lead to false cofirmatios of icorrect predictios, are about four times as dagerous as Type II errors. Type II errors lead to a false disbelief i effects that are actually real, but there seems more chace that these mistakes will be corrected i due course, at less cost tha the results of Type I errors. The outcome of such cosideratios is that the covetioal acceptable risk of a Type II error is ofte set at 1 i 5, i.e., a probability of 0.. The covetioally ucotroversial value for adequate statistical power is therefore set at 1-0. = 0.8. Aother way of expressig this is to say that people ofte regard the miimum acceptable statistical power for a proposed study as beig a 80% chace of a effect that really exists showig up as a sigificat fidig. If ayoe (e.g., a ethical committee) asks you What is the proposed power of your study, you are o fairly safe groud if you ca reply 0.8. But this is just a covetio ad it does deped, just like the settig of sigificace levels, o weighig the relative costs of doig the study, plus the costs of the various forms of gettig the wrog aswers, agaist the beefits of gettig the right aswer. Whe you are decidig o acceptable values for α ad β, for a give study, you eed to cosider the seriousess of each type of error. The more serious the error, the less ofte you will be willig to allow it to occur. Therefore, you should demad smaller probability values for risks of more serious errors. Ideally, you wat to have high power to detect effects that you care about, ad low power for ay small effect that would be meaigless. This will affect decisios about, e.g., the umbers of subjects to test i a psychological experimet. 5

8 Example proposed i Miitab's Help pages: suppose you wat to claim that childre i your school scored higher tha the geeral populatio o a stadardised achievemet test. You eed to decide how much higher tha the geeral populatio your test scores eed to be so you are ot makig claims that are misleadig. If your mea test score is a mere 0.7 poits higher tha the geeral populatio, o a 100 poit test, do you really wat to detect this as a sigificat differece? Probably ot. (But for a ote of cautio, see Rosethal (1991), p outlied below). Cosiderig Statistical Power Whe Reviewig Scietific Research I this situatio, we are usually cosiderig the implicatios of a ull fidig. The researchers report, perhaps, that A did ot correlate with B sigificatly, or that there was o sigificat differece betwee the experimetal group ad the cotrol group. They are ulikely to say this about their mai fidigs, or they would probably ever have got their paper published. But if ay substative argumet is ever beig made o the basis of ot fidig ay sigificat effect (e.g., o evidece that X is dagerous to health, we should certaily be alert to whether or ot the power of the study was ever adequate for H o to be rejected. More ofte, researchers make positive claims o the basis of ull results whe discussig ay checks they may have made cocerig extraeous, or perhaps eve potetially cofoudig, variables - e.g., whether the experimetal group ad the cotrol group showed statistically sigificat differeces i itelligece scores, social class, etc. I decidig whether to accept the fact that there was o sigificat differece betwee groups as ay kid of evidece that they were similar, we eed to thik about whether the compariso had adequate statistical power. 6

9 Statistical Power Aalysis I Miitab Miitab provides power ad sample size calculatios uder its mai STAT meu. It caters for the followig procedures: Stat > Power ad Sample Size > 1-Sample Z 1-Sample t -Sample t 1 Proportio Proportios Oe-Way ANOVA -Level Factorial Desig Plackett-Burma Desig These facilities i Miitab are very easy to use, as should be evidet from the accompayig Miitab examples, Summary: Factors That Ifluece Power The followig 3 sets of factors ifluece power: Sample Size. As sample size icreases, power icreases. Alpha, the probability that you are prepared to accept for makig a Type I error (i.e., the level of sigificace that you are prepared to use, e.g., 0.05.). As α, the probability of a Type I error, icreases, β, the probability of a Type II error, decreases. Sice power is 1 - β, as α icreases, ad the sigificace level gets less striget, so statistical power to fid a real effect power also icreases. Note that if you demad a very striget level of sigificace, you are less likely to get a sigificat result ad your statistical power decreases. I Psychology, uduly high levels of strigecy i the sigificace testig of post-anova comparisos has probably bee a major cause of failure i fidig predicted effects that have really bee there i the populatio (Rosethal, 1991). Rosethal (1991) Psychosomatic Medicie 53:

10 s, the stadard deviatio, which gives a estimate of variability i the populatio. As s icreases, power decreases, because effects get lost i the oise. the real size of the effect that we are lookig for i the populatio.. As the size of the effect i the populatio decreases, power decreases. Footote Note of cautio: Be aware that ot all small effects are meaigless - e.g. i the study of chages i serious risks, most effects are small, but they ca still be very importat. Rosethal (1991) gives a tellig example i a study with very high statistical power (due to its sample comprisig,071 cases) o the effect of aspiri o the icidece of coroary heart disease i America physicias. The correlatio betwee takig aspiri ad ot havig a heart attack came out as (ad that is the value of Pearso's r, ot the sigificace level!) but that was equivalet to a reductio of 4% i the icidece of heart attacks - a weak effect that was far too strog to igore. I fact, the America Medical Associatio closed the trial dow prematurely o the stregth of these iterim results, because it had show that the risk associated with beig i the cotrol group ad ot takig prophylactic aspiri was uacceptably large. The participats i the study were members of the AMA. Usig Miitab To Calculate Power Ad Miimum Sample Size Example 1: Statistical power of a t-test o scores i groups Suppose we have two samples, each with = 13, ad we that propose to use the 0.05 sigificace level, Differece betwee meas is 0.8 stadard deviatios (i.e., Cohe's d = 0.8) 1. Click o the Stat Meu, select Power ad Sample size, ad from that select -sample t. : A dialogue box appears: If you ow click o Help, you wo't really eed to read this documet - but carry o doig so if you wat to fid out ow how easy it is.. Go to the top sectio of the dialogue box, Calculate power from sample size Agaist Sample size: eter 13. Agaist Differece: eter Go to the bottom left sectio of the dialogue box, Sigma, which will hold your estimate of the stadard deviatio i the populatio. Check that it cotais the value

11 4. Click o the Optios butto ad check that the default sigificace level of 0.05 is show. Click the Optios Box OK butto. 5. Now that you have specified sample sizes of 13 ad a effect size of 0.8 stadard deviatios, ad alpha = 0.05, you click o OK. You get the followig output i the Sessio Widow:- MTB > Power; SUBC> TTwo; SUBC> Sample 13; SUBC> Differece 0.8; SUBC> Sigma 1.0. Power ad Sample Size -Sample t Test Testig mea 1 = mea (versus ot =) Calculatig power for mea 1 = mea + differece Alpha = 0.05 Assumed stadard deviatio = 1 Sample Differece Size Power The sample size is for each group. Miitab respods that the power will be If, i the populatio, there really is a differece of 0.8 betwee the members of the two categories that would be sampled i the two groups, the usig sample sizes of 13 each will have a 49.9% chace of gettig a result that will be sigificat at the 0.05 level. The power value of approximately 0.5 is probably uacceptably low. We see from the Miitab output that this is based o a -tailed test. If we are usig power aalysis, we probably kow eough about what we are doig to have a theory that predicts which group should have the higher scores, so perhaps a oe-tailed test is called for. To repeat the aalysis, basig it o a 1-tailed test, we repeat the procedure but after clickig to obtai the Optios dialogue box, we chage the Alterative Hypothesis. Example : Required sample size for a give power: group compariso Suppose we had reaso to expect that, as above, i the populatio, there is a effect waitig to be foud, with a magitude of 0.8 stadard deviatios betwee groups. Suppose we ited doig a oe-tailed test, with sigificace level is As above, pull dow the Miitab Stat Meu, select Power ad Sample size, ad from that select -sample t. 9

12 . Click the radio butto: Calculate sample size for each power. Agaist Power values, eter 0.8. Agaist Differece, eter 0.8. (It is a coicidece that i this example, both values are the same.) 3. Go to the bottom left sectio of the dialogue box, Sigma, which will hold your estimate of the stadard deviatio i the populatio. Check that it cotais the value As before, click the dialogue box Optios butto ad i the Optios dialogue box select the Greater tha Alterative hypothesis radio butto. Click the OK buttos. MTB > Power; SUBC> TTwo; SUBC> Differece 0.8; SUBC> Power 0.8; SUBC> Sigma 1.; SUBC> Alterative 1. Power ad Sample Size -Sample t Test Testig mea 1 = mea (versus >) Calculatig power for mea 1 = mea + differece Alpha = 0.05 Assumed stadard deviatio = 1 Sample Target Differece Size Power Actual Power The sample size is for each group. Miitab s output shows that for a target power of at least 0.8, there must be at least 1 cases i each of the two groups tested. This will give a actual power of Iterestigly, this result does ot tally precisely with the power estimate give i Table of Cohe s 199 paper i Psychological Bulleti. Maybe Cohe was proposig a -tailed test. Calculatig the umber of cases required for a power of 0.8, whe the differece betwee group meas is 0.8 stadard deviatios, if the t-test is to be -tailed, is left as a exercise for the reader. Example 3: Example of calculatig power for a oe-way ANOVA Suppose you are about to udertake a ivestigatio to determie whether or ot 4 treatmets affect the yield of a product usig 5 observatios per treatmet. You kow that the mea of the cotrol group should be aroud 8, ad you would like to fid sigificat differeces of +4. Thus, the maximum differece you are cosiderig is 4 uits. Previous research suggests the populatio σ is

13 1. As above, pull dow the Miitab Stat Meu, select Power ad Sample size, ad from that select Oe-way ANOVA.. I Number of levels, eter I Sample sizes, eter I Values of the maximum differece betwee meas, eter I Stadard deviatio, eter Click the OK butto. MTB > Power; SUBC> OeWay 4; SUBC> Sample 5; SUBC> MaxDifferece 4; SUBC> Sigma Power ad Sample Size Oe-way ANOVA Alpha = 0.05 Assumed stadard deviatio = 1.64 Number of Levels = 4 SS Sample Maximum Meas Size Power Differece The sample size is for each level. To iterpretig the results, if you assig five observatios to each treatmet level, you have a power of 0.83 to detect a differece of 4 uits or more betwee the treatmet meas. Miitab ca also display the power curve of all possible combiatios of maximum differece i mea detected ad the power values for oeway ANOVA with 5 samples per treatmet. The symbol o the curve represets the differece value you specified. Power ad Sample Size Calculatios for Other Desigs ad Tests The methods are very similar for all the optios that Miitab offers, such as differeces betwee proportios ad ANOVA desigs. It is pretty self-evidet what to do oce you kow how to do it for the t-test. The o-lie help that you get by clickig the Help butto i the power calculatio dialogue box is very good. 11

14 Cohe (199) remais a very useful itroductory guide to power ad effect size, i less the 5 pages. It icludes a table with may useful power ad effect size calculatios already doe for the reader 1

15 Sample Size Equatios Backgroud Theory Five differet sample size equatios are preseted i this sectio: Each separate descriptio is desiged to stad-aloe from the others. Each discussio icludes the sample size equatio, a descriptio of each term i the equatio, a table of appropriate coefficiets, ad a worked example. The examples icluded all refer to moitorig with a quadrat-based samplig procedure. The equatios ad calculatios also work with other kids of moitorig data such as measuremets of plat height, umber of flowers, or measures of cover. For the equatios that deal with comparig differet sample meas, all comparisos show are for two-tail tests. If a oe-tail test is desired, double the false-chage (Type I) error rate (α) ad look up the ew doubled α value i the table of coefficiets (e.g., use α = 0.0 istead of α = 0.10 for a oe-tailed test with a false-chage (Type I error rate of α = 0.10). The coefficiets used i all of the equatios are from a stadard ormal distributio (Z α ad Z ß ) istead of the t-distributio (ta ad t ß ). These two distributios are early idetical at large sample sizes but at small sample sizes ( < 30) the Z coefficiets will slightly uderestimate the umber of samples eeded. The correctio procedure described for the first example (usig the sample size correctio table, below) already adjusts the sample size usig the appropriate t-value. For the other equatios, ta ad t ß values ca be obtaied from a t-table ad used i place of the Z α ad Z ß coefficiets that are icluded with the sample size equatios. The appropriate ta coefficiet for the false-chage (Type I) error rate ca be take directly from the α colum of a t-table at the appropriate degrees of freedom (v). For example, for a false-chage error rate of 0.10 use the α = 0.10 colum. The appropriate t ß coefficiet for a specified missed-chage error level ca be looked up by calculatig (1-power) ad lookig up that value i the appropriate α colum. For example, for a power of 0.90, the calculatios for t ß would be (1-.90) = 0.0. Use the α = 0.0 colum at the appropriate degrees of freedom (v) to obtai the appropriate t-value. 13

16 Determiig The Necessary Sample Size For Estimatig A Sigle Populatio Mea Or A Sigle Populatio Total With A Specified Level Of Precisio. Estimatig a sample mea vs. total populatio size. The sample size eeded to estimate cofidece itervals that are withi a give percetage of the estimated total populatio size is the same as the sample size eeded to estimate cofidece itervals that are withi that percetage of the estimated mea value. The istructios below assume you are workig with a sample mea. Determiig sample size for a sigle populatio mea or a sigle populatio total is a two or three-step process. (1) The first step is to use the equatio provided below to calculate a ucorrected sample size estimate. () The secod step is to cosult the Sample Size Correctio Table appearig below these istructios to come up with the corrected sample size estimate. The use of the correctio table is ecessary because the equatio below uder-estimates the umber of samples that will be eeded to meet the specified level of precisio. The use of the table to correct the uderestimated sample size is simpler tha usig a more complex equatio that does ot require correctio. (3) The third step is to multiply the corrected sample size estimate by the fiite populatio correctio factor if more tha 5% of the populatio area is beig sampled. (1) Calculate a iitial sample size usig the followig equatio: Z B s Where: 14

17 Z α s B The ucorrected sample size estimate. The stadard ormal coefficiet from the table below. The stadard deviatio. The desired precisio level expressed as half of the maximum acceptable cofidece iterval width. This eeds to be specified i absolute terms rather tha as a percetage. For example, if you wated your cofidece iterval width to be withi 30% of your sample mea ad your sample mea = 10 plats/quadrat the B = 0.30 x 10 = 3.0. Table Of Stadard Normal Deviates (Z α ) for Various Cofidece Levels Cofidece level Alpha (α) level Z α 80% % % % () To obtai the adjusted sample size estimate, cosult the correctio table of these istructios. is the ucorrected sample size value from the sample size equatio. is the corrected sample size value. (3) Additioal correctio for samplig fiite populatios. The above formula assumes that the populatio is very large compared to the proportio of the populatio that is sampled. If you are samplig more tha 5% of the whole populatio the you should apply a correctio to the sample size estimate that icorporates the fiite populatio correctio factor (FPC). This will reduce the sample size. The formula for correctig the sample size estimate with the FPC for cofidece itervals is: 1 N Where: ' The ew FPC-corrected sample size. The corrected sample size from the sample size correctio table. N The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of oe quadrat. 15

18 Example: Maagemet objective: Restore the populatio of species Y i populatio Z to a desity of at least 30 plats/quadrat by the year 001 Samplig objective: Obtai estimates of the mea desity ad populatio size of 95% cofidece itervals withi 0% of the estimated true value. Results of pilot samplig: Mea ( x ) = 5 plats/quadrat. Stadard deviatio (s) = 7 plats. Give: The desired cofidece level is 95% so the appropriate Z α from the table above is The desired cofidece iterval width is 0% (0.0) of the estimated true value. Sice the estimated true value is 5 plats/quadrat, the desired cofidece iterval (B) is 5 x 0.0 = 5 plats/quadrat. Calculate a uadjusted estimate of the sample size eeded by usig the sample size formula: Z s B Roud 7.53 plots up to 8 plots for the uadjusted sample size. To adjust this prelimiary estimate, go to the sample size correctio table ad fid = 8 ad the correspodig value i the 95% cofidece level portio of the table. For = 8, the correspodig value is = 15. The corrected estimated sample size eeded to be 95% cofidet that the estimate of the populatio mea is withi 0% (±5 plats) of the true mea is 15 quadrats. Additioal correctio for samplig fiite populatios: The above formula assumes that the populatio is very large compared to the proportio of the populatio that is sampled. If you are samplig more tha 5% of the whole populatio area the you should apply a correctio to the sample size estimate that icorporates the fiite populatio correctio factor (FPC). This will reduce the sample size. The formula for correctig the sample size is as follows: 16

19 The formula for correctig the sample size estimate with the FPC for cofidece itervals is: 1 N Where: ' The ew FPC-corrected sample size. The corrected sample size from the sample size correctio table. N The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of oe quadrat. Example: If the pilot data described above was gathered usig a 1m x 10m (10 m ) quadrat ad the total populatio beig sampled was located withi a 0m x 50m macroplot (1000 m ) the N = 1000m /10m = 100. The corrected sample size would the be: 1 N The ew, FPC-corrected, estimated sample size to be 95% cofidet that the estimate of the populatio mea is withi 0% (±5 plats) of the true mea is 13 quadrats. Determiig The Necessary Sample Size For Detectig Differeces Betwee Two Meas With Temporary Samplig Uits. s Z MDC Z Where: 17

20 s Z α Z ß MDC The ucorrected sample size estimate. sample stadard deviatio. Z-coefficiet for the false-chage (Type I) error rate from the table below. Z-coefficiet for the missed-chage (Type II) error rate from the table below. Miimum detectable chage size. This eeds to be specified i absolute terms rather tha as a percetage. For example, if you wated to detect a 0% chage i the sample mea from oe year to the ext ad your first year sample mea is 10 plats/quadrat the MDC is 0.0 x 10 = plats/quadrat. Table of stadard ormal deviates for Z α False-chage (Type I) error rate (α) Z α Table of stadard ormal deviates for Z ß Missed-chage (Type II) error rate (ß) Power Z ß Example: Maagemet objective: Icrease the desity of species F at Site Y by 0% betwee 1999 ad 004. Samplig objective: I wat to be 90% certai of detectig a 0% i mea plat desity ad I am willig to accept a 10% chace that I will make a false-chage error (coclude that a chage took place whe it really did ot). Results from pilot samplig: 18

21 Mea (x) = 5 plats/quadrat Stadard deviatio (s) = 7 plats. Give: The acceptable False-chage error rate (α) is 0.10 so the appropriate Z α from the table is The desired Power is 90% (0.90) so the Missed-chage error rate (ß) is 0.10 ad the appropriate Z ß, coefficiet from the table is 1.8. The Miimum Detectable Chage (MDC) is 0% of the 1993 value or.0 x 5 = 5 plats/quadrat. Calculate the estimated ecessary sample size usig the equatio provided above: s Z Z MDC Roud up 33.4 to 34 plots. Fial estimated sample size eeded to be 90% cofidet of detectig a chage of 5 plats betwee 1993 ad 1994 with a false-chage error rate of 0.10 is 34 quadrats. The sample size correctio table is ot eeded for estimatig sample sizes for detectig differeces betwee two populatio meas. Correctio for samplig fiite populatios: The above formula assumes that the populatio is very large compared to the proportio of the populatio that is sampled. If you are samplig more tha 5% of the whole populatio area the you should apply a correctio to the sample size estimate that icorporates the fiite populatio correctio factor (FPC). This will reduce the sample size. The formula for correctig the sample size estimate is as follows: The formula for correctig the sample size estimate with the FPC for cofidece itervals is: 1 N Where: 19

22 ' The ew sample size based upo iclusio of the fiite populatio correctio factor The corrected sample size from the sample size correctio table. N The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of the samplig uit. Example: If the pilot data described above was gathered usig a 1m x 10m (10 m ) quadrat ad the total populatio beig sampled was located withi a 0m x 50m macroplot (1000 m ) the N = 1000m /10m = 100. The corrected sample size would the be: 1 N Roud up 5.37 to 6. The ew, FPC-corrected estimated sample size eeded to be 90% certai of detectig a chage of 5 plats betwee 1993 ad 1994 with a false-chage error rate of 0.10 is 6 quadrats. Note o the statistical aalysis for two sample tests from fiite populatios. If you have sampled more tha 5% of a etire populatio the you should also apply the fiite populatio correctio factor to the results of the statistical test. This procedure ivolves dividig the test statistic by the square root of the fiite populatio factor (1-/N). For example, if your t-statistic from a particular test tured out to be ad you sampled = 6 quadrats out of a total N = 100 possible quadrats, the your correctio procedure would look like the followig: t t 1 N Where: 0

23 t t' N The t-statistic from a t-test. The corrected t-statistic usig the FPC. The sample size from the equatio above. The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of each idividual samplig uit. You would eed to look up the p-value of t' = 1.91 i a t-table at the appropriate degrees of freedom to obtai the correct p-value for this statistical test. Determiig The Necessary Sample Size For Detectig Differeces Betwee Two Meas Whe Usig Paired Or Permaet Samplig Uits. Whe paired samplig uits are beig compared or whe data from permaet quadrats are beig compared betwee two time periods, the sample size determiatio requires a differet procedure tha if samples are idepedet of oe aother. The equatio for determiig the umber of samples ecessary to detect some true differece betwee two sample meas is: s Z MDC Z Where: s Z α Z ß MDC The ucorrected sample size estimate. sample stadard deviatio. Z-coefficiet for the false-chage (Type I) error rate from the table below. Z-coefficiet for the missed-chage (Type II) error rate from the table below. Miimum detectable chage size. This eeds to be specified i absolute terms rather tha as a percetage. For example, if you wated to detect a 0% chage i the sample mea from oe year to the ext ad your first year sample mea is 10 plats/quadrat the MDC is 0.0 x 10 = plats/quadrat. 1

24 Table of stadard ormal deviates for Z α False-chage (Type I) error rate (α) Z α Table of stadard ormal deviates for Z ß Missed-chage (Type II) error rate (ß) Power Z ß If the objective is to track chages over time with permaet samplig uits ad oly a sigle year of data is available, the you will ot have a stadard deviatio of differeces betwee the paired samples. If you have a estimate of the likely degree of correlatio betwee the two years of data, ad you assume that the amog samplig uits stadard deviatio is goig to be the same i the secod time period, the you ca use the equatio below to estimate the stadard deviatio of differeces. s diff s 1 1 corr diff Where: s diff s 1 corr diff Estimated stadard deviatio of the differeces betwee paired samples. Sample stadard deviatio amog samplig uits at the first time period. Correlatio coefficiet betwee samplig uit values i the first time period ad samplig uit values i the secod time period.

25 Example Maagemet Objective: Achieve at least a 0% higher desity of species F at Site Y i ubured areas compared to bured areas i Samplig objective: I wat to be able to detect a 90% differece i mea plat desity i ubured areas ad adjacet bured areas. I wat to be 90% certai of detectig that differece, if it occurs, ad I am willig to accept a 10% chace of detectig that differece, if it occurs, ad I am willig to accept a 10% chage that I will make a false-chage error (coclude that a differece exists whe it really did ot). Results from pilot samplig: Five paired quadrats were sampled where oe member of the pair was excluded from burig ad the other member of the pair was bured. umber of plats/quadrat Quadrat umber bured ubured Differece betwee bured ad ubured x =4.0 s =1.9 x =7.80 s =3.7 x =3.60 s =1.67 MTB > set c1 DATA> DATA> set c DATA> DATA> ame c1 'bured' MTB > ame c 'ubured' MTB > let c3 = 'ubured' - 'bured' MTB > ame c3 'differece' MTB > Describe 'bured' 'ubured' 'differece'; SUBC> Mea; SUBC> StDeviatio. Descriptive Statistics: bured, ubured, differece Variable Mea StDev bured ubured differece Give: The samplig objective specified a desired miimum detectable differece (i.e., equivalet to the MDC) of 0%. Takig the larger of the two mea values ad multiplyig by 0% leads to: 7.80 x 0.0 = MDC = 1.56 plats quadrat. 3

26 The appropriate stadard deviatio to use is 1.67, the stadard deviatio of the differeces betwee the pairs. The acceptable False-chage error rate (α) is 0.10 so the appropriate Z α from the table is The desired Power is 90% (0.90) so the Missed-chage error rate (ß) is 0.10 ad the appropriate Z ß coefficiet from the table is 1.8. Calculate the estimated ecessary sample size usig the equatio provided above: s Z Z MDC Roud up 9.77 to 10 plots. Fial estimated sample size eeded to be 90% certai of detectig a true differece of 1.56 plats/quadrat betwee the bured ad ubured quadrats with a false-chage error rate of 0.10 is 10 quadrats. Example Maagemet objective: Icrease the desity of species F at Site Q by 0% betwee 1999 ad 00. Samplig objective: I wat to be able to detect a 0% differece i mea plat desity of species F at Site Q betwee 1999 ad 001. I wat to be 90% certai of detectig that chage, if it occurs, ad I am willig to accept a 10% chace that I will make a false-chage error (coclude that a differece exists whe it really did ot). The procedure for determiig the ecessary sample size for this example would be very similar to the previous example. Just replace bured ad ubured i the data table with 1999 ad 00 ad the rest of the calculatios would be the same. Because the sample size determiatio procedure eeds the stadard deviatio of the differece betwee two samples, you will ot have the ecessary stadard deviatio term to plug ito the equatio util you have two years of data. The stadard deviatio of the differece ca be estimated i the first year if some estimate of the correlatio coefficiet betwee samplig uit values i the first time period ad the samplig uit values i the secod time period is available (see the s diff equatio above). 4

27 Correctio for samplig fiite populatios: The above formula assumes that the populatio is very large compared to the proportio of the populatio that is sampled. If you are samplig more tha 5% of the whole populatio area the you should apply a correctio to the sample size estimate that icorporates the fiite populatio correctio factor (FPC). This will reduce the sample size. The formula for correctig the sample size estimate is as follows: 1 N Where: ' The ew sample size based upo iclusio of the fiite populatio correctio factor. The corrected sample size from the sample size correctio table. N The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of the samplig uit. Example If the pilot data described above was gathered usig a 1m x 10m (10 m ) quadrat ad the total populatio beig sampled was located withi a 10m x 50m macroplot (500 m ) the N = 500m /10m = 50. The corrected sample size would the be: 1 N Roud up 8.33 to 9. The ew, FPC-corrected estimated sample size eeded to be 90% cofidet of detectig a true differece of 1.56 plats/quadrat betwee the bured ad ubured quadrats with a false-chage error rate of 0.10 is 9 quadrats. Note o the statistical aalysis for two sample tests from fiite populatios. If you have sampled more tha 5% of a etire populatio the you should also apply the fiite populatio correctio factor to the results of the statistical test. This procedure ivolves dividig the test statistic by the square root of (1-/N). For 5

28 example, if your t-statistic from a particular test tured out to be 1.78 ad you sampled = 9 quadrats out of a total N = 50 possible quadrats, the your correctio procedure would look like the followig: t t 1 N Where: t t' N The t-statistic from a t-test. The corrected t-statistic usig the FPC. The sample size from the equatio above. The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of each idividual samplig uit. You would eed to look up the p-value of t' = i a t-table at the appropriate degrees of freedom to obtai the correct p-value for this statistical test. Determiig The Necessary Sample Size For Estimatig A Sigle Populatio Proportio With A Specified Level Of Precisio. Determiig the ecessary sample size for estimatig a sigle populatio proportio with a specified level of precisio. The equatio for determiig the sample size for estimatig a sigle proportio is: pqz d Where: 6

29 Estimated ecessary sample size. Z α The coefficiet from the table of stadard ormal deviates below. p The value of the proportio as a decimal percet (e.g., 0.45). q 1-p d The desired precisio level expressed as half of the maximum acceptable cofidece iterval width. This is also expressed as a decimal percet (e.g., 0.15) ad this represets a absolute rather tha a relative value. For example, if your proportio value is 30% ad you wat a precisio level of ± 10% this meas you are targetig a iterval width from 0% to 40%. Use 0.10 for the d-value ad ot 0.30 x 0.10 = Table of stadard ormal deviates (Z α ) for various cofidece levels Cofidece level Alpha (α) level (Z α ) 80% % % % Example: Maagemet objective: Maitai at least a 40% frequecy (i 1m quadrats) of species Y i populatio Z over the ext 5 years. Samplig objective: Estimate percet frequecy with 95% cofidece itervals o wider tha ± 10% of the estimated true value. Results of pilot samplig: The proportio of quadrats with species Z is estimated to be p = 65% (0.65). Because q = (1-p), q = = Give: The desired cofidece level is 95% so the appropriate Z æ from the table above is The desired cofidece iterval width (d) is specified as 10% (0.10). 7

30 Usig the equatio provided above: pqz d Roud up to 88. The estimated sample size eeded to be 95% cofidet that the estimate of the populatio percet frequecy is withi 10% (±0.10) of the true percet frequecy is 88 quadrats. This sample size formula works well as log as the proportio is more tha 0.0 ad less tha If you suspect the populatio proportio is less tha 0.0 or greater tha 0.80, use.0 or 0.8, respectively, as a coservative estimate of the proportio. Correctio for samplig fiite populatios: The above formula assumes that the populatio is very large compared to the proportio of the populatio that is sampled. If you are samplig more tha 5% of the whole populatio area the you should apply a correctio to the sample size estimate that icorporates the fiite populatio correctio factor (FPC). This will reduce the sample size. The formula for correctig the sample size estimate is as follows: 1 N Where: ' The ew sample size based upo iclusio of the fiite populatio correctio factor. The corrected sample size from the sample size correctio table. N The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of the samplig uit. Example: If the pilot data described above was gathered usig a 1m x 1m (1 m ) quadrat ad the total populatio beig sampled was located withi a 5m x 5m macroplot (65 m ) the N = 65m /1m = 65. The corrected sample size would the be: 8

31 1 N Roud up to 78. The ew, FPC-corrected, estimated sample size eeded to be 95% cofidet that the estimate of the populatio percet frequecy is withi 10% (± 0.10) of the true percet frequecy is 78 quadrats. Alterately icludig a term for Type II errors gives p(1 p)( z / z ). d Where: Estimated ecessary sample size. z α/ The coefficiet of the stadard ormal deviates, covetioally α =.05. The probability of a Type I error. z β The coefficiet of the stadard ormal deviates, covetioally β =.. The probability of a Type II error, meaig 1 β is power. p The value of the proportio as a decimal percet (e.g. 0.45). d The desired precisio level expressed as half of the maximum acceptable cofidece iterval width. This is also expressed as a decimal percet (e.g. 0.15) ad this represets a absolute rather tha a relative value. For example, if your proportio value is 30% ad you wat a precisio level of ±10% this meas you are targetig a iterval width from 0% to 40%. Use 0.10 for the d-value ad ot 0.30 x 0.10 = For example suppose that the respose rate of a patiet populatio uder study after treatmet is expected to be aroud 50% (i.e. p = 0.50). At α = 0.05, the required sample size for havig a 80% power (i.e. β = 0.) for correctly detectig a differece betwee the post-treatmet respose rate ad the referece value of 30% (i.e. d = 0.) is p(1 p)( z d / z ) 0.5(1 0.5)( ) A cross check via Miitab MTB > Power; 9

32 SUBC> POe; SUBC> PCompare.7; SUBC> Power.8; SUBC> PNull.5; SUBC> GPCurve. Power ad Sample Size Test for Oe Proportio Testig p = 0.5 (versus ot = 0.5) Alpha = 0.05 Sample Target Compariso p Size Power Actual Power Determiig The Necessary Sample Size For Detectig Differeces Betwee Two Proportios With Temporary Samplig Uits. Z Z p q p q p p Where: Estimated ecessary sample size. Z α Z ß Z-coefficiet for the false-chage (Type I) error rate from the table below. Z-coefficiet for the missed-chage (Type II) error rate from the table below. p 1 The value of the proportio for the first sample as a decimal (e.g., 0.65). q p 1. p The value of the proportio for the secod sample as a decimal (e.g., 0.45). q 1 - p. Table of stadard ormal deviates for Z α False-chage (Type I) error rate (α) Z α

33 Table of stadard ormal deviates for Z ß Missed-chage (Type II) error rate (ß) Power Z ß Example: Maagemet objective: Decrease the frequecy of ivasive weed F at Site G by 0% betwee 1999 ad 001. Samplig objective: I wat to be 90% certai of detectig a absolute chage of 0% frequecy ad I am willig to accept a 10% chace that I will make a false-chage error (coclude that a chage took place whe it really did ot). Note that the magitude of chage for detectig chage over time for proportio data is expressed i absolute terms rather tha i relative terms (relative terms where used i earlier examples that dealt with sample meas values). The reaso absolute terms are used istead of relative terms relates to the type of data beig gathered (percet frequecy is already expressed as a relative measure). Thik of takig your populatio area ad dividig it ito a grid where the size of each grid cell equals your quadrat size. Whe you estimate a percet frequecy, you are estimatig the proportio of these grid cells occupied by a particular species. If 45% of all the grid cells i the populatio are occupied by a particular species the you hope that your sample values will be close to 45%. If over time the populatio chages so that ow 65% of all the grid cells are occupied, the the true percet frequecy has chaged from 45% to 65%, represetig a 0% absolute chage. Results from pilot samplig: The proportio of quadrats with species Z i 1999 is estimated to be p 1 = 65% (0.65). Because q 1 = 1-p 1, q 1 = =

34 Because we are iterested i detectig a 0% shift i percet frequecy, we will assig p = This represets a shift of 0% frequecy from 1999 to 001. A declie was selected istead of a icrease (e.g., from 65% frequecy to 85% frequecy) because sample size requiremets are higher at the mid-rage of frequecy values (i.e., closer to 50%) tha they are closer to 0 or 100. Stickig closer to the mid-rage gives us a more coservative sample size estimate. Because q 1 = 1-q, q 1 = = Give: The acceptable False-chage error rate (α) is 0.10 so the appropriate Z α from the table is The desired Power is 90% (0.90) so the Missed-chage error rate (p) is ad the appropriate Z ß coefficiet from the table is 1.8. Usig the equatio provided above: Z Z p1q1 pq p p Roud up to 10. The estimated sample size eeded to be 90% sure of detectig a shift of 0% frequecy with a startig frequecy of 65% ad a false-chage error rate of 0.10 is 10 quadrats. Correctio for samplig fiite populatios: The above formula assumes that the populatio is very large compared to the proportio of the populatio that is sampled. If you are samplig more tha 5% of the whole populatio area the you should apply a correctio to the sample size estimate that icorporates the fiite populatio correctio factor (FPC). This will reduce the sample size. The formula for correctig the sample size estimate is as follows: 1 N Where: ' The ew sample size based upo iclusio of the fiite populatio correctio factor. The corrected sample size from the sample size correctio table. 3

35 N The total umber of possible quadrat locatios i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of the samplig uit. Example: If the pilot data described above was gathered usig a 1m x 1m (1m ) quadrat ad the total populatio beig sampled was located withi a 10m x 30m macroplot (300 m ) the N = 300m /1m = 300. The corrected sample size would the be: 1 N Roud up to 77. The ew, FPC-corrected estimated sample size eeded to be 90% sure of detectig a absolute shift of 0% frequecy with a startig frequecy of 65% ad a false-chage error rate o quadrats. Note o the statistical aalysis for two sample tests from fiite populatios. If you have sampled more tha 50% of a etire populatio the you should also apply the fiite populatio correctio factor to the results of the statistical test. For proportioig data this procedure ivolves dividig the test statistic by ( 1 N ). For example, if your χ -statistic from a particular test tured out to be.706 ad you sampled -77 quadrats out of a total N = 300 possible quadrats, the your correctio procedure would look like the followig: 1 Where: χ N The χ - statistic from a χ - statistic -test. The corrected χ - statistic usig the FPC. The sample size from the equatio above. N The total umber of possible quadrat locatio i the populatio. To calculate N, determie the total area of the populatio ad divide by the size of each idividual samplig uit. You would eed to look up the p-value of χ = i a χ table for the appropriate degrees of freedom to obtai the corrected p-value for this statistical test. 33