Sample size estimation

Transcription

1 Sample size estimation

2 Steps for calculating the sample size per stratum 1. Choose the stratification (e.g. regions, district,.) 2. Define the population (N) of each strata 3. Decide on key indicator(s) 4. Estimate mean & variance or prevalence of key indicator 5. Decide on precision and confidence level 6. Calculate the initial total sample size (n) according to the budget/time 7. Use simple random sample per strata to select your representative sample Final sample size

3 Calculate sample size random sample To estimate sample size, you need to know: Estimate of the prevalence or mean & STDev of the key indicator (e.g. 30% poor food consumption) Precision desired (for example: ± 5%) Level of confidence (for example: 95%) Population (only if below 10,000, otherwise it will not influence the required sample size) Expected response rate (for example: 90%) Number of eligible individuals per household (if applicable) Note: Precision is the variability of the estimate Confidence Level is the probability of the same result if you re-sampled, all other things equal Confidence Interval is the interval around the estimate for which we have a desired confidence level Choose Size of Confidence Interval (Precision) given a confidence level As long as the target population is more than a few thousand households, it will not influence the required sample size. Only if less, the sample size requirements will reduce slightly.

4 Prevalence vs. mean Prevalence is the total number of cases for a variable of interest that is typically binary within a population divided by its total population (e.g. stunting incidence, unemployment) Mean is the expected value of a variable of interest that is typically continuous or within a prescribed range for a given population (e.g. height, weight, age) For the purpose of calculating sample size: Treat variables as prevalence rates only when they are naturally binary DO NOT threshold continuous indicators when calculating sample size even if for analysis purposes they are turned into prevalence indicators e.g. the Food Consumption Score (FCS) is a continuous indicator (0-112) but it is tresholded at 21 and 35 for poor and borderline prevalence

5 Choosing the right distribution For continuous variables we must choose a probability distribution that best fits the data Step 1: ALWAYS plot a histogram of past data and choose the distributional form! Two of the most common ones encountered at WFP are: Normal Distribution Negative Binomial Distribution Prevalence rates always follow a binomial distribution which is why they are mathematically easy to deal with

6 Sample size formula for prevalence (single survey using random sampling) To calculate sample size for estimate of prevalence with 95% confidence limit n = x (P)(1-P) d = Z value for 95% confidence limits P = Estimated prevalence (e.g. 0.3 for 30%) (P)(1-P) = variance for a binary (binomial) variable d = ½ of desired confidence interval (e.g for ± 5%)

7 Sample size formula for continuous variable (single survey using random sampling) To calculate sample size for estimate with 95% confidence limit: n = 1 1 N + μ d 2 (1.2) σ = Z value for 95% confidence limits μ = Expected mean σ 2 = variance of variable d = ½ of desired confidence interval (e.g for ± 5%) μ*d = absolute value of confidence interval N= population of each strata.

8 Sample size formula for cluster sampling To calculate sample size for estimate of prevalence with 95% confidence interval taking into account cluster sampling: N = DEFF x x (P)(1-P) d 2 DEFF = Design effect 1.96 = Z value for p = 0.05 or 95% confidence intervals P = Estimated prevalence d = Desired precision (for example, 0.05 for ± 5%)

9 What is Design effect? Ratio of the actual variance from the sampling method used, to the hypothetical variance under simple random sampling For clustered sampling: N = # of Samples K = # of clusters M = # of samples per cluster D eff > 1 always for cluster sampling because it can never be more efficient than random sampling, vise versa for stratified sampling

10 Design effect Design effect increases when Key indicators are highly geographically clustered (e.g. water source, access to health care) Water source Access to health care When number of clusters are decreased and size of clusters are increased To minimize design effect Include more clusters of smaller size Stratify sample into more homogeneous groups All clusters should be of same size

11 Example1: Key indicator Normally distributed Food Consumption Score

12 Example 1: Iraq Case Study 1. Choose the stratification (Strata): 18 Governorates of Iraq 2. Define Population (N) of each strata: Governorate(s) Population 2015 Anbar Babil Najaf Baghdad Basrah Diyala Duhok Erbil Kerbala Kirkuk Missan Wassit Muthanna Qadissiya Ninewa Salah al-din Sulaymaniyah Thi-Qar Key Indicator: Food Consumption Score (FCS)

13 4. Calculate the Mean (μ) and Standard deviation (STDev) of the FCS (Key Indicator) The STDev was rounded for having just two levels 5. Decide on precision and confidence level: 90% confidence interval Z=1.645 N.B. A confidence interval of 10% at 90% confidence is the absolute minimum! The bigger the CI then the worse the ability to detect trends!

14 6. Initial total sample size (n): 2200 surveyed respondents The required sample size needed in each Governorate calculated using the equation (1.2) will be:

15 Simple Random Sampling After estimating the sample size in each governorate, rescale it to the total sample size of 2200 (n) using the following formula: n n n IRAQ Governorate(s) Population FCS Mean FCS Stdev SRS 5%-90% Anbar Babil Najaf Baghdad Basrah Diyala Duhok Erbil Kerbala Kirkuk Missan Wassit Muthanna Qadissiya Ninewa Salah al-din Sulaymaniyah Thi-Qar Iraq n IRAQ Given 5/90 -> scaling by

16 Example2: Negative binomial distributed indicator reduced Coping Strategy Index

17 Stratified Random Sampling Example 2: Malawi Case Study 1. Define the total Population (N): Malawi Total Define the population in each Strata (Nh): (Strata= 10 aggregated districts of Malawi) District(s) Population Blantyre-Mwanza-Neno-Balaka Chikwawa-Nsanje Chiradzulu-Mulanje-Thyolo-Zomba-Phalombe Dedza-Ntcheu Dowa-Ntchisi-Kasungu-Mchinji Lilongwe Machinga-Mangochi Mzimba-Karonga-Rumphi Nkhata Bay-Chitipa-Likoma Nkhotakota-Salima

18 3. Key Indicator: reduced Coping Strategy Index (rcsi) 4. Calculate the Mean (μ) and Standard deviation (STDev) of the rcsi Stratified Random Sampling District(s) Population rcsi Mean rcsi Stdev Blantyre-Mwanza-Neno-Balaka Chikwawa-Nsanje Chiradzulu-Mulanje-Thyolo-Zomba-Phalombe Dedza-Ntcheu Dowa-Ntchisi-Kasungu-Mchinji Lilongwe Machinga-Mangochi Mzimba-Karonga-Rumphi Nkhata Bay-Chitipa-Likoma Nkhotakota-Salima Malawi Total The STDev was rounded for having just two levels StDev = StDev = µ+µ 2 µ+µ 2 n h,baseline n h,baseline Where nh,baseline = sample size of each strata of the baseline with h=1,,10 Where nh,baseline = sample size of each strata of the baseline with h=1,,10 5. Decide on precision and confidence level: 90% confidence interval Z=1.645 N.B. A confidence interval of 10% at 90% confidence is the absolute minimum! The bigger the CI then the worse the ability to detect trends!

19 Simple Random Sampling 6. Total Sample Size (n): 2000 surveyed respondents The required sample size needed in each Governorate calculated using the equation (1.2) will be: District(s) Population rcsi Mean rcsi Stdev SRS 5%-90% Blantyre-Mwanza-Neno-Balaka Chikwawa-Nsanje Chiradzulu-Mulanje-Thyolo-Zomba-Phalombe Dedza-Ntcheu Dowa-Ntchisi-Kasungu-Mchinji Lilongwe Machinga-Mangochi Mzimba-Karonga-Rumphi Nkhata Bay-Chitipa-Likoma Nkhotakota-Salima

20 Rescaling it to the total sample size of 2000 individuals: Final sample size District(s) Population rcsi Mean rcsi Stdev SRS 5%-90% Given 5/90 -> scaling by 2000 Blantyre-Mwanza-Neno-Balaka Chikwawa-Nsanje Chiradzulu-Mulanje-Thyolo-Zomba-Phalombe Dedza-Ntcheu Dowa-Ntchisi-Kasungu-Mchinji Lilongwe Machinga-Mangochi Mzimba-Karonga-Rumphi Nkhata Bay-Chitipa-Likoma Nkhotakota-Salima Applying Human Judgement Rounding/Appying Human Judgement*

21 Number of Attempts Malawi case Given: The Response rate (25%) Estimated sample size in each strata: Aggregated Districts The desired attempts=estimated sample size/response rate Aggregated Districts Rounding/Appying Human Judgement* Desired Attempts Blantyre-Mwanza-Neno-Balaka Chikwawa-Nsanje Chiradzulu-Mulanje-Thyolo-Zomba-Phalombe Dedza-Ntcheu Dowa-Ntchisi-Kasungu-Mchinji Lilongwe Machinga-Mangochi Mzimba-Karonga-Rumphi Nkhata Bay-Chitipa-Likoma Nkhotakota-Salima Rounding/Appying Human Judgement* Blantyre-Mwanza-Neno-Balaka 240 Chikwawa-Nsanje 210 Chiradzulu-Mulanje-Thyolo-Zomba-Phalombe 180 Dedza-Ntcheu 210 Dowa-Ntchisi-Kasungu-Mchinji 135 Lilongwe 245 Machinga-Mangochi 210 Mzimba-Karonga-Rumphi 180 Nkhata Bay-Chitipa-Likoma 210 Nkhotakota-Salima 180