Demographic and Health Surveys Methodology

Transcription

1 samplng and household lstng manual Demographc and Health Surveys Methodology Ths document s part of the Demographc and Health Survey s DHS Toolkt of methodology for the MEASURE DHS Phase III project, mplemented from Ths publcaton was produced for revew by the Unted States Agency for Internatonal Development (USAID). It was prepared by MEASURE DHS/ICF Internatonal.

2 [THIS PAGE IS INTENTIONALLY BLANK]

3 Demographc and Health Survey Samplng and Household Lstng Manual ICF Internatonal Calverton, Maryland USA September 2012

4 MEASURE DHS s a fve-year project to assst nsttutons n collectng and analyzng data needed to plan, montor, and evaluate populaton, health, and nutrton programs. MEASURE DHS s funded by the U.S. Agency for Internatonal Development (USAID). The project s mplemented by ICF Internatonal n Calverton, Maryland, n partnershp wth the Johns Hopkns Bloomberg School of Publc Health/Center for Communcaton Programs, the Program for Approprate Technology n Health (PATH), Futures Insttute, Camrs Internatonal, and Blue Raster. The man objectves of the MEASURE DHS program are to: 1) provde mproved nformaton through approprate data collecton, analyss, and evaluaton; 2) mprove coordnaton and partnershps n data collecton at the nternatonal and country levels; 3) ncrease host-country nsttutonalzaton of data collecton capacty; 4) mprove data collecton and analyss tools and methodologes; and 5) mprove the dssemnaton and utlzaton of data. For nformaton about the Demographc and Health Surveys (DHS) program, wrte to DHS, ICF Internatonal, Beltsvlle Drve, Sute 300, Calverton, MD 20705, U.S.A. (Telephone: ; fax: ; e-mal: nfo@measuredhs.com; Internet: Recommended ctaton: ICF Internatonal Demographc and Health Survey Samplng and Household Lstng Manual. MEASURE DHS, Calverton, Maryland, U.S.A.: ICF Internatonal

5 TABLE OF CONTENTS TABLES AND FIGURES... v 1 DEMOGRAPHIC AND HEALTH SURVEYS SAMPLING POLICY General prncples Exstng samplng frame Full coverage Probablty samplng Sutable sample sze Smple desgn Household lstng and pre-selecton of households Good sample documentaton Confdentalty Exactness of survey mplementaton Survey objectves and target populaton Survey doman Samplng frame Conventonal samplng frame Alternatve samplng frames Evaluaton of the samplng frame Stratfcaton Sample sze Sample sze and samplng errors Sample sze determnaton Sample allocaton Two-stage cluster samplng procedure Sample take per cluster Optmum sample take Varable sample take for self-weghtng Household lstng Household selecton n the central offce Household ntervews Samplng weght calculaton Why we need to weght the survey data Desgn weghts and samplng weghts How to calculate the desgn weghts... 23

6 Correcton of unt non-response and calculaton of samplng weghts Normalzaton of samplng weghts Standard weghts for HIV testng De-normalzaton of standard weghts for pooled data Calbraton of samplng weghts n case of bas Data qualty and samplng error reportng Sample documentaton Confdentalty HOUSEHOLD LISTING OPERATION Introducton Defnton of terms Responsbltes of the lstng staff Locatng the cluster Preparng locaton and sketch maps Collectng a GPS waypont for each cluster Lstng of households Segmentaton of large clusters Qualty control Prepare the household lstng forms for household selecton Appendx 2.1 Example lstng forms Appendx 2.2 Symbols for mappng and lstng Appendx 2.3 Examples of completed mappng and lstng forms SELECTED SAMPLING TECHNIQUES Smple random samplng Equal probablty systematc samplng Samplng theory Excel templates for systematc samplng Probablty proportonal to sze samplng Samplng theory Operatonal descrpton and examples Complex samplng procedures SURVEY ERRORS Errors of coverage and non-response Coverage errors Delberate restrctons of coverage Non-response v

7 4.1.4 Response rates Samplng errors SAMPLE DOCUMENTATION Introducton Sample desgn document Introducton Samplng frame Structure of the sample and the samplng procedure Selecton probablty and samplng weght Sample fle Results of Survey mplementaton Samplng errors Samplng parameters n DHS data fles Glossary of terms References v

8

9 TABLES AND FIGURES Table 1.1 Table 1.2 Sample sze determnaton for estmatng current use of a modern contraceptve method among currently marred women Sample sze determnaton for estmatng the prevalence of full vaccnaton coverage among chldren aged months Table 1.3 Sample allocaton: Proportonal allocaton Table 1.4 Sample allocaton: Power allocaton Table 1.5 Optmal sample take for currently marred women currently usng any contraceptve method based on ntracluster correlaton ρ and survey cost rato c 1 / c2 from past surveys Table 5.1 Dstrbuton of EAs and average sze of EA by regon and by type of resdence Table 5.2 Dstrbuton of households by regon and by type of resdence Table 5.3 Sample allocaton of clusters and households by regon and by type of resdence Table 5.4 Expected number of ntervews by regon and by type of resdence Table 5.5 An example sample fle Table 5.6 Example table for the results of survey mplementaton Table 5.7 Example appendx table for the results of the women s survey mplementaton Table 5.8 Example appendx table for the results of the men s survey mplementaton Table 5.9 Example table for samplng errors Fgure 3.1 Smple household selecton wth a sub-sample Fgure 3.2 Selecton of runs wth a sub-sample Fgure 3.3 Smple self-weghtng selecton wthout sample sze control Fgure 3.4 Self-weghtng selecton wth runs and wthout sample sze control Fgure 3.5 Self-weghtng selecton wth sample sze control Fgure 3.6 Self-weghtng selecton wth runs and wth sample sze control Fgure 3.7 Manual household selecton n the feld Fgure 3.8 Part of an Excel template for stratfed samplng Fgure 3.9 Part of an example for a provnce crossed urban-rural stratfed PPS samplng Fgure 3.10 Part of an example sample fle from a stratfed PPS samplng v

10

11 1 DEMOGRAPHIC AND HEALTH SURVEYS SAMPLING POLICY 1.1 General prncples Scentfc sample surveys are cost-effcent and relable ways to collect populaton-level nformaton such as socal, demographc and health data. The MEASURE DHS project s a worldwde project mplemented across varous countres and at multple ponts n tme wthn a country. In order to acheve comparablty, consstency and the best qualty n survey results, samplng actvtes n the Demographc and Health Surveys (DHS) should be guded by a number of general prncples. Ths manual presents general gudelnes on samplng for DHS surveys, although modfcatons may be requred for country-specfc stuatons. The key prncples of DHS samplng nclude: Use of an exstng samplng frame Full coverage of the target populaton Probablty samplng Usng a sutable sample sze Usng the most smple desgn possble Conductng a household lstng and pre-selecton of households Provdng good sample documentaton Mantanng confdentalty of ndvdual s nformaton Implementng the sample exactly as desgned Exstng samplng frame A probablty sample can only be drawn from an exstng samplng frame whch s a complete lst of statstcal unts coverng the target populaton. Snce the constructon of a new samplng frame s lkely to be too expensve, DHS surveys should use an adequate pre-exstng samplng frame whch s offcally recognzed. Ths s possble for most of the countres where there has been a populaton census n recent years. Census frames are generally the best avalable samplng frame n terms of coverage, cartographc materals and organzaton. However, an evaluaton of the qualty and the accessblty of the frame should be consdered durng the development of the survey desgn, and a detaled study of the samplng frame s necessary before drawng the sample. In the absence of a census frame, a DHS survey can use an alternatve samplng frame, such as a complete lst of vllages or communtes n the country wth all necessary dentfcaton nformaton ncludng a measure of populaton sze (e.g. number of households), or a master sample whch s large enough to support the DHS desgn Full coverage A DHS survey should cover 100 percent of the target populaton n the country. The target populaton for the DHS survey s all women age and chldren under fve years of age lvng n resdental households. Most surveys also nclude all men age The target populaton may vary from country to country or from survey to survey, but the general samplng prncples are the same. In some cases, excluson of some areas may be necessary because of extreme naccessblty, volence or nstablty, but these ssues need to be consdered at the very begnnng of the survey, before the sample s drawn. 1 The age range vares from survey to survey and may be 15-49, 15-54, or

12 1.1.3 Probablty samplng A scentfc probablty samplng methodology must be used n DHS surveys. A probablty sample s defned as one n whch the unts are selected randomly wth known and nonzero probabltes. Ths s the only way to obtan unbased estmaton and to be able to evaluate the samplng errors. The term probablty samplng excludes purposve samplng, quota samplng, and other uncontrolled non-probablty methods because they cannot provde evaluaton of precson and/or confdence of survey fndngs Sutable sample sze Sample sze s a key parameter for DHS surveys because t s drectly related to survey budget, data qualty and survey precson. Theoretcally, the larger the sample sze, the better the survey precson, but ths s not always true n practce. Survey budget s not the only mportant factor n determnng the sample sze. Desred precson, the number of domans, capablty of the mplementng organzaton, data qualty concerns and cost effectveness are essental constrants n determnng the total sample sze. Thus a sutable sample sze s also a key parameter to guarantee data qualty Smple desgn In large-scale surveys, non-samplng errors (coverage errors, errors commtted n survey mplementaton and data processng, etc.) are usually the most mportant sources of error and are expensve to control and dffcult to evaluate quanttatvely. It s therefore mportant to mnmze them n survey mplementaton. In order to facltate accurate mplementaton of the survey, the samplng desgn for DHS should be as smple and straghtforward as possble. Macro s experence from 25 years of DHS surveys shows that a two-stage household-based sample desgn s relatvely easy to mplement and that qualty can be mantaned Household lstng and pre-selecton of households The DHS standard procedure recommends that households be pre-selected n the central offce pror to the start of feldwork rather than by teams n the feld who may have pressures to bas the selecton. The ntervewers are asked to ntervew only the pre-selected households. In order to prevent bas, no changes or replacements are allowed n the feld. To perform pre-selecton of households, a complete lst of all resdental households n each of the selected sample clusters s necessary. Ths lst s usually obtaned from a household lstng operaton conducted before the man survey. In some surveys, the household lstng operaton may be combned wth the man survey to form a sngle feld operaton, and households can be selected n the feld from a complete lstng. Combnng the household lstng and survey data collecton n one feld operaton s less expensve; however, t provdes ncentve to leave households off the household lst to reduce workload, thus reducng the representatveness of the survey results. Close supervson s needed durng the feld work to prevent ths problem. Separate lstng and data collecton operatons are thus requred for ths reason. Intervewers selectng households n the feld wthout a complete lstng s not acceptable for DHS surveys Good sample documentaton DHS surveys are usually year-long projects conducted by dfferent people specalzed n dfferent aspects of survey mplementaton, so good sample documentaton s necessary to guarantee the exact mplementaton of the project. The sample documentaton should nclude a sample desgn 2

13 document and the lst of prmary samplng unts. The sample desgn document should explan n detal the methodology, the samplng procedure, the sample sze, the sample allocaton, the survey domans and the stratfcaton. Ths should also form the bass for an appendx to the DHS fnal report descrbng the sample desgn. The sample lst should nclude all dentfcaton nformaton for all of the selected sample ponts, along wth ther probablty of selecton Confdentalty Confdentalty s a major concern n DHS, especally when human bo-markers are collected such as blood samples for HIV testng. The DHS surveys are anonymous surveys whch do not allow any potental dentfcaton of any sngle household or ndvdual n the data fle. Confdentalty s also a key factor affectng the response rate to senstve questons regardng sexual actvty and partners. In partcular, n surveys that nclude HIV testng DHS polcy requres that PSU and household codes are scrambled n the fnal data to further anonymze the data and the orgnal sample lst s destroyed Exactness of survey mplementaton Exactness of sample mplementaton s the last element n achevng good samplng precson. No matter how carefully a survey s desgned and how complete the materals for conductng samplng actvtes are, f the mplementaton of the samplng actvtes by samplng staff (offce staff responsble for selectng sample unts, feld workers responsble for the mappng and household lstng and ntervewers responsble for data collecton) s not preformed exactly as desgned, serous bas and msleadng results may occur. In the sectons that follow, DHS polces related to sample desgn and mplementaton are descrbed. 1.2 Survey objectves and target populaton The man objectve of DHS surveys s to collect up-to-date nformaton on basc demographc and health ndcators, ncludng housng characterstcs, fertlty, chldhood mortalty, contraceptve knowledge and use, maternal and chld health, nutrtonal status of mothers and chldren, knowledge, atttudes and behavor toward HIV/AIDS and other sexually transmtted nfectons (STI), women s status. The target populaton for DHS s defned as all women of reproductve age (15-49 years old) and ther young chldren under fve years of age lvng n ordnary resdental households. However, n some countres, the coverage may be restrcted to ever-marred women. The man ndcator topcs nclude: Total fertlty and age specfc fertlty rates Age at frst sex, frst brth, and frst marrage Knowledge and use of contracepton Unmet need for famly plannng Brth spacng Antenatal care Place of delvery Assstance from sklled personnel durng delvery Knowledge of HIV/AIDS and other STIs Hgher-rsk sexual behavor Condom use Chldhood vaccnaton coverage 3

14 Treatment of darrhea, fever, and cough Infant and under-fve mortalty rates Nutrtonal status Snce the target populaton can be easly found n resdental households, DHS s a householdbased survey. 1.3 Survey doman In DHS surveys, an mportant objectve s to compare the survey results for dfferent characterstcs such as urban and rural resdence, dfferent admnstratve or geographc regons, or dfferent educatonal levels of respondents. A survey doman or study doman s a sub-populaton for whch separate estmaton of the man ndcators s requred. There are two knds of survey domans: desgn domans and analyss domans. A desgn doman conssts of a sub-populaton whch can be dentfed n the samplng frame and therefore can be handled ndependently n the sample sze and samplng procedures, usually consstng of geographc areas or admnstratve unts. For example, urban and rural dfferences are very frequently requested; therefore, urban and rural areas are usually separate desgn domans for Demographc and Health Surveys. An analyss doman s a sub-populaton whch cannot be dentfed n the samplng frame, such as domans specfed by ndvdual characterstcs. These may nclude women wth secondary or hgher educaton, pregnant women, chldren months, and chldren havng darrhea n the two weeks precedng the survey. In order for survey estmates to be relable at the doman level, t s necessary to ensure that the number of cases n each survey doman s suffcent, especally when desred levels of precson are requred for partcular domans. For a desgn doman, adequate sample sze s acheved by allocatng the target populaton at the survey desgn stage nto the requested desgn domans, and then calculatng the sample sze for the specfc desgn domans by takng the precson requred nto account. On the other hand, for an analyss doman, t s dffcult to guarantee a specfed precson because t s dffcult to control the sample sze at the desgn stage. However, f pror estmates of the average number of target ndvduals per household are avalable, then t s possble to control the precson for an analyss doman. For example, f survey estmates are requred for the nutrtonal status of chldren under age 5 s requred and estmates of the number of chldren under age 5 per household are avalable, t s then possble to calculate a sample sze to gve a certan level of precson. DHS reports also produce some ndcators for second level domans such as vaccnaton coverage of chldren age months wthn a regon, where regon s the frst level doman, and chldren months s the second level doman. Cauton must be pad to the precson requred for a second level doman because the second level doman usually ncludes a very small sub-populaton. If doman-level estmates are requred, t s better to avod a large number of domans because otherwse a very large sample sze wll be needed. The number of domans and the desred level of precson for each must be taken nto account n the budget calculaton and assessment of the mplementaton capabltes of the mplementng organzaton. The total sample sze needed s the sum of sample szes needed n all exclusve (frst level) domans. 1.4 Samplng frame A samplng frame s a complete lst of all samplng unts that entrely covers the target populaton. The exstence of a samplng frame allows a probablty selecton of samplng unts. For a mult-stage survey, a samplng frame should exst for each stage of selecton. The samplng unt for the frst stage of selecton s called the Prmary Samplng Unt (PSU); the samplng unt for the second stage of selecton s called the Secondary Samplng Unt (SSU), and so on. In most cases, DHS 4

15 surveys are two-stage surveys. Note that each stage of sample selecton wll nvolve samplng errors, so t s better to avod more than two stages f addtonal stages of selecton are not necessary. The avalablty of a sutable samplng frame s a major determnant of the feasblty of conductng a DHS survey. Ths ssue should be addressed n the earlest stages of plannng for a survey. A samplng frame for a DHS survey could be an exstng samplng frame, an exstng master sample, or a sample of a prevously executed survey of suffcently large sample sze, whch allows for the selecton of subsamples of desred sze for the DHS survey Conventonal samplng frame The best frame s the lst of Enumeraton Areas (EAs) from a recently completed populaton census. An EA s usually a geographc area whch groups a number of households together for convenent countng purposes for the census. A complete lst of EAs whch covers the survey area entrely s the most deal frame for DHS surveys. In most cases, a lst of EAs from a recent census s avalable. Ths lst should be thoroughly evaluated before t s used. The samplng frame used for DHS should be as up-to-date as possble. It should cover the whole survey area, wthout omsson or overlap. Basc cartographc materals should exst for each area unt or at least for groups of unts wth clearly defned boundares. Each area unt should have a unque dentfcaton code or a seres of codes that, when combned, can serve as a unque dentfcaton code. Each unt should have at least one measure of sze estmate (populaton and/or number of households). If other characterstcs of the area unts (e.g., socoeconomc level) exst, they should be evaluated and retaned as they may be used for stratfcaton. A pre-exstng master sample (whch s a random sample from the census frame) can be accepted only where there s confdence n the master sample desgn, ncludng detaled samplng desgn parameters such as samplng method, stratfcaton, and ncluson probablty for the selected prmary samplng unts. The task for the DHS survey s then to desgn a sub-samplng procedure, whch produces a sample n lne wth DHS requrements. Ths wll not always be possble. However, the larger the master sample s n relaton to the desred DHS sub-sample, the more flexblty there wll be for developng a sub-samplng desgn. A key queston wth a pre-exstng sample s whether the lstng of dwellngs/households s stll current or whether t needs to be updated. If updatng s requred, use of a pre-exstng sample may not be economcal. The potental advantages of usng a pre-exstng sample are: 1) economy, and 2) ncreased analytc power through comparatve analyss of two or more surveys. The dsadvantages are: 1) the problem of adaptng the sample to DHS requrements, and 2) the problem of repeated ntervews wth the same household or person n dfferent surveys, resultng n respondent fatgue or contamnaton. One way to avod ths last problem s to keep just the prmary samplng unts from the pre-exstng sample and reselect the households for the DHS survey Alternatve samplng frames When nether a census frame nor a master sample s avalable then alternatve frames should be consdered. Examples of such frames are: A lst of electoral zones wth estmated number of qualfed voters for each zone A grdded hgh resoluton satellte map wth estmated number of structures for each grd A lst of admnstratve unts such as vllages wth estmated populaton for each unt A man concern when usng alternatve frames are coverage problems, that s, does the frame completely cover the target populaton? Usually checkng the qualty of an alternatve frame s more dffcult because of a lack of nformaton ether from the frame tself or from admnstratve sources. 5

16 Another problem s the sze of the prmary samplng unt. Snce the alternatve frame s not specfcally created for a populaton census or household based survey, the sze of the PSUs of such frames may be too large or too small for a DHS survey. A thrd problem s dentfyng the boundares of the samplng unts due to the lack of cartographc materals. In the frst two examples of alternatve samplng frames, the standard DHS two-stage samplng procedure can be appled by treatng the electoral zones or the grds of satellte map as the PSUs. In the thrd case, when a lst of admnstratve unts larger than vllages (e.g. sub-dstrcts, wards or communes) s avalable, for example, a complete lst of all communes n a country may be easer to get than a complete lst of vllages, then t s necessary to use a selecton procedure that ncludes more than two stages. In the frst stage, select a number of communes; n each of the selected communes, construct a complete lst of all vllages resdng n the commune; select one vllage per commune as a DHS cluster, then proceed wth the subsequent household lstng and selecton as n a standard DHS. Ths procedure works best when the number of communes s large and the commune sze s small. A lst of admnstratve unts that are small n number but large n sze s not sutable for a DHS samplng frame because ths stuaton wll result n large samplng errors, as explaned later n Secton Evaluaton of the samplng frame No matter what knd of samplng frame wll be used, t s always necessary to check the qualty of the frame before selectng the sample. Followng are several thngs that need to be checked when usng a conventonal samplng frame: Coverage Dstrbuton Identfcaton and codng Measure of sze Consstency There are several easy but useful ways to check the qualty of a samplng frame. For example, for a census frame, check the total populaton of the samplng frame and the populaton dstrbuton among urban and rural areas and among dfferent regons/admnstratve unts obtaned from the frame wth that from the census report. Any mportant dfferences may ndcate that there may be coverage problems. If the frame provdes nformaton on populaton and households for each EA, then the average number of household members can be calculated, and a check for extreme values can help to fnd ncorrect measures of sze of the PSUs. If nformaton on populaton by sex s avalable for each EA, then a sex rato can be calculated for each EA, and a check for extreme values can help to dentfy non-resdental EAs. If the EAs are assocated wth an dentfcaton (ID) code, then check the ID codes to dentfy mscoded or msplaced EAs. A samplng frame wth full coverage and of good qualty s the frst element for a DHS survey; therefore, efforts should be made to guarantee a good start for the project. For a natonally representatve survey, geographc coverage of the survey should nclude the entre natonal terrtory unless there are strong reasons for excludng certan areas. If areas must be excluded, they should consttute a coherent doman. A survey from whch a number of scattered zones have been excluded s dffcult to nterpret and to use. 1.5 Stratfcaton Stratfcaton s the process by whch the survey populaton s dvded nto subgroups or strata that are as homogeneous as possble usng certan crtera. Explct stratfcaton s the actual sortng and separatng of the unts nto specfed strata. Wthn each stratum, the sample s desgned and 6

17 selected ndependently. It s also possble to systematcally sample unts from an ordered lst (wth a fxed samplng nterval between selected unts) to acheve the effect of stratfcaton. For example, n DHS survey, t s not unusual for the PSUs wthn the explct strata to be sorted geographcally. Ths s called mplct stratfcaton. The prncpal objectve of stratfcaton s to reduce samplng errors. In a stratfed sample, the samplng errors depend on the populaton varance exstng wthn the strata but not between the strata. For ths reason, t pays to create strata wth low nternal varablty (or hgh homogenety). Another major reason for stratfcaton s that, where marked dfferences exst between subgroups of the populaton (e.g., urban vs. rural areas), stratfcaton allows for a flexble sample desgn that can be dfferent for each subgroup. Stratfcaton should be ntroduced only at the frst stage of samplng. At the dwellng/household selecton stage, systematc samplng s used for convenence; however, no attempt should be made to reorder the dwellng/household lst before selecton n the hope of ncreasng the mplct stratfcaton effect. Such efforts generally have a neglgble effect. Stratfcaton can be sngle-level or mult-level. In sngle-level stratfcaton, the populaton s dvded nto strata accordng to certan crtera. In mult-level stratfcaton, the populaton s dvded nto frst-level strata accordng to certan crtera, and then the frst-level strata are subdvded nto second-level strata, and so on. A typcal two-level stratfcaton nvolves frst stratfyng the populaton by regon at the frst level and then by urban-rural wthn each regon. A DHS survey usually employs mult-level stratfcaton. Strata should not be confused wth survey domans. A survey doman s a populaton subgroup for whch separate survey estmates are desred (e.g., urban areas/rural areas). A stratum s a subgroup of homogeneous unts (e.g., subdvsons of an admnstratve regon) n whch the sample may be desgned dfferently and s selected separately. Survey domans and strata can be the same but they need not be. For example, survey domans could be the frst-level stratum n a mult-level stratfcaton. On the other hand, a survey doman could consst of one or several lower-level strata. DHS surveys typcally use explct stratfcaton by separatng urban and rural resdence wthn each regon. Where data are avalable, explct stratfcaton could also be done on the bass of socoeconomc zones or more drectly relevant characterstcs such as the level of female lteracy or the presence of health facltes n the areas. These knds of nformaton could be obtaned from admnstratve sources. Wthn each explct stratum, the unts can then be ordered accordng to locaton, thus provdng further mplct geographc stratfcaton. 1.6 Sample sze Sample sze and samplng errors The estmates from a sample survey are affected by two types of errors: samplng errors and non-samplng errors. Samplng errors are the representatve errors due to samplng of a small number of elgble unts from the target populaton nstead of ncludng every elgble unt n the survey. Samplng errors are related to the sample sze and the varablty among the samplng unts. Samplng errors can be statstcally evaluated after the survey. Non-samplng errors result from problems durng data collecton and data processng, such as falure to locate and ntervew the correct household, msunderstandng of the questons on the part of ether the ntervewer or the respondent, and data entry errors. Non-samplng errors are related to the capacty of the mplementng organzaton, and experence shows that (1) non-samplng errors are always the most mportant source of error n a survey, and (2) t s dffcult to evaluate the magntude of non-samplng errors once a survey s complete. Theoretcally, wth the same survey methodology and under the same survey condtons, 7

18 the larger the sample sze, the better the survey precson. However, ths relatonshp does not always hold true n practce, because non-samplng errors tend to ncrease wth survey scale and sample sze. The challenge n decdng on the sample sze for a survey s to balance the demands of analyss and precson wth the capacty of the mplementng organzaton and the constrants of fundng. A common measure of precson for estmatng an ndcator s ts relatve standard error (RSE) whch s defned as ts standard error (SE) dvded by the estmated value of the ndcator. The standard error of an estmator s the representatve error due to samplng. The relatve standard error descrbes the amount of samplng error relatve to the ndcator level and s ndependent of the scale of the ndcator to be estmated; therefore, a unque RSE can be appled to a reference ndcator for all domans. If a unque RSE s desred for all domans, the doman sample sze depends on the varablty and the sze of the doman. The total sample sze s the sum of the sample szes over all domans for whch desred precson are requred. The followng are some concepts related to sample sze calculaton. 1. The standard error of an estmator when estmatng a proporton wth a smple random samplng wthout replacement 2 s gven by: 1 - f N SE = SQRT P(1 P) n N 1 where n s the sample sze (number of completed ntervews), P s the proporton, N s the target populaton sze, and f=n/n s the samplng fracton. When N s large and n s relatvely small, the above quantty can be approxmated by: Therefore the RSE of the estmator s gven by: P(1 P) SE SQRT n P(1 P) RSE( P) SQRT / P n 1 / P 1 = SQRT n 2. For a requred precson wth a relatve standard error α, the net sample sze (number of completed ntervews) needed for a smple random samplng s gven by: (1 / P 1) n = 2 α 3. Snce a smple random samplng s not feasble for a DHS, the sample sze for a complex survey wth clusterng such as the DHS can be calculated by nflatng the above calculated sample sze by usng a desgn effect (Deft). Deft s a measure of effcency of cluster samplng compared to a drect smple random samplng of ndvduals, defned as the rato between the standard error usng the gven sample desgn and the standard error that would result f a smple random sample had been used. A Deft value of 1.0 ndcates that the sample desgn s 2 A smple random sample would be a random selecton of ndvduals or households drectly from the target populaton. Ths s not feasble for DHS surveys because a lst of all elgble ndvduals or households s not avalable. 8

19 as effcent as a smple random sample, whle a value greater than 1.0 ndcates the ncrease n the samplng error due to the use of a more complex and less statstcally effcent desgn. The net sample sze needed for a cluster samplng wth same relatve standard error s gven by: n = Deft 2 (1 / P 1) 2 α 4. The formula for calculatng the fnal sample sze n terms of the number of households whle takng non-response nto account (the formula used n the templates for sample sze calculaton as shown n Table 1.1) s gven by: 2 (1/ P 1) n = Deft 2 α ( R Rh d) where n Deft P α R R h d s the sample sze n households; s the desgn effect (a default value of 1.5 s used for Deft f not specfed); s the estmated proporton; s the desred relatve standard error; s the ndvdual response rate; s the household gross response rate; and s the number of elgble ndvduals per household. The household gross response rate s the number of households ntervewed over the number selected. DHS reports typcally report the net household response rate whch s the number of households ntervewed over the number vald households found n the feld (.e. excludng vacant and destroyed dwellngs.) 5. If the target populaton s small (such as n a sub-natonal survey), a fnte populaton correcton of the above calculated sample sze should be appled. The fnal sample sze n s calculated by n = n + n / N where n 0 s the ntal sample sze calculated n pont number 4, and N s the target populaton sze. 6. The relatonshp between the RSE and the sample sze shows that, f one reduces a desred RSE to half, then the sample sze needed wll ncrease 4 tmes. For example, the sample sze for a RSE of 5% s 4 tmes larger than the sample sze for a RSE of 10% (see Tables 1.1 and 1.2 n the next secton). Ths means that t s very expensve to reduce the RSE by ncreasng the sample sze. Therefore, when desgnng the sample sze, the effcency of the desgn must be consdered, that s, the balance between the gan n precson and the ncrease n sample sze (or survey cost). 7. The wdth of the confdence nterval s determned by the RSE. Wth a confdence level of 95%, 2*P*RSE s the half-length of the confdence nterval for P. For example, for RSE=0.10 and P=0.20, the half-length of the confdence nterval s 0.04, whch means the confdence nterval for P s (0.16, 0.24). (DHS reports +/-2*SE nstead of +/-1.96*SE as 95% confdence nterval for conservatve purposes). 9

20 1.6.2 Sample sze determnaton The total sample sze for a DHS survey wth a number of survey domans (desgn doman) s the sum of the sample szes over all domans. An approprate sample sze for a survey doman s the mnmum number of persons (e.g., women age 15-49, currently marred women 15-49, chldren under age fve) that acheves the desred survey precson for core ndcators at the doman level. If fundng s tght and fxed, the sample sze s the maxmum number of persons that the fundng can cover. Precson at the natonal level s usually not a problem. In almost all cases, sample sze s decded to guarantee precson at doman level wth approprate allocaton of the sample. So apart from survey costs, the total sample sze depends on the desred precson at doman level and the number of domans. If a reasonable precson s requred at doman level, experence from the MEASURE DHS program shows that a mnmum number of 800 completed ntervews wth women s necessary for some of the woman-based ndcators for hgh fertlty countres (e.g. total fertlty rate, contraceptve prevalence rate, chldhood mortalty rates); for low fertlty countres, the mnmum doman sample sze can reach 1,000 completed ntervews or more. Table 1.1 below llustrates the calculaton of sample sze for a doman accordng to dfferent levels of desred RSE for estmatng the ndcator the proporton of currently marred women who are current users of a modern contraceptve method. Table 1.1 Sample sze determnaton for estmatng current use of a modern contraceptve method among currently marred women Estmated proporton p 0.20 Total target populaton Estmated desgn effect (Deft) 1.40 # of target ndvduals/hh 1.05 Indvdual response rate 0.96 HH gross response rate 0.92 Desred Net Sample Sample sze Expected 95% confdence lmts RSE sze ndvdual Household SE Lower Upper Note: The confdence lmts are calculated as P±2*SE. 10

21 Assumng the doman sze s large enough such that the fnte populaton correcton s neglgble, Table 1.1 gves the requred gross sample sze n terms of number of households wth estmated parameters from a DHS survey. The target populaton s currently marred women age 15-49; the estmated parameters are: the proporton of currently marred women who are current users of any modern contraceptve method, the desgn effect (Deft), the number of target ndvduals (number of currently marred women 15-49) per household, the ndvdual and the household response rates. For example, wth an estmated prevalence of 20%, f we requre a RSE of 10%, we should select 846 households n ths partcular doman. Wth a gross household response rate (the number of households completed over the total number selected) of 92% and an ndvdual response rate of 96%, we expect to obtan 784 completed ntervews of currently marred women age The estmated quanttes at the top of the table used as nput to the calculaton can usually be obtaned from prevous surveys or from admnstratve records. The total sample sze for a survey wth several domans s the sum of the sample szes obtaned n the above table for each doman. If the same precson requred and the same ndcator level apply to all domans, then the total sample sze s the sample sze calculated for one doman multpled by the number of domans. Wth ths example, the total sample sze for a survey havng sx domans wth approxmately the same level of modern contraceptve use among currently marred women and the same precson request for each doman would be 5076 households. The Sample sze determnaton template located n the Appendx can be used to determne requred sample szes. Table 1.2 Sample sze determnaton for estmatng the prevalence of full vaccnaton coverage among chldren aged months Estmated proporton p 0.29 Total target populaton Estmated desgn effect (Deft) 1.22 # of target ndvduals/hh 0.11 Indvdual response rate 0.96 HH gross response rate 0.92 Desred Net Sample Sample sze Expected 95% confdence lmts RSE sze ndvdual household SE Lower Upper Note: The default value of Deft s set to be 1.5. Specfy f dfferent. The confdence lmts are calculated as P±2*SE. If response rate s not provded, the sample sze calculated s net sample sze. 11

22 Table 1.2 shows a smlar example for the ndcator proporton of chldren aged months who are fully mmunzed. In ths case, the target populaton s chldren aged months. The estmated number of target ndvduals per household s much smaller than the number of currently marred women per household gven n Table 1.1. So for the same sample sze calculated n Table 1.1, we can only get a RSE of above 20% at doman level. Wth a RSE of 10%, we need to select 3746 households n ths partcular doman whch seems unrealstc f we have several domans for the survey. Ths example shows that for a mult-ndcator survey, the sample sze requred can be very dfferent from ndcator to ndcator. So the choce of the reference ndcator upon whch the sample sze s calculated s an mportant ssue. The reference ndcator whch s used for sample sze determnaton should have demographc mportance, moderate value and moderate populaton coverage,.e. apply to a szable proporton of the populaton. Wth the same sample sze calculated n Table 1.1 for a survey havng sx domans, the RSE for the whole sample for estmatng full mmunzaton among chldren months s between 8% and 9%. The doman sample szes often need to be balanced between domans due to budget constrants. In practce t s often the case that the total sample sze s fxed accordng to fundng avalable and mplementaton capacty, and then the sample s allocated to each doman and to each stratum wthn the doman. In the case of very tght budget constrants, we may equally allocate the total sample to the domans. In some cases, we may want to oversample a specfc doman to conduct some n-depth analyss for a certan rare phenomenon. The method (and the tables) presented n the followng secton may be used to allocate the sample at the doman level because the domans are usually frst-level strata. Regardless of the method used for allocaton, the calculaton of doman sample sze can gve us an dea about the precson we may acheve n each doman wth a gven sample sze. 1.7 Sample allocaton In cases where the total sample sze or doman sample sze has been fxed, we need to approprately allocate the sample to dfferent domans (or dfferent strata wthn a doman). Ths allocaton s amed at strengthenng the samplng effcency at the natonal level or doman level and reducng samplng errors. Assumng a constant cost across domans/strata, the optmum allocaton of the sample depends on the sze of the doman/stratum and the varablty of the ndcator to be estmated S xh n N For a gven total sample sze n the optmum allocaton for varable x s gven by: h h n = n H h N h= 1 S h N S h xh xh S xh The optmum allocaton s only optmal for the ndcator on whch the allocaton s based; that allocaton may not be approprate for other ndcators. For a multpurpose survey, f the domans/strata are not too dfferent n sze, a safe allocaton that s good for all ndcators s a proportonal allocaton, wth sample sze proportonal to the doman/stratum sze. n = n N h h = H h=1 N h Nh n N 12

23 Ths allocaton ntroduces a constant samplng fracton across doman/strata wth: f h = n h N h = n N Because DHS surveys are multpurpose surveys, a proportonal allocaton of sample s recommended f the domans/strata are not too dfferent n sze. However, f the domans/strata szes are very dfferent, the smaller domans/strata may receve a very small sample sze. If a desred precson s requred at doman/stratum level, by assumng equal relatve varatons across strata, a power allocaton (Banker, 1988) wth an approprate power value ) may be used to guarantee suffcent sample sze n small domans/strata. α ( 0 α 1 n h = n H M h= 1 α h M α h A power allocaton s an allocaton proportonal to the power of a sze measure M. A power value of 1 gves proportonal allocaton; a power value of 0 gves equal sze allocaton; a power value between 0 and 1 gves an allocaton between proportonal allocaton and equal sze allocaton. Proportonal allocaton s good for natonal level ndcators, but may not meet the precson request at doman level; whle an equal sze allocaton s good for comparson across domans, but may affect the precson at natonal level. A power allocaton wth power values between 0 and 1 s a tradeoff between the natonal level precson and the doman level precson. Snce the sample sze s usually large at the natonal level, the natonal level precson s not a concern. In Table 1.3 below, we gve an example of a proportonal sample allocaton of 15,000 ndvduals to 11 domans and to ther urban-rural areas. The mnmum doman sample sze s 384 for doman 2, whch s too small for estmatng the total fertlty rate (TFR) and chldhood mortalty rates. The largest sample sze s for doman 11 whch may be unnecessarly large. The actual total sample sze gven n the total row may be slghtly dfferent from the desred sample sze because of roundng. 13

24 Table 1.3 Sample allocaton: Proportonal allocaton Total sample sze => Power value doman=> Power value urban=> Seral Doman/ Sample Allocaton Specfc Allocaton Doman/Stratum Proporton Num stratum Name/ID urban Urban Rural Doman Urban Rural sze 1 Doman Doman Doman Doman Doman Doman Doman Doman Doman Doman Doman Total If we mpose a condton such that the sample sze should not be smaller than 1000 n each doman, after tryng varous power values, we fnd that a power value of 0.25 s approprate, as shown n Table 1.4. In ths case, we would have a mnmum sample sze of 1,022 for doman 2. Snce doman 11 has only urban areas, the power allocaton among the domans brought down the urban percentage n the sample. In order for urban areas to be properly represented, over samplng s appled n the urban areas of the other domans. Wth a power value of 0.65, the urban proporton n the sample s close to the proporton of the target populaton. Table 1.4 Sample allocaton: Power allocaton Total sample sze => Power value doman=> 0.25 Power value urban=> 0.65 Seral Doman/ Sample Allocaton Specfc Allocaton Doman/Stratum Proporton Num stratum Name/ID urban Urban Rural Doman Urban Rural sze 1 Doman Doman Doman Doman Doman Doman Doman Doman Doman Doman Doman Total In Table 1.4, the small domans are oversampled compared wth a proportonal allocaton. Oversamplng some small domans s frequently practced f doman level precson s requred. 14

25 However, oversamplng a small doman too much wll harm the precson at natonal level. To prevent ths, t s recommended to regroup the small domans to form domans of moderate sze, especally when there s a very unequal populaton dstrbuton among geographc domans, however, ths s sometmes not possble due to poltcal consderatons. The above dscusson also apples to sample sze allocaton to strata wthn a doman where the doman sample sze s fxed. A proportonal allocaton wth sample sze proportonal to stratum sze s good for all ndcators and provdes the best precson for the doman as a whole. 1.8 Two-stage cluster samplng procedure The MEASURE DHS program utlzes a convenent and practcal sample selecton procedure for household based surveys developed on the bass of experence from past surveys a two-stage cluster samplng procedure. A cluster s a group of adjacent households whch serves as the PSU for feld work effcency. Intervewng a certan number of households n the same cluster can reduce greatly the amount of travel and tme needed durng data collecton. In most cases, a cluster s an EA wth a measure of sze equal to the number of households or the populaton n the EA, provded by the populaton census. At the frst stage, a stratfed sample of EAs s selected wth probablty proportonal to sze (PPS): n each stratum, a sample of a predetermned number of EAs s selected ndependently wth probablty proportonal to the EA s measure of sze. In the selected EAs, a lstng procedure s performed such that all dwellngs/households are lsted. Ths procedure s mportant for correctng errors exstng n the samplng frame, and t provdes a samplng frame for household selecton. At the second stage, after a complete household lstng s conducted n each of the selected EAs, a fxed (or varable) number of households s selected by equal probablty systematc samplng n the selected EAs. In each selected household, a household questonnare s completed to dentfy women age 15-49, men age (15-54 or n some surveys) and chldren under age fve. Every elgble woman wll be ntervewed wth an ndvdual questonnare, and every elgble man wll be ntervewed wth an ndvdual men s questonnare n those households selected for the men s ntervew. The advantages of ths two-stage cluster samplng procedure can be summarzed as follows: 1) It guarantees a representatve sample of the target populaton when a lst of all target ndvduals s not avalable whch prohbts a drect samplng of target ndvduals; 2) A household lstng procedure after the selecton of the frst stage and before the man survey provdes a samplng frame for household selecton n the central offce; 3) The use of resdental households as the second-stage samplng unt guarantees the best coverage of the target populaton; and 4) It reduces unnecessary samplng errors by avodng more than two stages of selecton (whch usually uses a large PSU n the frst stage of selecton). See more detals n Sectons 1.10 and 1.11 on household lstng and selecton, Chapter 2 on household lstng, and Sectons 3.2 and 3.3 of Chapter 3 on systematc samplng and samplng wth probablty proportonal to sze (PPS). 15

26 1.9 Sample take per cluster Once the total sample sze s determned and allocated to dfferent survey domans/strata, t should be decded how many ndvduals (sample take) should be ntervewed per sample cluster and then convert the doman/stratum sample sze to number of clusters. Snce the survey cost can be very dfferent across the survey domans/strata, the sample take can have a bg nfluence on the total survey budget. Wth a fxed sample sze, a small sample take s good for survey precson because of the reducton of the desgn effect, but s expensve because more clusters are needed. The number of clusters affects the survey budget more than the overall sample sze due to the travel between clusters durng data collecton, whch represents an mportant part of feld costs n rural areas. The MEASURE DHS program proposes a sample take of about women per rural cluster. In urban areas, the cost advantage of a large take s generally smaller, and MEASURE DHS recommends a take of about women per urban cluster. Snce n most DHS surveys, the number of elgble women age s very close to one per household, the sample take of ndvduals s equvalent to the sample take of households; therefore, n the followng sectons we refer to the sample take (or cluster take) as the number of sample households per cluster Optmum sample take The optmum number of households to be selected per cluster depends on the varable under consderaton, the ntracluster correlaton ρ, and the survey cost rato c 1 / c2, where c 1 represents the cost per cluster ncludng manly the cost assocated wth travellng between the clusters for survey mplementaton (household lstng and ntervew); whle c 2 represents the cost per ndvdual ntervew (the ntervewng cost) and other costs of dong feldwork wthn a cluster. A larger sample take per cluster and fewer clusters reduces survey feld costs f the cost rato s hgh, but t could also reduce the survey precson f the ntracluster correlaton s strong. The MEASURE DHS Program has accumulated nformaton on samplng errors for selected varables for many surveys throughout the world. Usng ths nformaton, Alaga and Ren (2006) conducted a research study to determne the optmum sample take per cluster. The results of the study have nformed current practce n DHS surveys. If the average cluster sze s around 250 households, a sample take of households per cluster s wthn the acceptable range n most surveys. The research also supports the practce of settng a larger sample take n rural clusters than n urban clusters. Usually, the cost rato n urban areas s smaller than that n rural areas. Ths would lead to a smaller sample take n an urban cluster than n a rural cluster. In sum, ths research ndcates that for the most mportant survey ndcators, a sample take between 20 to 25 households s approprate n urban clusters and a sample take between 25 to 30 households s approprate n rural clusters. Based on values of c 1 / c2 and ρ obtaned from eght surveys, Table 1.5 below shows optmal sample takes for the ndcator proporton of currently marred women currently usng any contraceptve method. Ths ndcator has a moderate ntracluster correlaton relatve to other mportant survey ndcators. 16