Article. Methods for oversampling rare subpopulations in social surveys. by Graham Kalton

Transcription

1 Component of Statistics Canada Catalogue no X Business Survey Metods Division Article Metods for oversampling rare subpopulations in social surveys by Graam Kalton December 2009

2 Survey Metodology, December Vol. 35, No. 2, pp Metods for oversampling rare subpopulations in social surveys Graam Kalton 1 Abstract Surveys are frequently required to produce estimates for subpopulations, sometimes for a single subpopulation and sometimes for several subpopulations in addition to te total population. Wen membersip of a rare subpopulation (or domain) can be determined from te sampling frame, selecting te required domain sample size is relatively straigtforward. In tis case te main issue is te extent of oversampling to employ wen survey estimates are required for several domains and for te total population. Sampling and oversampling rare domains wose members cannot be identified in advance present a major callenge. A variety of metods as been used in tis situation. In addition to large-scale screening, tese metods include disproportionate stratified sampling, two-pase sampling, te use of multiple frames, multiplicity sampling, location sampling, panel surveys, and te use of multi-purpose surveys. Tis paper illustrates te application of tese metods in a range of social surveys. Key Words: Sample allocation; Screening; Disproportionate stratified sampling; Two-pase sampling; Multiple frames; Location sampling; Panel surveys; Multi-purpose surveys. 1. Introduction I feel very privileged to ave been invited to present tis year s paper in te Waksberg Invited Paper Series, a series tat onors Joe Waksberg for is numerous contributions to survey metodology. I was extremely fortunate to ave ad te opportunity to work wit Joe at Westat for many years and, as did many oters, I benefited greatly from tat experience. Wen faced wit an intractable sampling problem, Joe ad a flair for turning te problem on its end and producing a workable solution. Since te problem often concerned te sampling of rare populations, I ave cosen to review metods for sampling rare populations for tis paper. One of te major developments in survey researc over te past several decades as been te continuously escalating demand for estimates for smaller and smaller subclasses (subpopulations) of te general population. Tis paper focuses on tose subclasses termed domains tat are planned for separate analysis at te sample design stage. Some examples of domains tat ave been taken into account in te sample designs of various surveys include a country s states or provinces, counties or districts; racial/ etnic minorities; ouseolds living in poverty; recent birts; persons over 80 years of age; recent immigrants; gay men; drug users; and disabled persons. Wen te domains are small (also known as rare populations), te need to provide adequate sample sizes for domain analysis can create major callenges in sample design. Tis paper reviews te different probability sampling metods tat are used to generate samples for estimating te caracteristics of rare populations wit required levels of precision. Sampling metods for estimating te size of a rare population are not explicitly addressed, altoug similar metods are often applicable. However, capture-recapture and related metods are not addressed in tis paper. An important issue for sample design is weter te aim of a survey is to produce estimates for a single domain or many domains. Altoug muc of te literature on te sampling of rare populations discusses sample designs for a single rare domain (e.g., drug users), in practice surveys are often designed to produce estimates for many domains (e.g., eac of te provinces in a country or several racial/etnic groups). Te U.S. National Healt and Nutrition Examination Survey (NHANES) is an example of a survey designed to produce estimates for many domains, in tis case defined by age, sex, race/etnicity and low-income status (Moadjer and Curtin 2008). In sample designs tat include many domains, te domains may be mutually exclusive (e.g., provinces or te cells of te crossclassification of age group and race/etnicity) or tey may be intersecting (e.g., domains defined separately by age group and by race/etnicity). Te size of a domain is a key consideration. Kis (1987) proposed a classification of major domains of peraps 10 percent or more of te total population, for wic a general sample will usually produce reliable estimates; minor domains of 1 to 10 percent, for wic te sampling metods in tis paper are needed; mini-domains of 0.1 to 1 percent, estimates for wic mostly require te use of statistical models; and rare types comprising less tan 0.01 percent of te population, wic generally cannot be andled by survey sampling metods. Many surveys aim to produce estimates for some major domains, some minor domains and occasionally even some mini-domains. 1. Graam Kalton, Westat, 1600 Researc Blvd., Rockville, MD 20850, U.S.A. graamkalton@westat.com.

3 126 Kalton: Metods for oversampling rare subpopulations in social surveys Since te sample sizes for most surveys are sufficient to produce estimates of reasonable precision for major domains, tere is generally no need to adopt te kinds of oversampling procedures reviewed in tis paper. However, tere are some important design features tat sould be considered. It is, for example, valuable to take major domains into account in creating te strata for te survey. Tis consideration is of particular importance wit geograpically defined domains and multistage sampling. If a geograpic domain is not made into a design stratum, te number of primary sampling units (PSUs) selected in tat domain is a random variable; te sampled PSUs in strata tat cut across te domain boundaries may or may not be in te domain, creating problems for domain estimation. It is also valuable to ave a sizable number of sampled PSUs in eac geograpical domain in order to be able to compute direct variance estimates of reasonable precision, implying te need to spread te sample across a large number of PSUs. At te estimation stage, it is preferable, were possible, to apply nonresponse and noncoverage poststratification-type adjustments at te domain rater tan te national level. Sing, Gambino and Mantel (1994) and Marker (2001) discuss design issues and Rao (2003, pages 9-25) discusses estimation issues for major domains. Major domains will receive little attention in tis paper. At te oter end of te size continuum, even wit te use of special probability sampling metods, te sample sizes possible for most surveys are not large enoug to produce standard design-based, or direct, estimates of caracteristics for multiple domains wen many of te domains are minidomains or rare types. An obvious exception is a national population census, but censuses too ave teir limitations. Since tey are conducted infrequently (in many countries only once a decade), teir estimates are dated a particular concern for mini-domains, wic can experience rapid canges. Also, te content of a census must be severely limited in terms of te range of topics and dept of detail. Very large continuous surveys suc as te American Community Survey (U.S. Census Bureau 2009a; Citro and Kalton 2007), te Frenc rolling census (Durr 2005) and te German Microcensus (German Federal Statistical Office 2009) ave been developed to address te need for more upto-date data for small domains, but a restriction on content remains (altoug te content of te German Microcensus does vary over time). Oter exceptions occur at te border between mini-domains and minor domains. For example, since 2007 te Canadian Community Healt Survey as provided estimates on te ealt status of te populations of eac of Canada s 121 ealt regions based on an annual survey of around 65,000 persons aged 12 and over, wit te production of annual and biennial data files (Statistics Canada 2008). By combining te samples across multiple years, researcers are able to produce estimates for rare populations of various types. In general, owever, te maximum sample size possible for a survey on a specific topic is not adequate to yield a large set of mini-domain estimates of acceptable precision. Yet policy makers are making increasing demands for local area data at te mini-domain level. Tis demand for estimates for mini-domains, mainly domains defined at least in part by geograpical administrative units, is being addressed by te use of statistical modeling tecniques, leading to model-dependent, indirect, small area estimates. Tus, for example, te U.S. Census Bureau s Small Area Income and Poverty Estimates program produces indirect estimates of income and poverty statistics for 3,141 counties and estimates of poor scool-age cildren for around 15,000 scool districts every year, based on data now collected in te American Community Survey and predictor variables obtained from oter sources available at te local area level, suc as tax data (U.S. Census Bureau 2009b). A compreensive treatment of indirect estimation using small area estimation tecniques, a metodology tat falls outside te scope of tis paper, can be found in Rao (2003). Apart from location sampling, discussed in Section 3.6, tis paper also does not address te various metods tat ave been developed for sampling oter types of minidomains of muc interest to social researcers and epidemiologists, domains tat are often idden populations in tat te activities defining tem are clandestine, suc as intravenous drug use (Watters and Biernacki 1989). A range of metods as been developed under te assumption tat te members of te mini-domains know eac oter. Te broad class of suc designs is termed link-tracing designs (see te review by Tompson and Frank 2000). Tey are adaptive designs in tat te units are selected sequentially, wit tose selected at later stages dependent on tose selected earlier (Tompson and Seber 1996; Tompson 2002). Snowball sampling was one of te early metods of an adaptive, cain-referral sample design. It starts wit some initial sample of rare domain members (te seeds), and tey in turn identify oter members of te domain. Wile it bears a resemblance to network (multiplicity) sampling (described in Section 3.5), snowball sampling lacks te probability basis of te latter tecnique, i.e., known, non-zero, selection probabilities for all members of te domain. A version of snowball sampling as been termed respondent-driven sampling (RDS) (Heckatorn 1997, 2007). Volz and Heckatorn (2008) develop a teory for RDS tat is based on four assumptions: (1) tat respondents know ow many members of te network are linked to tem (te degree); (2) tat respondents recruit oters from teir personal network at random; (3) tat network connections are reciprocal; and

4 Survey Metodology, December (4) tat recruitment follows a Markov process. Te need for tese modeling assumptions for statistical inference is te difference between cain-referral sample designs and te conventional probability sample designs used in surveys wic do not need to invoke suc assumptions. It is apparent tat RDS is appropriate only for mini-domains for wic clear networks exist. Te metod is used mainly in local area settings, but Katzoff, Sirken and Tompson (2002) and Katzoff (2004) ave suggested tat te seeds could come from a large-scale survey, suc as te U.S. National Healt Interview Survey. Tis paper focuses on te use of probability sampling metods to produce standard design-based, or direct, estimates for caracteristics of rare populations, building on previous reviews (e.g., Kis 1965a; Kalton and Anderson 1986; Kalton 1993a, 2003; Sudman and Kalton 1986; Sudman, Sirken and Cowan 1988; and Flores Cervantes and Kalton 2008). Muc of te literature deals wit te sampling issues tat arise wen te rare population is te sole subject of study. However, as noted above, surveys are often required to produce estimates for many different domains as well as for te total population. Section 2 reviews te design issues involved wen te survey as design objectives for multiple domains wose members can be identified from te sampling frame. Te main part of te paper, Section 3, provides a review of a range of metods tat ave been used to sample rare populations wose members cannot be identified in advance. Te paper ends wit some concluding remarks in Section Multi-domain allocations Te issue of sample allocation arises wen a survey is being designed to produce estimates for a number of different domains, for subclasses tat cut across te domains, as well as for te total population. In most applications, domains vary considerably in size wit at least some of tem being rare domains. Assume tat tere are H mutually exclusive and exaustive domains tat are identified on te sampling frame. Under te commonly made assumptions tat te variance of an estimate for domain can be expressed as V / n and tat survey costs are te same across domains, te optimum allocation for estimating te overall population mean is n W, were W is te proportion of te population in domain. Assuming tat te domain estimates are all to ave te same precision, te optimum allocation is n = n / H for all domains. Tese two allocations are in conflict wen te W vary greatly, as often occurs wen te domains are administrative areas of te country, suc as states, provinces, counties or districts. In suc cases, adopting te optimum allocation for one objective leads to a serious loss of precision for te oter. However, a compromise allocation tat falls between te two optimum allocations often works well for bot objectives. Several compromise solutions exist. One, proposed by Kis (1976, 1988), is to determine te domain sample sizes by te following formula: n IW + (1 I ) H, 2 2 were I and (1 I) represent te relative importance of te national estimate and te domain (e.g., administrative district) estimates, respectively. If I = 1, te allocation is a proportionate allocation, as optimum for te national estimate, wereas if I = 0, te allocation is an equal allocation, as optimum for te domain estimates. Te coice of I is igly subjective, but I ave found tat I = 0.5 is often a good starting point, after wic a careful review of te allocation can lead to modifications. Bankier (1988) as proposed a similar compromise solution, termed a power allocation. Applied to te current example, te domain q sample sizes are determined from n W, were q is a power between 0 (equal allocation) and 1 (proportionate allocation). As an example, te 2007 Canadian Community Healt Survey was designed to attac about equal importance to te estimates for provinces and ealt regions. Te sample allocation to a province was based on its population size and its number of ealt regions. Witin a province, te sample was allocated between ealt regions using te Bankier allocation wit q = 0.5 (Statistics Canada 2008). A limitation to te Kis and Bankier procedures is tat tey may not allocate sufficient sample to small domains to produce estimates at te required level of precision. Tis limitation can be addressed by revising te initial allocations to satisfy precision requirements. An alternative approac addresses tis limitation directly: te allocation is determined by fixing a core sample tat will satisfy one of te objectives and ten supplementing tat sample as needed to satisfy te oter objective. Sing, Gambino and Mantel (1994) describe suc a design for te Canadian Labour Force Survey, wit a core sample to provide national and provincial estimates and, were needed, supplemental samples to provide subprovincial estimates of acceptable precision. Te Kis and Bankier scemes assume tat te same precision level is required for all small domains. Longford (2006) describes a more general approac in wic inferential priorities P d are assigned to eac domain d. As a an example, e proposes setting te priorities as Pd = Nd, were N d is te population size of domain d and a is a value cosen between 0 and 2. Te value a = 0 corresponds to te Kis and Bankier equal domain sample size assumption and a = 2 corresponds to an overall proportionate

5 128 Kalton: Metods for oversampling rare subpopulations in social surveys allocation. An intermediate value of a attaces greater priority to larger domains. Longford also extends te approac to incorporate an inferential priority for te overall estimate. A more general approac to sample allocation is via matematical programming, as as been proposed by a number of researcers (see, for example, Rodriguez Vera 1982). Tis approac can accommodate unequal variances across domains, intersecting domains, and multiple estimates for eac domain. Te U.S. Early Cildood Longitudinal Study Birt Coort (ECLS-B) provides an example wit intersecting domains, wit te sample selected from birt certificate records tat contained te requisite domain information. Tere were 10 domains of interest for te ECLS-B: birts classified by race (5 domains), birt weigt (3 domains) and twins or non-twins (2 domains). Te approac adopted first determined a minimum effective sample size (i.e., te actual sample size divided by te design effect) for eac domain. Wit te 30 cells of te cross-classification of birt weigt, race/ etnicity and twin/non-twin treated as strata, an allocation of te sample across te strata was ten determined to minimize te overall sample size wile satisfying te effective sample size requirements for all te domains (Green 2000). Wen tere are multiple domains of interest and multistage sampling is to be used, a variant of te usual measure of size for probability proportional to size (PPS) sampling can be useful for controlling te sample sizes in te sampled clusters (PSUs, second-stage units, etc.), provided tat reasonable estimates of te domain population sizes are available by cluster. Te requirements tat all sampled clusters ave approximately te same overall subsample size and tat sampled units in eac domain ave equal probabilities of selection can bot be met by sampling te clusters wit standard PPS metods, but wit a composite measure of size tat takes account of te differing sampling rates for different domains (Folsom, Potter and Williams 1987). As an example, in a survey of men in Englis prisons, te desired sampling fractions were 1 in 2 for civil prisoners ( C ), 1 in 21 for star prisoners wo are normally serving teir first term of imprisonment ( S ) and 1 in 45 for recidivists ( R ). Prisons were selected at te first stage of sampling, wit prison i being selected wit probability proportional to its composite measure of size R i + 2.2Si Ci, were te multipliers are te sampling rates relative to te rate for recidivists (Morris 1965, pages ). 3. Metods for oversampling rare domains Te main focus of tis paper is on te use of probability sampling metods to produce standard design-based, or direct, estimates for caracteristics of rare populations, often minor domains in Kis s terminology. As preparation for te subsequent discussion, it will be useful to note some features of different types of rare populations tat, togeter wit te survey s mode of data collection, are influential in te coice of sampling metods tat can be applied to generate required sample sizes for all domains. Some important features for consideration are summarized below: Is a separate frame(s) available for sampling a rare population? Can tose sampled be located for data collection? How up-to-date and complete is te frame? If an existing up-to-date frame contains only te rare population (wit possibly a few oter listings) and provides almost complete coverage, ten sampling can follow standard metods. If no single frame gives adequate coverage but tere are multiple frames tat between tem give good coverage, issues of multiple routes of selection arise (Section 3.4). Is te rare population concentrated in certain, identifiable parts of te sampling frame, or is it fairly evenly spread trougout te frame? If it is concentrated, disproportionate stratification can be effective (Section 3.2). If a sample is selected from a more general population, can a sampled person s membersip in te rare population be determined inexpensively, suc as from responses to a few simple questions? If so, standard screening metods may be used (Section 3.1). If accurate determination requires expensive procedures, suc as medical examinations, a two-pase design may be useful (Section 3.3). A related issue is weter some members of a rare population consider teir membersip to be sensitive; te likeliood tat members may be tempted to deny teir membersip may influence te coice of survey administration mode and oter aspects of screening. Are members of te rare population readily identified by oters? If so, some form of network, or multiplicity, sampling may be useful (Section 3.5). Are members of te rare population to be found at specific locations or events? If so, location sampling may be useful (Section 3.6). Is te rare population defined by a constant caracteristic (e.g., race/etnicity) or by a recent event (e.g., a ospital stay)? Te distinction between tese two types of caracteristics is important in considering te utility of panel surveys for sampling rare populations (Section 3.7).

6 Survey Metodology, December Te following sections review a range of metods for sampling rare populations. Altoug te metods are discussed individually, some are interrelated and, in practice, a combination of metods is often used. 3.1 Screening Some form of screening is generally needed wen te sampling frame does not contain domain identifiers. Tis section considers a straigtforward application of a screening design in wic a large first-pase sample is selected to identify samples of te members of te domains of interest, witout recourse to te tecniques described in later sections. Te first-pase sample size is te minimum sample size tat will produce te required (or larger) sample sizes for all of te domains. Te minimum first-pase sample size is determined by identifying te required sample size for one of te domains, wit all of te sample members of tat domain ten being included in te second-pase sample. Subsamples of oter domains are selected for te secondpase sample at rates tat generate te required domain sample sizes. If te survey is designed to collect data for only a subset of te domains (often only one domain), ten none of te members of te oter domains is selected for te second-pase sample. Since a very large screening sample size is needed to generate an adequate domain sample size wen one (or more) of te domains of interest is a rare population, te cost of screening becomes a major concern. In addition to te sampling metods discussed in later sections, tere are several strategies tat can be employed to keep costs low: Use an inexpensive mode of data collection, suc as telepone interviewing or a mail questionnaire, for te screening. Te second-pase data collection may be by te same mode or a different mode. Wen possible and useful, permit te collection of screening data from persons oter tan tose sampled. For example, oter ouseold members may be able to accurately report te rare population status of te sampled member. See te discussion below and also Section 3.5 on multiplicity sampling. Wen screening is carried out by face-to-face interviewing in a multistage design, it is efficient to select a large sample size in eac cluster. Compact clusters can also be used. Costs are reduced, and te precision of domain estimates is not seriously armed because te average domain sample sizes in te clusters will be relatively small. One possible means of reducing screening costs is to sare te costs across more tan one survey. For instance, te cild component of te ongoing U.S. National Immunization Survey (NIS) is a quarterly telepone survey tat screens ouseolds wit landline telepone numbers to locate cildren aged 19 to 35 monts, in order to ascertain vaccination coverage levels (Smit, Battaglia, Huggins, Hoaglin, Roden, Kare, Ezzati-Rice and Wrigt 2001; U.S. National Center for Healt Statistics 2009b). Te NIS largescale screening is also used to identify members of domains of interest for te State and Local Area Integrated Telepone Survey (SLAITS) program, wic addresses a variety of oter topics over time (U.S. National Center for Healt Statistics 2009c). Wen saring screening costs across a number of surveys, it is advantageous if te domains for te surveys are fairly disjoint sets in order to minimize te problems associated wit screening some respondents into more tan one survey. Wen no one is at ome to complete a face-to-face screening for a ouseold, it may be possible to obtain information from knowledgeable neigbors as to weter te ouseold contains a member of te rare population (e.g., a cild under 3 years of age). Tis approac (wic is used in NHANES) can appreciably reduce data collection costs wen a large proportion of te ouseolds do not contain members of te rare population. However, tere is a danger tat te approac may result in undercoverage; some protection is provided by requiring tat, if te first neigbor interviewed indicates tat te ouseold does not include a member of te rare population(s), te oter neigbor is also interviewed. Etical issues also must be considered, particularly for te identification of rare populations tat are sensitive in nature. An extension of te approac of collecting screening information from neigbors is known as focused enumeration. Tis tecnique, wic is a form of multiplicity sampling (see Section 3.5), involves asking te respondent at eac sampled, or core, address about te presence of members of te rare population in te n neigboring addresses on eiter side. In essence, te sample consists of 2n + 1 addresses for eac core address. If te respondent is unable to provide te screening information for one or more of te linked addresses, ten te interviewer must make contact at anoter address. Focused enumeration as been used wit n = 2 in te Britis Crime Survey (Bolling, Grant and Sinclair 2008) and te Healt Survey of England (Erens, Prior, Korovessis, Calderwood, Brookes and Primatesta 2001) to oversample etnic minorities. A limitation of te tecnique is tat it will likely produce some (possibly substantial) undercoverage. Evidence of te extent of undercoverage can be obtained by comparing te prevalence of te rare population in te core sample wit tat in te linked addresses. In surveys tat sample persons by first sampling ouseolds, survey designers often prefer to select one person per

7 130 Kalton: Metods for oversampling rare subpopulations in social surveys ouseold peraps allowing two persons to be sampled in large ouseolds to avoid contamination effects and prevent a witin-ouseold clustering omogeneity effect on design effects. Tis design is not always te best (Clark and Steel 2007), and tis particularly applies wen rare populations are sampled. Wen rare population members are concentrated in certain ouseolds (e.g., minority populations), te size of te screening sample can be appreciably reduced if more tan one person even all eligible persons can be taken in some ouseolds (see Hedges 1973). Elliott, Finc, Klein, Ma, Do, Beckett, Orr and Lurie (2008) suggest tat, for oversampling American Indian/Alaskan Native and Cinese minorities in te United States, taking all eligible persons in a ouseold as potential for U.S. ealt surveys. Te NHANES maximizes te number of sampled persons per ouseold. Since eac respondent is remunerated for participation, ouseolds wit more respondents receive more remuneration, a factor tougt to increase response rates (Moadjer and Curtin 2008). Note tat witin-ouseold omogeneity will ave little effect on design effects wen te data are analyzed by subgroup caracteristics (e.g., age and sex) tat cut across ouseolds. Te use of large-scale screening to identify rare populations raises tree issues, eac of wic could lead to a failure to acieve planned sample sizes unless precautions are taken. Te first results from te fact tat, wit screening, te sample size for a rare population is a random variable. As a result, te acieved sample size may be larger or smaller tan expected. Wen a minimum sample size is specified for a rare population, it may be wise to determine te sampling fraction to be used to ensure tat tere is, say, a 90 percent probability tat te acieved sample size will be at least as large as te specified minimum. Tis procedure was used in determining te sampling fractions for te many age, sex and income subdomains for te Continuing Survey of Food Intakes by Individuals (Goldman, Borrud and Berlin 1997). Te second issue raised by large-scale screening is tat te overall nonresponse rate must be considered. A sampled member of a rare population will be a nonrespondent if te screener information is not obtained, or if a member of te rare population is identified (peraps by a proxy informant) but does not respond to te survey items. Te overall nonresponse rate may well be muc iger tan would occur witout te screening component. Furtermore, te survey designers must consider te nature of te rare domain and te ways in wic members of tat domain will react to te survey content. A survey in wic new immigrants are asked about teir immigration experiences migt ave a very different response rate tan a survey in wic war veterans are asked about te medical and oter support services tey are receiving. Te tird issue is tat noncoverage can be a significant problem wen large-scale screening is used to identify rare populations. One source of noncoverage relates to te sampling frame used for te screener sample. Even toug a frame as good overall coverage, its coverage of a rare domain may be inadequate. For example, te noncoverage of a frame of landline telepone numbers is muc iger for ouseolds of younger people tan for te total population. Te designers of landline telepone surveys of suc rare domains as young cildren and college students terefore must carefully consider te potential for noncoverage biases. To address te problem of te substantial noncoverage of poor people in telepone surveys, te National Survey of America s Families, wic was designed to track te well-being of cildren and adults in response to welfare reforms, included an area sample of ouseolds witout telepones in conjunction wit te main random digit dialing (RDD) telepone sample (Waksberg, Brick, Sapiro, Flores Cervantes and Bell 1997). Anoter source of noncoverage is a failure to identify some members of te rare population at te screening stage. In particular, wen a survey aims to collect data only for members of a rare domain, some screening pase respondents may falsely report, and some interviewers may falsely record, tat te sampled persons are not members of tat domain. Tese misclassifications may be inadvertent or tey may be deliberately aimed at avoiding te secondpase data collection. Misclassification error can give rise to serious levels of noncoverage, particularly wen te rare population classification is based on responses to several questions, misreports to any one of wic leads to a misclassification (Sudman 1972, 1976). Wen te survey oversamples one or more rare domains as part of a survey of te general population, misclassifications are uncovered at te second pase, tus avoiding noncoverage. However, misclassifications still result in a smaller sample sizes for rare domains; in addition, te variation in sampling weigts between respondents selected as members of te rare domain and tose sampled as members of anoter domain can lead to a serious loss of precision. Noncoverage is more likely to arise wen screener data are collected from proxy informants. It is a particular problem wit focused enumeration. In a number of surveys of rare populations, te proportion of rare population members identified as been muc lower tan prevalence bencmarks. For example, te 1994 NIS ad an appreciable sortfall in te identified proportion of cildren aged 19 to 35 monts (4.1 percent compared to te predicted rate of 5 percent) (Camburn and Wrigt 1996). In te National Longitudinal Survey of Yout of 1997, only 75 percent of yout aged 12 to 23 years were located (Horrigan, Moore, Pedlow and Wolter 1999). Tese findings could be te result of iger nonresponse rates for

8 Survey Metodology, December members of te rare population, frame noncoverage of various types, or misclassifications of domain membersip. To produce te required sample size, an allowance for under-representation must be made at te design stage. Te noncoverage of an age domain appears to be greatest at te domain boundaries, peraps because respondents do not know exact ages (wit tose falsely screened out being lost and tose falsely screened in being detected and dropped later) or because of deliberate misreporting to avoid te follow-up interview. To counteract tis effect, it can be useful to start wit an initial screening for all ouseold members or for a broader age range and ten narrow down to te required age range later on. Weigting adjustments can be used in an attempt to mitigate biases caused by nonresponse and noncoverage, but tey are necessarily imperfect. Adjustments for a domain specific level of nonresponse require knowledge of te domain membersip of nonrespondents, but tat is often not available. Adjustments for noncoverage of a rare domain require accurate external data for te domain, data tat are often not available. Indeed, one of te purposes for some rare domain surveys is to estimate te domain size. Noncoverage is a major potential source of error in te estimation of domain size. 3.2 Disproportionate stratification A natural extension of te screening approac is to try to identify strata were te screening will be more productive. In te ideal circumstance, one or more strata tat cover all of te rare population and none from outside tat population are identified. Tat case requires no screening process. Oterwise, it is necessary to select samples from all te strata (apart from tose known to contain no rare population members) to ave complete coverage of te rare population. Te use of disproportionate stratification, wit iger sampling fractions in te strata were te prevalence of te rare population is iger, can reduce te amount of screening needed Teoretical background Consider initially a survey designed to provide estimates for a single rare population. Waksberg (1973) carried out an early teoretical assessment of te value of disproportionate stratification for tis case. Subsequent papers on tis topic include tose by Kalton and Anderson (1986) and Kalton (1993a, 2003). Te teoretical results sow tat tree main factors must be considered in determining te effectiveness of disproportionate stratification for sampling a single rare population: te prevalence rate in eac stratum, te proportion of te rare population in eac stratum, and te ratio of te full cost of data collection for members of te rare population to te screening cost involved in identifying members of tat population. If it is assumed tat (1) te element variances for te rare population are te same across strata and (2) te costs of data collection for members of te rare and non-rare populations are te same across strata, ten, wit simple random sampling witin strata, te optimum sampling fraction in stratum for minimizing te variance of an estimated mean for te rare population, subject to a fixed total budget, is given by f P P ( c 1) + 1, were P is te proportion of te units in stratum tat are members of te rare population and c is te ratio of te data collection cost for a sampled member of te rare population to te cost for a member of te non-rare population (Kalton 1993a). Te following formula provides te ratio of te variance of te sample mean wit te optimum disproportionate stratified sampling fractions to tat wit a proportionate stratified sample of te same total cost: R = ΣA P( c 1) + P / P P( c 1) + 1 were A is te proportion of te rare population in stratum and P is te prevalence of te rare population in te full population. In general, te variability in te optimum sampling fractions across te strata, and te gains in precision for te sample mean, decline as c increases. Tus, if te main survey data collection cost is ig as, for instance, wen te survey involves an expensive medical examination or if te screening cost is very low, ten disproportionate stratification may yield only minor gains in precision. Wen te main data collection cost adds noting to te screening cost, te ratio of main data collection cost to screening cost will be c = 1. In tis limiting situation, te formulas given above simplify to f P and R = 2 ( Σ A W ), were W is te proportion of te total population in stratum. Tese simple formulas provide a useful indication of te maximum variation in optimum sampling fractions and te maximum gains in precision tat can be acieved. Te square root function in te optimum sampling fraction formula makes clear tat te prevalences in te strata must vary a good deal if te sampling fractions are to differ appreciably from a proportionate allocation. For example, even if te prevalence in stratum A is four times as large as tat in stratum B, te optimum sampling fraction in stratum A is only twice as large as tat in stratum B. Te gains in precision (1 R) are large wen A is large wen W is small and vice versa. Wit only two strata, a stratum wit a prevalence five times as large as te overall prevalence (i.e., P / P = 5) will yield gains in precision of 25 percent or more ( (1 R) 0.25) only if tat stratum 2,

9 132 Kalton: Metods for oversampling rare subpopulations in social surveys includes at least 60 percent of te rare population (Kalton 2003, Table 1). In summary, wile generally useful, disproportionate stratification will yield substantial gains in efficiency only if tree conditions old: (1) te rare population must be muc more prevalent in te oversampled strata; (2) te oversampled strata must contain a ig proportion of te rare population; and (3) te cost of te main data collection per sampled unit must not be ig. In many cases, not all of tese tree conditions can be met, in wic case te gains will be modest. Furtermore, te results presented above are based on te assumption tat te true prevalence of te rare population in eac stratum is known, wereas in practice it will be out of date (for example, based on te last census) or will peraps simply ave been guesstimated. Errors in te prevalence estimates will reduce te precision gains acieved wit disproportionate stratification and could even result in a loss of precision. A major overestimation of te prevalence of te rare population, and ence of te optimum sampling fraction, in te ig-density stratum can result in a serious loss of precision for te survey estimates. It is terefore often preferable to adopt a conservative strategy, tat is, to adopt a somewat less disproportionate allocation, one tat moves in te direction of a proportionate allocation Applications Wen area sampling is used, data available from te last census and oter sources can be used to allocate te area clusters to strata based on teir prevalence estimates for te rare population. See Waksberg, Judkins and Massey (1997) for a detailed investigation of tis approac for oversampling various racial/etnic populations and te lowincome population using U.S. census blocks and block groups as clusters. Based on data from te 1990 Census, Waksberg and is colleagues found tat te approac generally worked well for Blacks and Hispanics but not for te low-income population. Wile te low-income population did exibit ig concentrations in some blocks and block groups, tose areas did not cover a ig proportion of tat population. Wen te survey designers ave access to a list frame wit names, te names can be used to construct strata of likely members of some racial/etnic groups. Tis situation arises, for instance, wit lists of names and telepone numbers and wen names are merged onto U.S. Postal Service (USPS) Delivery Sequence File addresses (no name merge is made in some cases). Te allocation to strata can be based on surnames only or on a combination of surname and first name (and even oter names also). Since women often adopt teir usbands surnames, te allocation is generally more effective for men tan women. Names can be reasonably effective for identifying Hispanics, Filipinos, Vietnamese, Japanese and Cinese, but not Blacks. A number of lists of names associated wit different racial/etnic groups ave been compiled, suc as te list of Spanis names compiled by te U.S. Census Bureau for te 1990s (Word and Perkins 1996). Several commercial vendors ave developed complex algoritms to perform racial/etnic classifications based on names (see Fiscella and Fremont 2006 for furter details). Te use of names in identifying race and etnicity as been of considerable interest to epidemiologists and demograpers, wo ave conducted a number of evaluations of tis metod (e.g., Lauderdale and Kestenbaum 2000; Elliott, Morrison, Fremont, McCaffrey, Pantoja and Lurie 2009). Tey often assess te effectiveness of te metod in terms of positive predictive value and sensitivity, wic are te equivalents of prevalence and te proportion of members of te domain wo are identified as suc by te instrument used for te classification. In te sampling context, besides limitations in te instrument, researcers also need to take into account tat sometimes names are not available and tat some available names may be incorrect (for example, wit address-based sampling, te names may be out-of-date, because te original family as moved out and a new family as moved into an address). Tese additional considerations serve to reduce te effectiveness of te name stratification, and depending on te particular circumstances, te reduction in effectiveness may be sizable. As wit stratification in general, te stratification factors used for sampling rare populations do not ave to be restricted to objective measures. Tey can equally be subjective classifications. Te only consideration is ow well tey serve te needs of te stratification (see Kis 1965b, pages , for an example of te effectiveness of te use of listers rapid classifications of dwellings into low, medium or ig socio-economic status for disproportionate stratification). Elliott, McCaffrey, Perlman, Marsall and Hambarsoomians (2009) describe an effective application of subjective stratification for sampling Cambodian immigrants in Long Beac, California. A local community expert rated all individual residences in sampled blocks as likely or unlikely to contain Cambodian ouseolds, based on externally observable cultural caracteristics suc as footwear outside te door and Buddist altars. Te residences allocated to te likely stratum (approximately 20 percent) were ten sampled at four times te rate tan te rest. Sometimes, wen te survey is concerned wit producing estimates only for a very rare population, disproportionate stratification may still require an excessive amount of screening. In tat circumstance, it may be necessary to sample from te strata were te prevalence is

10 Survey Metodology, December igest, dropping te oter strata and accepting some degree of noncoverage (or redefining te survey population to comprise only members of te rare population in te strata tat were sampled). Te Hispanic Healt and Nutrition Examination Survey of (HHANES) provides an illustration. For its samples of Mexican Americans in te Soutwest and Puerto Ricans in te New York City area, te HHANES sampled only from counties wit large numbers and/or percentages of Hispanics, based on 1980 Census counts (Gonzalez, Ezzati, Wite, Massey, Lago and Waksberg 1985). As anoter example of tis approac, Hedges (1979) describes a procedure for sampling a minority population tat is more concentrated in some geograpical districts, suc as census enumeration districts. In tis procedure, te districts are listed in order of teir prevalence of members of te rare population (obtained, say, from te last census), and ten te survey designers produce Lorenz curves of te cumulative distribution of rare population prevalence and te cumulative distribution of te proportions of rare population members covered. Wit te cumulative prevalence declining as te cumulative coverage increases, te survey designers can use tese distributions to select te combination of prevalence and proportion covered tat best fulfills teir requirements. Te issue ten to be faced is weter to make inferences to te covered population, or weter to make inferences to te full population by applying population weigting adjustments in an attempt to address te noncoverage bias. Wen a domain is very rare but a portion of it is eavily concentrated in a stratum, researcers sometimes sample tat stratum at a rate muc iger tan te optimum in order to generate a sizable number of cases. Altoug tis approac may produce a large sample of te rare population, te effective sample size (i.e., te sample size divided by te design effect) will be smaller tan if te optimum sampling fractions ad been used. Tus, from te perspective of te standard survey design-based mode of inference, tis approac is not appropriate. However, te researcers using tis approac often argue for a modelbased mode of inference in wic te sampling weigts are ignored. In my view, ignoring te sampling weigts is problematic. However, discussion of tis issue is outside te scope of tis paper. 3.3 Two-pase sampling Te screening approac treated in Sections 3.1 and 3.2 assumes tat identification of rare population members is relatively easy. Wen accurate identification is expensive, a two-pase design can be useful, starting wit an imperfect screening classification at te first pase, to be followed up wit accurate identification for a disproportionate stratified subsample at te second pase. Weter te two-pase approac is cost-effective depends in part on te relative costs of te imperfect classification and accurate identification: since te imperfect classifications use up some of te study s resources, tey must be muc less expensive tan te accurate identification. Deming (1977) suggests tat te ratio of te per-unit costs of te second- to te first-pase data collections sould be at least 6:1. Also, te imperfect classification must be reasonably effective in order to gain major benefits from a second-pase disproportionate stratification. Two- or even tree-pase sampling can often be useful in medical surveys of persons wit specific ealt conditions. Te first pase of te survey often consists of a screening questionnaire administered by survey interviewers, and te second pase is generally conducted by clinicians, often in a medical center. As one example, in a survey of epilepsy in Copia County, Mississippi, Haerer, Anderson and Scoenberg (1986) first ad survey interviewers administer to all ouseolds in te county a questionnaire tat ad been pretested to ensure tat it ad a ig level of sensitivity for detecting persons wit epilepsy. To avoid false negatives at tis first pase, a broad screening net was used in identifying persons wo would continue to te second pase. All tose so identified were te subjects for te second pase of te survey, wic consisted of brief neurological examinations conducted by a team of four senior neurologists in a public ealt clinic. A second example illustrates te use of anoter survey to serve as te first-pase data collection for studying a rare domain. In tis case, te Healt and Retirement Study (HRS) was used as te first pase for a study of dementia and oter cognitive impairment in adults aged 70 or older. Te HRS collects a wide range of measures on sample respondents, including a battery of cognitive measures. Using tese measures, te HRS respondents were allocated to five cognitive strata, wit a disproportionate stratified sample being selected for te second pase. Te expensive second-pase data collection consisted of a 3- to 4-our structured in-ome assessment by a nurse and neuropsycology tecnician. Te results of te assessment were ten evaluated by a geropsyciatrist, a neurologist and a cognitive neuroscientist to assign a preliminary diagnosis for cognitive status, wic was ten reassessed in te ligt of data in te person s medical records (Langa, Plassman, Wallace, et al. 2005). A tird example is a tree-pase design tat was used in a pilot study to identify persons wo would qualify for disability benefits from te U.S. Social Security Administration if tey were to apply for tem (Maffeo, Frey and Kalton 2000). At te first pase, a knowledgeable ouseold respondent was asked to provide information about te

11 134 Kalton: Metods for oversampling rare subpopulations in social surveys disability beneficiary status and impairment status of all adults aged 18 to 69 years in te ouseold. At te second pase, all tose classified into a stratum of severely disabled nonbeneficiaries and samples of te oter strata were interviewed in person and were ten reclassified as necessary into likely disability strata for te tird pase. At te tird pase, a disproportionate stratified sample of persons was selected to undergo medical examinations in mobile examination centers. A fairly common practice wit two-pase designs is to take no second- (or tird-) pase sample from te stratum of tose classified as nonmembers of te rare domain based on teir responses at te previous pase. Te proportion of te population in tat stratum is usually very ig, and te prevalence of te rare domain in it is very low (indeed, as in te Haerer, Anderson and Scoenberg (1986) study, te stratum is often conservatively defined wit te aim of avoiding te inclusion of tose wo migt possibly be members of te rare domain). As a result, a moderate-sized sample from tis stratum will yield almost no members of te rare domain. However, te cut-off strategy of taking no sample from tis stratum is risky. If te prevalence of te rare domain in tis large stratum is more tan minimal, a substantial proportion of te domain may go unrepresented in te final sample. 3.4 Multiple frames Sometimes sampling frames exist tat are more targeted on a rare population tan a general frame, but tey cover only part of te rare population. In tis situation, it can be efficient to select te sample from more tan one frame. For example, in te common case of oversampling etnic minorities, tere is sometimes a list frame available. Te persons on te list can be classified based on teir names as being likely to belong to a given etnic group (e.g., Cinese, Korean, Pacific Islanders, Vietnamese) to create a second, incomplete sampling frame from wic to sample, in addition to a more complete frame tat as a lower prevalence of te rare population (see, e.g., Elliott et al. 2008; Flores Cervantes and Kalton 2008). As wit disproportionate stratification (Section 3.2), major benefits derive from tis approac only wen te second frame as a ig prevalence and covers a sizable fraction of te rare population. See Lor (2009) for a review of te issues involved in sampling from multiple frames. Wit multiple frames, some members of te rare population may be included on several frames, in wic case tey may ave multiple routes of being selected into te sample. Tere are tree broad approaces for addressing tese multiplicities (Anderson and Kalton 1990; Kalton and Anderson 1986). Wen all te frames are list frames, as sometimes occurs in ealt studies, it may be possible to combine te frames into a single unduplicated list; owever, tis can often involve difficult record linkage problems. An alternative approac is to make te frames non-overlapping by using a unique identification rule tat associates eac member of te rare population wit only one of te frames, treating te listings on te oter frames as blanks (Kis 1965b, pages ). Samples are selected from eac of te frames witout regard to te duplication, but only te non-blank sampled listings are accepted for te final sample. Tis approac works best wen searces can be made for eac sampled unit on te oter frames; if te frames are put in a priority order and te unit is found on a prior frame to te one from wic te selection was made, te sampled listing would be treated as a blank. In tis case, te frames are strata; te sampled units are treated as subclasses witin te strata, allowing for te blank listings (Kis 1965b, pages ), and te analysis follows standard metods. Te use of te unique identification approac can, owever, be inefficient wen te persons sampled from one frame ave to be contacted to establis weter teir listings are to be treated as real or blank for tat frame. In tis case, it is generally more economical to collect te survey data for all sampled persons (i.e., to accept te multiple routes of selection). Tere are, owever, exceptions, as in te case of te National Survey of America s Families. Tat survey used a combination of an area frame and an RDD telepone frame, wit te area frame being used to cover only ouseolds witout telepones (Waksberg, Brick et al. 1997). It proved to be efficient to conduct a quick screening exercise wit ouseolds on te area frame to eliminate ouseolds wit telepones, retaining only te nontelepone ouseolds for te survey. Tere are two general approaces for taking multiple routes of selection into account in computing selection probabilities (Bankier 1986; Kalton and Anderson 1986). One metod calculates eac sampled unit s overall selection probability across all te frames and uses te inverse of tat probability as te base weigt for te analysis (leading to te Horvitz-Tompson estimator). For example, te overall selection probability for sampled unit i on two frames is pi= ( p1 i+ p2i p1 i p2i ) = [1 (1 p1 i )(1 p2i )], were p fi is te probability of te unit s selection from frame f = 1,2. A variant is to replace te overall selection probability wit te expected number of selections (leading to te Hansen-Hurwitz estimator), wic is easier to compute wen multiple frames are involved. Wit only two frames, te expected number of selections is ( p1 i + p2i ). Wen selection probabilities are small, tere is little difference between tese two estimators. Adjustments to compensate for nonresponse and to calibrate sample totals to known population totals can eiter be made to te overall selection probabilities p or tey can i