Determining Subsampling Rates for Nonrespondents

Transcription

1 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada Determiig Subsamplig Rates for Norespodets Rachel M. Harter, Traci L. Mach, Jaella F. Chaplie, Joh D. Wolke Natioal Opiio Research Ceter at the Uiversity of Chicago Board of Goverors of the Federal Reserve Abstract It is becomig icreasigly commo to subsample orespodets to icrease weighted respose rates, or to maitai respose rates while reducig costs. Respose rates ad costs are ot the oly factors affected by subsamplig, however. Subsamplig icreases variability i the weights, which i tur icreases the desig effect ad decreases the effective sample size. Subsamplig also may affect the umber of completed cases. At what rate should the orespodets be subsampled? If sufficiet iformatio is available from prior rouds or other sources, the decisio ca be made o the basis of costraits i terms of weighted respose rate, desig effect, cost, or the umber of completed cases. The costraits applied might deped o whether the subsamplig rate decisio is made prior to iitial sample selectio or i the midst of data collectio. We will review the literature, preset possible decisio rules, ad illustrate those rules with screeer data from the 003 Survey of Small Busiess Fiaces. Keywords: Subsamplig, Double Samplig, Two Phase Samplig, Sequetial Samplig, Norespose Bias. Itroductio Traditioally the survey desig challege has bee to make ifereces at a specified precisio level for miimal cost, or, coversely, to achieve the best possible precisio for a fixed cost. Decliig respose rates i recet years have added to the challege, icreasig the risk of orespose bias. Back i 938, Neyma proposed a sample desig that came to be kow as double samplig. I 946, Hase ad Hurwitz applied the idea to subsamplig orespodets for special hadlig to elicit respose. The iitial data collectio method, applied to the larger sample, was relatively iexpesive, while follow-up methods used for the subsample, though more effective, were also more expesive. For example, the iitial data collectio mode may be by mail, with i-perso iterviews attempted for a subsample of orespodets. Double samplig, also kow as two phase or sequetial samplig, as described i most stadard samplig textbooks, has a variety of differet uses. Cochra (977) ad Lohr (999) preset the method as a way to ) obtai valuable auxiliary data for ratio or regressio estimatio relatively iexpesively whe it is ot readily available for the whole populatio, or to ) stratify cases prior to ultimate sample selectio. Norespodet subsamplig is a special case of the latter i which the stratifyig iformatio is based o whether or ot cases respoded to a earlier relatively iexpesive mode of data collectio. All cases that respod iitially are assiged to the easy stratum i which all cases are sampled, ad orespodets are assiged to the difficult stratum for subsamplig. I some textbooks [Rao (000), Levy ad Lemeshow (99)], subsamplig of orespodets is the primary use of double samplig. Thompso (99) suggests double samplig of orespodets for callbacks to reduce orespose bias. As Hase, Hurwitz, ad Madow (953) poit out, double samplig combies the ecoomies of low cost data collectio i the first phase with the orespose bias reductio o the reduced subsample. Demig s textbook for busiess surveys (960) suggests double samplig i a example. A questio that aturally arises is at what rate should the orespodets be subsampled? I sectio two, we highlight specific studies that have subsampled orespodets, focusig o the subsamplig rates used ad, i some cases, the ratioal for usig them. I sectio three, we review the theory ad covetioal wisdom o the topic. I sectio four, we preset additioal ways of determiig acceptable subsamplig rates give various types of costraits. Fially, we use screeer data from the 003 Survey of Small Busiess Fiaces (SSBF) to illustrate these methods i sectio five.. Examples Existig surveys that employ double samplig use a variety of subsamplig rates, for a variety of reasos. Here are a few examples, some of which are described by Parker ad Dugoi (003)... America Commuity Survey (ACS) The America Commuity Survey (ACS) coducted by the U.S. Cesus Bureau cosists of three modes of data collectio i overlappig three-moth cycles. I the first moth of a cycle, the sample addresses determied to have a valid mailig address are set a questioaire The views expressed herei are those of the authors. They do ot ecessarily reflect the opiios of the Federal Reserve Board or its staff. 93

2 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada through the U.S. Postal Service. I the secod moth, telephoe cotact of all mail orespodets for which telephoe umbers are available is attempted. I the third moth, a subsample of the remaiig icomplete cases is selected for i-perso iterviewig. The phase subsamplig rates were -i-3 for uits without mailable addresses ad -i-3 otherwise [Tersie ad Starsiic (003)]. These rates were based o the tract-level expected rate of completed iterviews.. Chicago Health ad Social Life Survey (CHSLS) I the Chicago Health ad Social Life Survey (CHSLS), coducted by the NORC Populatio Research Ceter at the Uiversity of Chicago i 995 [NORC (996)], orespodets were subsampled whe the respose rates were lower tha origially expected. Although resultig i slightly fewer completes tha desired, by subsamplig -i-4 orespodets the weighted respose rate was boosted from 64 to 7 percet Natioal Survey of Family Growth (NSFG) The 006 Natioal Survey of Family Growth (NSFG) was coducted by the Uiversity of Michiga [ISR] for the Natioal Ceter for Health Statistics [Lepkowski et al. (006)]. The project team developed respose propesity models to predict whether each case would result i a completed screeer or iterview. Usig the models, the segmets (secodary samplig uits) were stratified o the umber of icompletes ad the respose propesity of the remaiig active cases. For each of these dimesios, segmets (clusters of households) were subsampled from high, medium-high, medium-low, ad low strata to reduce travel costs. Subsamplig rates varied by stratum to favor segmets likely to yield more completed cases for lower cost Geeral Social Survey (GSS) The most recet roud of the Geeral Social Survey (GSS), periodically coducted by NORC for the Natioal Sciece Foudatio ad other sposors, bega i 006. Built ito the desig, household level orespodets were subsampled at a rate of 45% across the board. While there was o uique best samplig rate, 45% was chose as providig a good balace betwee the uweighted umber of completes ad the weighted respose rates. 3. Review of Theory ad Geeral Priciples Accordig to Kish (965), double samplig works well whe the first phase cost of data collectio is dramatically less tha the secod phase cost. Otherwise, the extra complexities added to operatioalizig the desig ad calculatig weights may ot be worth the cost savigs. Kish rule of thumb states that secod phase data collectio should be at least te times that of the per-case first phase cost i order to be cosidered ecoomical. The implicit assumptio is that the secod phase data collectio approach is fudametally differet tha that of the first phase. I practice, sometimes subsamplig of orespodets is doe whe data collectio turs out to be more difficult or more costly tha aticipated. Perhaps data collectio must termiate early to stay withi budget or to boost a weighted respose rate. Applicatio of subsamplig methodology cotrols orespose bias much better tha simply termiatig data collectio whe the early, easier cases are complete. That is, rather tha beig a plaed desig feature, double samplig ca be used as a late-game approach to save the budget or weighted respose rate while attemptig to cotrol orespose bias. Though data collectio modes are ofte o differet betwee the two phases, it may just be that the secod phase cases are more aggressively pursued. Elliott et al. (000) suggest that subsamplig will save resources wheever ) the per-callback or per-iterview cost icreases, or ) the probability of obtaiig a successful iterview decreases, with each attempt. For relatively costat costs across attempts, cost savigs are likely to be small or, if the variace of the variable of iterest icreases with later iterviews, subsamplig is less desirable. O the other had, related decreases i variace makes subsamplig more effective. Several proposed subsamplig strategies follow. 3. Hase-Hurwitz Method The basic strategy as preseted by Haso ad Hurwitz (946) is to first determie the sample eeded to achieve the desired precisio level, assumig o orespose. The, if the cost structure ad expected respose rates for each phase are kow, solve for the iitial sample size ad subsamplig rate f that miimize cost subject to the desired precisio level. First, we preset some useful otatio. Let N = total umber of uits i the populatio = total umber of uits i the iitial sample = umber of first phase respodets = = umber of first phase orespodets * = umber of first phase orespodets selected for secod phase data collectio = umber of secod phase respodets * f = = subsamplig rate 94

3 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada r = = rate of respodig i first phase r = = rate of respodig i secod phase * y = value of survey variable Y for uit i i y = mea of survey variable Y amog first phase respodets y = mea of survey variable Y amog secod phase respodets σ = populatio variace of survey variable Y The double sample estimate of the populatio mea Y is Y ˆ = /. ( y y ) + Note that this estimator is the Hurvitz-Thompso estimator where the phase cases have a base weight that is the reciprocal of the iitial sample selectio probabilities, ad the phase cases have a weight i which the base weight is adjusted for subsamplig ad for orespose amog all phase cases. Suppose thatε is the desired variace level. The, uder simple radom samplig with o orespose, we ca solve for the effective sample size, ˆ, where Nσ ˆ = σ + ε N ( ) N. Now, if we wat the double samplig estimator to have the same variace, ε, we solve for the iitial sample size ad the subsamplig rate f, give the aticipated completio rates, r ad r. As poited out whe the authors derived the estimator, the umber of respodets eeded i each phase to achieve the desired precisio target is ot uique. However, suppose we kow i advace the per-uit cost structure, igorig fixed costs that do ot vary with sample size. The with costs give by C = c0+ c + c, where c = the per-uit cost for iitial cotact attempts 0 c = the per-uit costs for first phase completes, ad c = the per-uit costs for secod phase completes miimized for the desired variace level, the optimal iitial sample size is ( ) = ˆ + f Q, ad the optimal subsamplig rate is f c + cr cr 0 =, () where Q = Nr is the proportio of populatio uits that would ot have respoded i phase if a cesus had bee attempted with the same level of effort. I practice, the Haso-Hurwitz method is ot used much. Groves (989) discusses the problems with this method: ) the method takes ito accout samplig error oly, ot other sources of error; ) the completio rate i phase is assumed to be high; 3) mode effects betwee phases are igored; 4) o distictio is made betwee ocotacts ad refusals; 5) completio rates ad cost structures must be kow i advace. May of these same drawbacks are shared by other methods i the literature. 3. Demig Method Demig (953) poits out that orespose bias is serious eough i most surveys to warrat careful plaig of the umber of call attempts at the desig stage. The umber of call attempts must be balaced with cost issues, however. Demig s goal is to miimize cost for a specified mea squared error, or vice versa. I Demig s approach, all sample cases are attempted oce. The subsamplig occurs before the first callback attempt, ad oly the subsample is pursued for subsequet callback attempts. The Demig approach is to determie variace, orespose bias, ad cost, ad from that to determie the umber of callback attempts. The subsamplig rate is a by-product of this process. If we view phase of double samplig as the first attempt by Demig s method, ad treat phase as the first callback attempt, Demig s method with oe callback is the same as double samplig. The mea squared error of the estimator takes the form B C MSE = A + +, () f with a cost fuctio of the form Cost = D + Ef, the for ay iitial sample size, Demig s optimal subsamplig rate is CD f =. (3) BE Notice that the subsamplig rate depeds o the ratio of the costs D/E, ot their absolute idividual values. 95

4 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada The challege to usig this method is i defiig the MSE fuctio. The mea ad variace of the characteristic of iterest is assumed to differ for each attempt, which is why subsequet attempts reduce the orespose bias. However, the meas ad variaces for each attempt, or estimates of them, are eeded i advace for the MSE fuctio. As Groves (989) also otes, the method assumes cases are equally likely to respod o each attempt, which ofte is ot true. 3.3 Elliott-Little-Lewitzky Method Elliott et al. (000) preset a framework that allows for differet respose propesities for each callback attempt with give ozero costs related to refusals. They propose a efficiecy ratio as the cost uder the subsamplig approach to the cost uder the full-callback approach, assumig equal variaces. Suppose subsamplig is to occur at the m th callback attempt, ad subsampled cases will be subjected to a fixed umber of subsequet callback attempts. The strategy is to determie the value of m with a subsamplig rate f that sufficietly miimizes the efficiecy ratio uder. I geeral, m is typically a relatively early or middle callback attempt for which the proportio of remaiig iterviews eeded is ot large ad the proportio of expected costs remaiig is small. 4. Subsamplig Rates for Alterative Costraits Although most survey desigs have precisio requiremets for desired aalyses at their core, the requiremets may be expressed cotractually as sample size or respose rate requiremets. Cost may ot be a issue to the sposorig agecy with a fixed price cotract, while it is very much a issue to the cotractor; i cotrast, cost may be more of a issue to the sposorig agecy tha the cotractor uder a cost reimbursable-type cotract. I this sectio we look at alterative costraits ad the impact o subsamplig rates. 4. Required Completes As discussed earlier, subsamplig of orespodets may be part of the iitial desig, or it may be a ed game strategy to cotrol costs after the iitial sample has bee selected. I the latter case, there are fewer opportuities to optimize. Suppose, for example, that the umber of completed cases is specified i advace. Suppose, further, that the iitial sample size has already bee fixed. I the Hase-Hurwitz sceario where completio rates are fixed, the subsamplig rate is completely determied by the relatioship spec ( ) = r + r fr, (4) where is the specified umber of completes. spec Uder Hase-Hurwitz assumptios, meetig a sample size requiremet optimally must be cosidered at the desig stage (or for later batches usig parameters determied i the early batches). To miimize cost subject to the sample size costrait, we ote that the cost fuctio is liear i. The subsamplig rate f is determied by as show i (4) above, but we have the added costraits that 0 f, which determies a rage for possible values of. If c > c, the the cost is spec miimized whe is at the maximum of its rage,, r ad the subsamplig rate f is zero. I other words, if cost is the oly cosideratio, ot respose rate, ad if phase cases are less expesive tha phase cases, the simply elarge the iitial sample to complete spec cases i the first phase; subsamplig should ot be ecessary. O the other had, if c < c, the the total cost is miimized spec whe is at the miimum of its rage,, ad r+ ( r) r the subsamplig rate f is. That is, select to be the target umber of completes,, divided by the spec weighted respose rate (equatio 5 below) ad retai all first phase orespodets for secod phase data collectio. Suppose c = c, the ay value of withi this rage will do, provided f is determied by (4) above. 4. Respose Rates Costrait If the cotractual requiremet is to achieve a specified (weighted) respose rate, the i the Hase-Hurwitz sceario, the subsamplig rate is irrelevat. It ca be show that the weighted respose rate is a fuctio of r ad r oly: ( ) Weighted respose rate = r + r r. (5) From this equatio it ca be see that the weighted respose rate from a double sample will be larger tha r, the respose rate without subsamplig. If r is large or r is small, the subsamplig is likely to help oly a little. The biggest gais from subsamplig occur whe r is small adr is large. The subsamplig rate affects the respose rate oly if the samplig rate chages the fuds available for the subsample ad, i tur, the methodological approach ad completio rate for the subsample. That is, the subsamplig rate affects the overall weighted respose rate oly if the subsamplig rate affects the value of r. Uder Demig s method, for example, the subsamplig rate is tied to the umber of callback attempts. Uder the 96

5 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada Elliott-Little-Lewitzky method, if we thik of all call attempts prior to the m th attempt as phase, ad all subsequet call attempts up to K as phase, the the subsamplig rate, which is tied to cost ad the value of m, is also related to r adr because the completio rates deped o the umber of attempts i each phase. The challege is to have effective models relatig cost ad respose propesity. 4.3 Weightig Effect, Sample Size, Cost Costraits Subsamplig orespodets itroduces more variability i the weights, which icreases the desig effect ad reduces the effective sample size. Suppose the goal is to reduce costs by a certai percetage while keepig the weightig effect below a certai level (or maitaiig a effective sample size above a certai level). A two-step procedure is described here that takes both costraits ito accout. This method allows for multiple types of outcomes ad differet costs associated with each type of outcome. Suppose there are J possible fial outcomes ad that there may be differet costs associated with each outcome type. I this paper, we will assume four types of outcomes, each with a differet associated cost: ) phase complete, ) phase fial orespodet (e.g., hostile refusal or ocotact), 3) phase complete, ad 4) phase orespodet. Other outcomes are possible, such as ieligible, or dealig with ocotacts ad hostile refusals separately, but we use these four categories to simplify the algebra. The cost fuctio without subsamplig is give by J C = j = c p, (6) j j where p j is the proportio of cases from the iitial sample of that have outcome j, ad c j is the per-case cost associated with that outcome. This cost fuctio assumes all cases get full treatmet through phase if ot already fialized i phase. (Note that if the oly cases fialized i the first phase were the completes, the we would have p = r, p = 0, ad p3 = ( r ) r, leadig us back to the familiar Hase-Hurwitz otatio.) Now suppose that some outcomes i the first phase are subsampled before phase treatmet, ad all such classes are subsampled at the same rate f. The total umber of completes will be reduced uless we icrease the iitial sample size by a factor of M = ( p + p3) /( p + fp3), (7) the umber of completes with full treatmet ad o subsamplig divided by the umber of completes with subsamplig. At the same time, we wat the total cost to be reduced by, say, a factor of X. So the cost of the subsampled desig divided by the cost of the full treatmet desig should be less tha or equal to X. That is, C X, (8) C where [( c p + c p ) + f ( c p c )] C = +. M p4 The iequality above ca be solved for a iequality i f, determiig a acceptable rage of f that meets the cost costrait. The secod step is to fid a acceptable rage of subsamplig rates that does ot icrease the weightig effect beyod, say, W. To make thigs simpler, we assume that all cases have equal base weights, ad for all practical purposes, we ca assume that the base weights are scaled to. The weightig effect is approximated by plus the relative variace of the weights. We assume here that the phase completed cases retai the base weight, ad that the weights for phase completed cases are adjusted for subsamplig. Deviatig from the Hase- Hurwitz scheme, we assume that all completes are adjusted for orespose; further, we make the simplifyig assumptio that all completes get the same orespose adjustmet factor. The, the fial weights for phase completes are /( p + p3) ad for the phase completes are / [ f ( p + p3) ]. It ca be show that the mea weight is /( p + fp3 ), ad that the weightig effect is WEFF = + ( p p / f ) f p + p. (9) [( ) ( )] 3 / The fact that we icrease the iitial sample size by a factor to compesate for subsamplig does ot affect the relative variace of the weights. So, applyig a costrait o the weightig effect, we set WEFF W ad solve for f. The equality is a quadratic fuctio of f with two possible solutios that defie a rage of subsamplig rates f that meet the costrait o the weightig effect. It is ot ecessarily the case that either or both solutios to the quadratic equatio will be betwee zero ad oe. Values of f betwee zero ad oe are acceptable subsamplig rates. This rage ca the be combied with the acceptable rage for the cost reductio. Note that it is possible that o value of f or that a rage of values satisfies both costraits. 3 97

6 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada 5.0 Comparig Mai ad Followup Sample Resposes The 003 Survey of Small Busiess Fiaces (SSBF) was established to collect iformatio from the owers of a atioally represetative sample of up to 5,000 busiess eterprises. Data were gathered from small busiesses cocerig their fiacial relatioships, credit experieces, ledig terms ad coditios, icome ad balace sheet iformatio, the locatio ad types of fiacial istitutios used, ad other firm characteristics. Targeted were headquarter locatios of ogovermetal, odepository, oagricultural for-profit busiesses with fewer tha 500 employees. These firms also had to be i busiess i December 003 uder oe or more of their curret owers as well as at the time of data collectio. A stratified systematic sample of 37,600 busiesses was selected from the Du s Market Idetifiers (DMI) file, a busiess database maitaied by the Du & Bradstreet Corporatio (D&B). The 7 strata were defied by the cross-classificatio of busiess size, cesus divisio, ad urba/rural status. Data collectio had two stages: screeig ad mai iterview. The screeig was desiged to verify the ame of the busiess ower ad the physical address of the busiess, scree the busiess for eligibility to participate i the mai iterview, idetify the busiess s legal form of orgaizatio, ad record the fiscal year ed date. All sample firms were eligible for the screeer. The iitial cotact with respodets was a mailig cotaiig materials explaiig the purpose of the survey ad ecouragig participatio. Withi a few days of receivig the mailig, busiesses were called ad asked to complete the screeer. The sample was ultimately fielded i four batches, with each batch subject to screeig followed by mai study iterviewig of eligible firms. For both screeig ad mai iterviewig of the first three batches, attempted cotact was made with all firms durig a first phase. For the secod phase, orespodets were subsampled ad recotact attempts were made. Each phase cosisted of a series of call attempts, where the attempts followed a predetermied schedule desiged to icrease the likelihood of cotact util iterviews or appoitmets were achieved. Batch 4 was purposefully added late i the study to compesate for lower tha aticipated respose rates. There was o subsamplig for phase of batch 4. I this paper we illustrate the various ways of determiig subsamplig rates by usig the 5,666 screeer iterviews from batch of the study. Batch was selected because the first ad secod phases were relatively smooth ad free of last-miute tikerig (expaded time periods or icreased icetives) to boost respose rates. The discussio here is also simplified due to the fact that all sample firms were eligible for screeig. Oce the call desig for the phase screeig was complete, mai study eligibility had bee determied for r = 50% of the Batch cases; i.e. =,838. Screeer orespodets eligible for subsamplig were idetified from the remaiig,88 cases. Busiesses with which o huma cotact was ever established durig phase ad fialized orespodets, such as laguage barriers ad hostile refusals were excluded from phase subsamplig. Usig a subsamplig rate * of f = 60%, =,099 of the,98 cases eligible for phase subsamplig were selected. This phase yielded = 359 completes with r = 33%. Follow-up for ocotacts was a separate process altogether. If the screeig iterview were coducted today, the portio of the costs that deped o sample size, as i Hase ad Hurwitz, ca be summarized as Cost = $.98 + $ $6.03, where c 0 = $.98 per case (=5,666) costs for materials ad labor related to the advaced mailout c = $7.48 per case ( =,838) costs for phoe charges ad callig ceter labor (usig regular iterviewers) c = $6.03 per case ( = 359 ) costs for phoe charges ad callig ceter labor (usig experieced refusal coverters) 5. Hase-Hurwitz Method, Miimize Cost If we assume that =5,666 had bee obtaied accordig to the Hase-Hurwitz approach to achieve the variace requiremet, the the subsamplig rate as determied by equatio () would be 86.4%. 5. Hase-Hurwitz Method, Required Completes If our per-uit first ad secod costs are equal ad a specified umber of completed screeers are required, the the subsamplig rate should be determied by equatio (4). Figure shows subsamplig rates for various values of required completed screeers assumig a iitial sample size of 5,666 cases, with first ad secod phase completio rates equal 50% ad 33%, respectively. So, for example if we were hopig for 300 completed screeers, which was our actual cout, a subsamplig rate of 40% would be idicated. Agai usig the above assumptios for completio rates, if the goal had bee to miimize costs ad meet a 98

7 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada specified umber of completes without regard to variace cosideratios or respose rates, the we would have bee better off takig a larger iitial sample ad ot subsamplig at all. I fact, this is idicated because c >. c Figure : Phase Subsamplig Rates for Sample Size Requiremets 5.3 Weightig Effect, Sample Size, ad Cost Suppose our goal had bee to reduce costs at least 0% ad icrease the weightig effect by o more tha 0%. For this illustratio we assume that all cases had equal base weights, ad that the weightig effect is solely attributed to subsamplig ad orespose adjustmets. The Hase-Hurwitz cost structure above is ot ideal for this setup sice costs associated with orespodets have bee trasferred to the screeer completes. Istead, we will assume the followig hypothetical costs associated with each outcome: c = $5, c = $, c 3 = $5, c = $0. 4 If there had bee o subsamplig, we estimate the proportios of the sample we would have observed i each outcome to be p =.50, p =., p =.3, 3 p = Required Completes With subsamplig, we icrease the iitial sample by M as Cost defied i (7), ad we defie the cost ratio.8 Cost usig equatio (8). So, solvig the iequality, we have f That is, ay subsamplig rate less tha or equal to 79% will reduce costs by at least 0%, eve with the icreased iitial sample size. Now we solve (9) for values of f that satisfy the costrait that the WEFF must be o more tha.. The two values of f that solve the equality are.466 ad.44. Sice f is restricted to be betwee 0 ad, the values of f that meet the WEFF criterio are betwee.47 ad. Combiig this with the cost criterio, we see that f must be i the rage [.47,.79] to meet both costraits. Icidetally, if we had applied the orespose adjustmets to oly the phase completes i defiig (9), it is ulikely there would have bee values of f less tha that kept the WEFF withi rage, give that r is substatially larger tha r. 5.4 Demig Method The cost fuctio for the Hase-Hurwitz method above ca be rewritte algebraically ito Demig s structure as Cost = f. The ratio that cotributes to the subsamplig rate is =. The other ratio that cotributes to the subsamplig rate, C comes from the MSE fuctio i (). B The subsamplig rate is the square root of the product of these ratios as i equatio (3). Figure shows differet subsamplig rates for various values of the ratio C. B MSE Ratio C/B Figure : Subsamplig Rates for Various Ratios from the MSE Fuctio 6. Discussio The illustratios i the previous sectio are limited by the productio data retaied from the 003 SSBF. It would be good to track ad retai completio rates ad costs at 99

8 Papers preseted at the ICES-III, Jue 8-, 007, Motreal, Quebec, Caada each attempt, eablig us to try to quatify the impact of chagig the attempt at which subsamplig is performed as well as the subsamplig rate as i the Elliott-Little- Lewitzky method. The subtleties of subsamplig orespodets are i the iteractios betwee subsamplig ad redeployig the fuds to boost respose rates for the subsampled cases. Most of the methods preseted here are too simple to capture these subtleties. There are other methods for dealig with orespose ad orespose bias that are similar to the methods preseted here without actually subsamplig orespodets. The focus here, however, is o subsamplig orespodets ad determiig the rate at which to subsample. Several of the methods preseted are variatios o the traditioal cost/error models that lead to optimal solutios. Fellegi ad Suter (974) ad Groves (989) discuss the limitatios of cost/error models. The short summary is that the solutios are optimal oly i a very arrow sese. Furthermore, oe must take ito accout the variace ad bias resultig from the methods used [Oh ad Scheure (983)]. For the practitioer, the methods described here are iteded as guidelies for establishig a rage of reasoable optios for determiig subsamplig rates. The choice of method(s) will deped o the iformatio available ad the study costraits. Refereces Cochra, W.G. (977). Samplig Techiques, 3 rd Editio. New York: Joh Wiley & Sos. Demig, W.E. (953). O a Probability Mechaism to Attai a Ecoomic Balace Betwee the Resultat Error of Respose ad the Bias of Norespose. Joural of the America Statistical Associatio, 48 (64), pp Demig, W.E. (960). Sample Desig i Busiess Research. New York: Joh Wiley & Sos. Elliott, M.R., R.J.A. Little, ad S. Lewitzky (000). Subsamplig Callbacks to Improve Survey Efficiecy. Joural of the America Statistical Associatio, 95 (45), pp Fellegi, I.P ad A.B. Suter (974). Balace Betwee Differet Sources of Survey Errors Some Caadia Experieces. Sakhya, Series C, 36, Part 3, pp.9-4. Groves, R.M. (989). Survey Errors ad Survey Costs. New York: Joh Wiley & Sos. Hase, M.H., W.N. Hurwitz, ad W.G. Madow (953). Sample Survey Methods ad Theory, Vol. I: Methods ad Applicatios. New York: Joh Wiley & Sos. Kish, L. (965). Survey Samplig. New York: Joh Wiley & Sos. Lepkowski, J.M., W.D. Mosher, K.E. Davis et al. (006). Natioal Survey of Family Growth, Cycle 6: Sample desig, weightig, imputatio, ad variace estimatio. Natioal Ceter for Health Statistics, Vital Health Stat (4). Levy, P.S. ad S. Lemeshow (99). Samplig of Populatios: Methods ad Applicatios. New York: Joh Wiley & Sos. Lohr, S.L. (999). Samplig: Desig ad Aalysis. Pacific Grove: Duxbury Press. Neyma, J. (938). Cotributio to the Theory of Samplig Huma Populatios, Joural of the America Statistical Associatio, 33, pp Natioal Opiio Research Ceter (NORC) (996). Samplig Desig for the CHSLS. Iteral NORC techical paper. Oh, H.L. ad F.J. Scheure (983). Weightig Adjustmet for Uit Norespose, i Icomplete Data i Sample Surveys, Vol., Theory ad Bibliographies, Ed. By Madow, W.G., I. Olkig, ad D.B. Rubi. New York: Academic Press. Parker, E. ad B. Dugoi (003). Subsamplig for Effective Survey Maagemet: Review of Established Practices. Paper preseted to the Geeral Social Survey Board of Overseers. Rao, P.S.R.S. (000). Samplig Methodologies with Applicatios. Boca Rato: Chapma& Hall/CRC. Tersie, A. ad M. Starsiic (003). Optimum Norespose Subsamplig Rate for the America Commuity Survey. ASA Proceedigs of the Joit Statistical Meetigs, pp Thompso, S.K. (99). Samplig. New York: Joh Wiley & Sos. Hase, M.H. ad W.N. Hurwitz (946). The Problem of No-Respose i Sample Surveys. Joural of the America Statistical Associatio, 4 (36), pp