Eight Survey Rules of Thumb

Transcription

1 Eight Survey Rules of Thumb Fritz Scheuren NORC Eight Survey Rules of Thumb p. 1/3

2 Outline of Presentation Eight Survey Rules of Thumb p. 2/3

3 Outline of Presentation This Brief Background Eight Survey Rules of Thumb p. 2/3

4 Outline of Presentation This Brief Background About Rules of Thumb Eight Survey Rules of Thumb p. 2/3

5 Outline of Presentation This Brief Background About Rules of Thumb Why These 8? Eight Survey Rules of Thumb p. 2/3

6 Outline of Presentation This Brief Background About Rules of Thumb Why These 8? Each Rule Covered Quickly Eight Survey Rules of Thumb p. 2/3

7 Outline of Presentation This Brief Background About Rules of Thumb Why These 8? Each Rule Covered Quickly Some Concluding Remarks Eight Survey Rules of Thumb p. 2/3

8 Outline of Presentation This Brief Background About Rules of Thumb Why These 8? Each Rule Covered Quickly Some Concluding Remarks About Overstatements Eight Survey Rules of Thumb p. 2/3

9 Outline of Presentation This Brief Background About Rules of Thumb Why These 8? Each Rule Covered Quickly Some Concluding Remarks About Overstatements Challenge to Practitioners Eight Survey Rules of Thumb p. 2/3

10 Outline of Presentation This Brief Background About Rules of Thumb Why These 8? Each Rule Covered Quickly Some Concluding Remarks About Overstatements Challenge to Practitioners The Good Stuff Eight Survey Rules of Thumb p. 2/3

11 Background Reminders Eight Survey Rules of Thumb p. 3/3

12 Background Reminders Original Sampling Paradigm Eight Survey Rules of Thumb p. 3/3

13 Background Reminders Original Sampling Paradigm Taught by Example Eight Survey Rules of Thumb p. 3/3

14 Background Reminders Original Sampling Paradigm Taught by Example Written Down in 40s thru 60s Eight Survey Rules of Thumb p. 3/3

15 Background Reminders Original Sampling Paradigm Taught by Example Written Down in 40s thru 60s Slow Refinements 70s thru 90s Eight Survey Rules of Thumb p. 3/3

16 Background Reminders Original Sampling Paradigm Taught by Example Written Down in 40s thru 60s Slow Refinements 70s thru 90s Growing Dissatisfaction too Eight Survey Rules of Thumb p. 3/3

17 Background Reminders Original Sampling Paradigm Taught by Example Written Down in 40s thru 60s Slow Refinements 70s thru 90s Growing Dissatisfaction too Consolidate and Move on? Eight Survey Rules of Thumb p. 3/3

18 Background Reminders Original Sampling Paradigm Taught by Example Written Down in 40s thru 60s Slow Refinements 70s thru 90s Growing Dissatisfaction too Consolidate and Move on? Biemer- Lyberg as Example Eight Survey Rules of Thumb p. 3/3

19 About Rules of Thumb Eight Survey Rules of Thumb p. 4/3

20 About Rules of Thumb Valuable Starting Points Eight Survey Rules of Thumb p. 4/3

21 About Rules of Thumb Valuable Starting Points Useful As Historical Record Eight Survey Rules of Thumb p. 4/3

22 About Rules of Thumb Valuable Starting Points Useful As Historical Record Form a Checklist Eight Survey Rules of Thumb p. 4/3

23 About Rules of Thumb Valuable Starting Points Useful As Historical Record Form a Checklist Needed as Mnemonics? Eight Survey Rules of Thumb p. 4/3

24 About Rules of Thumb Valuable Starting Points Useful As Historical Record Form a Checklist Needed as Mnemonics? Reminders of Past Workarounds Eight Survey Rules of Thumb p. 4/3

25 About Rules of Thumb Valuable Starting Points Useful As Historical Record Form a Checklist Needed as Mnemonics? Reminders of Past Workarounds A Community Project? Eight Survey Rules of Thumb p. 4/3

26 Why These 8? Eight Survey Rules of Thumb p. 5/3

27 Why These 8? Eight is Lucky? Eight Survey Rules of Thumb p. 5/3

28 Why These 8? Eight is Lucky? Ok, No Good Reason! Eight Survey Rules of Thumb p. 5/3

29 Why These 8? Eight is Lucky? Ok, No Good Reason! Sampling v. Nonsampling Tradeoffs Eight Survey Rules of Thumb p. 5/3

30 Why These 8? Eight is Lucky? Ok, No Good Reason! Sampling v. Nonsampling Tradeoffs Model Based v. Design Based? Eight Survey Rules of Thumb p. 5/3

31 Why These 8? Eight is Lucky? Ok, No Good Reason! Sampling v. Nonsampling Tradeoffs Model Based v. Design Based? To Talk about Deming Eight Survey Rules of Thumb p. 5/3

32 Why These 8? Eight is Lucky? Ok, No Good Reason! Sampling v. Nonsampling Tradeoffs Model Based v. Design Based? To Talk about Deming Eight Survey Rules of Thumb p. 5/3

33 Notes on Nonresponse Presentation In the slides that follow much has been left unwritten because it was intended to be said. Those spoken words are not included here, although not to be too cryptic, I have added a few notes to hint at what I was trying to say. Eight Survey Rules of Thumb p. 6/3

34 Notes on Nonresponse Presentation First, the (obviously vast) topic of nonresponse has been confined to unit nonresponse. No mention is made of item nonresponse, for example. Arguably many points carry over nonetheless. Eight Survey Rules of Thumb p. 7/3

35 Notes on Nonresponse Presentation Second, the topic has been treated from an historical perspective, following the approach I have been taking in my History Corner series in The American Statistician (TAS). In this connection the November 2004 TAS issue may be of particular value, as it brings out some of my points in more detail. Eight Survey Rules of Thumb p. 8/3

36 Notes on Nonresponse Presentation Third, the formulas provided are more LOGOS that prescriptions to be used in real life applications. They were part of the arguments that the original framers of these ideas used but are oversimplified. For example, most of these ideas would be imbedded in pre or post strata to give them more plausibility and content. Eight Survey Rules of Thumb p. 9/3

37 Notes on Nonresponse Presentation Fourth, in some cases the formula has been associated in the slides with their author. But this has not always been done so each of authors of these formulas or rules has been given mention below. Eight Survey Rules of Thumb p. 10/3

38 Notes on Nonresponse Presentation Fifth, the bias/variance tradeoff slide (No.1) was taken from the introductory chapters of the HHM (Hansen, Hurwitz and Madow) and Cochran sampling texts. Their result has been repeated here to introduce how our professional (over) emphasis on sampling was sold. Those texts carved out the sampling role from the much larger data collection issues - all too successfully it might be added. Eight Survey Rules of Thumb p. 11/3

39 Notes on Nonresponse Presentation Sixth, the response (and implicitly nonresponse) bias factors formula (No.2), original with me, was an attempt to succinctly indicate how wrongly our practice continues to overemphasize sampling, even 50+ years later. We need to add significantly to our records the extensive paradata that is routinely assembled in our surveys and bring it forward to the client and to future survey developers. Eight Survey Rules of Thumb p. 12/3

40 Notes on Nonresponse Presentation Seventh, the missingness mixture slides (Nos. 3 to 5) are my interpretation of Rubin s seminal work on missingness (Biometrika 1976), where the emphasis is on the ubiquity with which all forms of missing unit nonresponse existing simultaneously in nearly all practical settings. Our failure to act on this knowledge is a weakness in our practice. Eight Survey Rules of Thumb p. 13/3

41 Notes on Nonresponse Presentation Eighth, the Hansen-Hurwitz subsampling idea for nonresponse is discussed next (No.6). Here the attempt is to illustrate how we could reinterpret their work using Rubin s idea of the multiple forms of missingness and, thereby, reduce the variance penalty if we have sufficient confidence in our missingness model. The interview mode change made as part of the original idea is sometimes lost and I regret that I did not say more about it last week. Eight Survey Rules of Thumb p. 14/3

42 Notes on Nonresponse Presentation Ninth, the Sekar-Deming capture/recapture application to nonresponse (No. 7) concludes the talk, except for some discussion of implications. Since these ideas are familiar in their dual systems incarnation, there is less to say here that elsewhere. Eight Survey Rules of Thumb p. 15/3

43 Notes on Nonresponse Presentation Tenth, the concluding discussion slides were just my take on our history Many good suggestions were made for better terminology by those present. Several objected to Rubin s use of the phrase "Missing at Random," abbreviated as MAR. Why not say, instead, they argued, that if we have the right variables present (say in the frame), then we can condition on them and thus eliminate some of the bias. Eight Survey Rules of Thumb p. 16/3

44 Eight Rules or Equations Eight Survey Rules of Thumb p. 17/3

45 Eight Rules or Equations 1. Bias versus Sampling Error Eight Survey Rules of Thumb p. 17/3

46 Eight Rules or Equations 1. Bias versus Sampling Error 2. Response Bias Factors Eight Survey Rules of Thumb p. 17/3

47 Eight Rules or Equations 1. Bias versus Sampling Error 2. Response Bias Factors 3. Mixtures of Missingness Eight Survey Rules of Thumb p. 17/3

48 Eight Rules or Equations 1. Bias versus Sampling Error 2. Response Bias Factors 3. Mixtures of Missingness 4. Data Collection Missingness Eight Survey Rules of Thumb p. 17/3

49 Eight Rules or Equations 1. Bias versus Sampling Error 2. Response Bias Factors 3. Mixtures of Missingness 4. Data Collection Missingness 5. Data Analysis Missingness Eight Survey Rules of Thumb p. 17/3

50 Eight Rules or Equations 1. Bias versus Sampling Error 2. Response Bias Factors 3. Mixtures of Missingness 4. Data Collection Missingness 5. Data Analysis Missingness 6. Subsamples for Nonresponse Eight Survey Rules of Thumb p. 17/3

51 Eight Rules or Equations 1. Bias versus Sampling Error 2. Response Bias Factors 3. Mixtures of Missingness 4. Data Collection Missingness 5. Data Analysis Missingness 6. Subsamples for Nonresponse 7. Bias or Hard-Core Non-Response Rates Eight Survey Rules of Thumb p. 17/3

52 Eight Rules or Equations 1. Bias versus Sampling Error 2. Response Bias Factors 3. Mixtures of Missingness 4. Data Collection Missingness 5. Data Analysis Missingness 6. Subsamples for Nonresponse 7. Bias or Hard-Core Non-Response Rates 8. Robust confidence interval estimation Eight Survey Rules of Thumb p. 17/3

53 1. Bias versus Sampling Error Eight Survey Rules of Thumb p. 18/3

54 1. Bias versus Sampling Error σ 2 MSE[f(Y )] = n + B2 σ 2 = n (1 + R2 ) (e.g., as shown in the Hansen, Hurwitz and Madow text.) Eight Survey Rules of Thumb p. 18/3

55 2. Survey Bias Factors f(y,m,r,c,d)dm dr dc = g(y,d) Implicitly, this formal integration symbolizes that final survey data files usually treat process fixes as complete. Leaving out the paradata concerning these flaws continues in a way the historical theme in slide No. 1. Eight Survey Rules of Thumb p. 19/3

56 3. Mixtures of Missingness In general m = m MCAR + m MAR + m NMAR can become specialized at various survey stages Eight Survey Rules of Thumb p. 20/3

57 4. Data Collection Missingness Eight Survey Rules of Thumb p. 21/3

58 4. Data Collection Missingness m = m MCAR + m MAR + m NMAR Usually becomes m = m NMAR! Implicitly, given what is done Eight Survey Rules of Thumb p. 21/3

59 4. Data Collection Missingness Some Alternatives Eight Survey Rules of Thumb p. 22/3

60 4. Data Collection Missingness Some Alternatives Subsampling Solutions, m = m NMAR Eight Survey Rules of Thumb p. 22/3

61 4. Data Collection Missingness Some Alternatives Subsampling Solutions, m = m NMAR Previous Experience, See Example Eight Survey Rules of Thumb p. 22/3

62 4. Data Collection Missingness Some Alternatives Subsampling Solutions, m = m NMAR Previous Experience, See Example Adaptive Approach, Decide on Fly Eight Survey Rules of Thumb p. 22/3

63 4. Data Collection Missingness Some Alternatives Subsampling Solutions, m = m NMAR Previous Experience, See Example Adaptive Approach, Decide on Fly Enrich the Frame, Make More m = m MAR Eight Survey Rules of Thumb p. 22/3

64 4. Data Collection Missingness Some Alternatives Subsampling Solutions, m = m NMAR Previous Experience, See Example Adaptive Approach, Decide on Fly Enrich the Frame, Make More m = m MAR Use a Composite Eight Survey Rules of Thumb p. 22/3

65 5. Data Analysis Missingness Eight Survey Rules of Thumb p. 23/3

66 5. Data Analysis Missingness m = m MCAR + m MAR + m NMAR Usually becomes m = m MAR! Again, given what is done Eight Survey Rules of Thumb p. 23/3

67 5. Data Analysis Missingness Some Alternatives Eight Survey Rules of Thumb p. 24/3

68 5. Data Analysis Missingness Some Alternatives Noncontact Nonresponse m = m NMAR? Eight Survey Rules of Thumb p. 24/3

69 5. Data Analysis Missingness Some Alternatives Noncontact Nonresponse m = m NMAR? Refused Nonresponse m = m MAR! Eight Survey Rules of Thumb p. 24/3

70 5. Data Analysis Missingness Some Alternatives Noncontact Nonresponse m = m NMAR? Refused Nonresponse m = m MAR! Adaptively Balanced Adjustment Eight Survey Rules of Thumb p. 24/3

71 5. Data Analysis Missingness Some Alternatives Noncontact Nonresponse m = m NMAR? Refused Nonresponse m = m MAR! Adaptively Balanced Adjustment Sensitivity Analysis Eight Survey Rules of Thumb p. 24/3

72 5. Data Analysis Missingness Some Alternatives Noncontact Nonresponse m = m NMAR? Refused Nonresponse m = m MAR! Adaptively Balanced Adjustment Sensitivity Analysis Weight Trimming Eight Survey Rules of Thumb p. 24/3

73 6. Hansen-Hurwitz Formula...But Reinterpreted Eight Survey Rules of Thumb p. 25/3

74 6. Hansen-Hurwitz Formula...But Reinterpreted ȳ = p R ȳ R + p M ȳ M where p R = "responding" fraction of the original sample p M = remaining "non-responding" fraction of the original sample Eight Survey Rules of Thumb p. 25/3

75 6. Hansen-Hurwitz Formula...But Reinterpreted (cont d) Eight Survey Rules of Thumb p. 26/3

76 6. Hansen-Hurwitz Formula...But Reinterpreted (cont d) s 2 M s 2 R ˆv(ˆȳ) p R n + p M vn + 1 [ pr (ȳ R ˆȳ) 2 + p M (ȳ M ˆȳ) 2] n 1 where v = sub-sampling rate and other terms, e.g., s 2 R, are just the standard notation Eight Survey Rules of Thumb p. 26/3

77 6. Subsamples for Nonresponse Mixture Example Eight Survey Rules of Thumb p. 27/3

78 6. Subsamples for Nonresponse Mixture Example Mainly Subsample m N Noncontacts Eight Survey Rules of Thumb p. 27/3

79 6. Subsamples for Nonresponse Mixture Example Mainly Subsample m N Noncontacts Treat Most (80%?) as m N = m NMAR Eight Survey Rules of Thumb p. 27/3

80 6. Subsamples for Nonresponse Mixture Example Mainly Subsample m N Noncontacts Treat Most (80%?) as m N = m NMAR Lightly Subsample m R Refusals Eight Survey Rules of Thumb p. 27/3

81 6. Subsamples for Nonresponse Mixture Example Mainly Subsample m N Noncontacts Treat Most (80%?) as m N = m NMAR Lightly Subsample m R Refusals If there are good covariates present Eight Survey Rules of Thumb p. 27/3

82 6. Subsamples for Nonresponse Mixture Example Mainly Subsample m N Noncontacts Treat Most (80%?) as m N = m NMAR Lightly Subsample m R Refusals If there are good covariates present Treat "Bulk" as m R = m MAR + m MCAR Eight Survey Rules of Thumb p. 27/3

83 6. Subsamples for Nonresponse Mixture Example Mainly Subsample m N Noncontacts Treat Most (80%?) as m N = m NMAR Lightly Subsample m R Refusals If there are good covariates present Treat "Bulk" as m R = m MAR + m MCAR "Bulk" can sometimes be estimated? Eight Survey Rules of Thumb p. 27/3

84 7. Sekar-Deming "Bias" Rates Eight Survey Rules of Thumb p. 28/3

85 7. Sekar-Deming "Bias" Rates Employ two samples on similar subjects Eight Survey Rules of Thumb p. 28/3

86 7. Sekar-Deming "Bias" Rates Employ two samples on similar subjects Include cases from first survey in second Eight Survey Rules of Thumb p. 28/3

87 7. Sekar-Deming "Bias" Rates Employ two samples on similar subjects Include cases from first survey in second Unlike Hansen-Hurwitz, obtain both Respondents and Nonrespondents Eight Survey Rules of Thumb p. 28/3

88 7. Sekar-Deming "Bias" Rates Employ two samples on similar subjects Include cases from first survey in second Unlike Hansen-Hurwitz, obtain both Respondents and Nonrespondents Create a 2 2 Table where the cells are rr = responded both times rn = responded first time only nr = responded second time only nn = never responded Eight Survey Rules of Thumb p. 28/3

89 7. Capture/Recapture Table [ rr rn nr nn ] Eight Survey Rules of Thumb p. 29/3

90 7. Potential Bias Rate Eight Survey Rules of Thumb p. 30/3

91 7. Potential Bias Rate dd = (rn)(nr) rr Eight Survey Rules of Thumb p. 30/3

92 7. Potential Bias Rate dd = (rn)(nr) rr Then bb = nn dd m can be considered the potential bias rate Eight Survey Rules of Thumb p. 30/3

93 7. Application Experiences Eight Survey Rules of Thumb p. 31/3

94 7. Application Experiences UI Nonprofit Nonresponse Rate = 27 % UI Nonprofit Potential Bias Rate = 11 % Eight Survey Rules of Thumb p. 31/3

95 7. Application Experiences UI Nonprofit Nonresponse Rate = 27 % UI Nonprofit Potential Bias Rate = 11 % NSAF Nonresponse Rate = 38 % NSAF Potential Bias Rate = 15 % Eight Survey Rules of Thumb p. 31/3

96 7. Application Experiences UI Nonprofit Nonresponse Rate = 27 % UI Nonprofit Potential Bias Rate = 11 % NSAF Nonresponse Rate = 38 % NSAF Potential Bias Rate = 15 % NORC Sales Tax Nonresponse Rate = 79 % NORC Sales Tax Potential Bias Rate = 28 % Eight Survey Rules of Thumb p. 31/3

97 Potential Next Steps Eight Survey Rules of Thumb p. 32/3

98 Potential Next Steps These Rules of Thumb obviously will not work everywhere but have been good starting points in many practical settings Eight Survey Rules of Thumb p. 32/3

99 Potential Next Steps These Rules of Thumb obviously will not work everywhere but have been good starting points in many practical settings Some reallocation of resources in survey execution and adjustment seems worthy of consideration in such settings Eight Survey Rules of Thumb p. 32/3

100 Potential Next Steps These Rules of Thumb obviously will not work everywhere but have been good starting points in many practical settings Some reallocation of resources in survey execution and adjustment seems worthy of consideration in such settings Estimating the missingness mixture fractions is key here. One way to do this has been proposed but more are needed Eight Survey Rules of Thumb p. 32/3

101 Provide to End Users and Future Survey Designers Eight Survey Rules of Thumb p. 33/3

102 Provide to End Users and Future Survey Designers Metadata/Paradata should include all relevant circumstances of the interview Eight Survey Rules of Thumb p. 33/3

103 Provide to End Users and Future Survey Designers Metadata/Paradata should include all relevant circumstances of the interview For example number of Calls, who was pesent, Language used, Length of interview, Data on case from Frame, etc Eight Survey Rules of Thumb p. 33/3

104 Provide to End Users and Future Survey Designers Metadata/Paradata should include all relevant circumstances of the interview For example number of Calls, who was pesent, Language used, Length of interview, Data on case from Frame, etc Nonrespondent cases should also go forward s a final deliverable, with details like point of Breakoff, Frame characteristics, number and conditions around Attempts, etc Eight Survey Rules of Thumb p. 33/3

105 Kish-Hess Data Files Eight Survey Rules of Thumb p. 34/3

106 Kish-Hess Data Files Employ Kish-Hess idea of consciously reusing nonrespondents but expand Eight Survey Rules of Thumb p. 34/3

107 Kish-Hess Data Files Employ Kish-Hess idea of consciously reusing nonrespondents but expand Keep safe identifiers of both respondents and nonrespondents for possible later use Eight Survey Rules of Thumb p. 34/3

108 Kish-Hess Data Files Employ Kish-Hess idea of consciously reusing nonrespondents but expand Keep safe identifiers of both respondents and nonrespondents for possible later use For bias reduction/analyses, as originally proposed, or for potential bias rate calculations, as described here Eight Survey Rules of Thumb p. 34/3

109 Kish-Hess Data Files Employ Kish-Hess idea of consciously reusing nonrespondents but expand Keep safe identifiers of both respondents and nonrespondents for possible later use For bias reduction/analyses, as originally proposed, or for potential bias rate calculations, as described here Could greatly lower long-term costs, while improving quality and enhancing credibility Eight Survey Rules of Thumb p. 34/3

110 Reanalysis Opportunities Eight Survey Rules of Thumb p. 35/3

111 Reanalysis Opportunities Consider public archiving of the augmented files Eight Survey Rules of Thumb p. 35/3

112 Reanalysis Opportunities Consider public archiving of the augmented files Revisit the augmentation regularly, as our understanding of what are the key paradata items can grow or change Eight Survey Rules of Thumb p. 35/3

113 Reanalysis Opportunities Consider public archiving of the augmented files Revisit the augmentation regularly, as our understanding of what are the key paradata items can grow or change Examine whether partnerships across organizations would have merit in measuring potential bias rates Eight Survey Rules of Thumb p. 35/3

114 Unintended Consequences and Complications Eight Survey Rules of Thumb p. 36/3

115 Unintended Consequences and Complications Look at statements made to potential respondents, should a sample of them be purposely returned Eight Survey Rules of Thumb p. 36/3

116 Unintended Consequences and Complications Look at statements made to potential respondents, should a sample of them be purposely returned Examine confidentiality issues that might arise even if former response status of a case was to be disclosed Eight Survey Rules of Thumb p. 36/3

117 Unintended Consequences and Complications Look at statements made to potential respondents, should a sample of them be purposely returned Examine confidentiality issues that might arise even if former response status of a case was to be disclosed Run cognitive tests of these ideas, for potential respondents and clients Eight Survey Rules of Thumb p. 36/3

118 8. Robust Confidence Interval Estimation Eight Survey Rules of Thumb p. 37/3

119 8. Robust Confidence Interval Estimation If the ratio is less than 1.0, use the average DEFT to make your adjustment for a given category. Eight Survey Rules of Thumb p. 37/3

120 8. Robust Confidence Interval Estimation If the ratio is less than 1.0, use the average DEFT to make your adjustment for a given category. If the ratio is 1.0 to 2.0, use the actual DEFT. Using the actual DEFT implies that the calculated estimate of the standard error is the appropriate one to use. Eight Survey Rules of Thumb p. 37/3

121 8. Robust Confidence Interval Estimation Eight Survey Rules of Thumb p. 38/3

122 8. Robust Confidence Interval Estimation If the ratio is greater than 2.0, subtract the average DEFT from the actual DEFT and use the remainder to make the adjustment. After you compute the ratio of the actualădeft to the average DEFT, takeăthe following into consideration: Eight Survey Rules of Thumb p. 38/3

123 8. Robust Confidence Interval Estimation If the ratio is greater than 2.0, subtract the average DEFT from the actual DEFT and use the remainder to make the adjustment. After you compute the ratio of the actualădeft to the average DEFT, takeăthe following into consideration: If the ratio is less than 1.0, use the average DEFT to make your adjustment for a given category. Eight Survey Rules of Thumb p. 38/3

124 8. Robust Confidence Interval Estimation If the ratio is greater than 2.0, subtract the average DEFT from the actual DEFT and use the remainder to make the adjustment. After you compute the ratio of the actualădeft to the average DEFT, takeăthe following into consideration: If the ratio is less than 1.0, use the average DEFT to make your adjustment for a given category. If the ratio is 1.0 to 2.0, use the actual DEFT. Using the actual DEFT implies that the calculated estimate of the standard error is the appropriate one to use. Eight Survey Rules of Thumb p. 38/3

125 8. Robust Confidence Interval Estimation If the ratio is greater than 2.0, subtract the average DEFT from the actual DEFT and use the remainder to make the adjustment. After you compute the ratio of the actualădeft to the average DEFT, takeăthe following into consideration: If the ratio is less than 1.0, use the average DEFT to make your adjustment for a given category. If the ratio is 1.0 to 2.0, use the actual DEFT. Using the actual DEFT implies that the calculated estimate of the standard error is the appropriate one to use. If the ratio is greater than 2.0, subtract the average DEFT from the actual DEFT and use the remainder to make the adjustment. Eight Survey Rules of Thumb p. 38/3

126 THANK YOU... Eight Survey Rules of Thumb p. 39/3