# A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey

Save this PDF as:

Size: px
Start display at page:

Download "A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey"

## Transcription

1 A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey Daifeng Han and David Cantor Westat, 1650 Research Blvd., Rockville, MD Abstract One method for studying nonresponse bias is to analyze estimates by level of effort (LOE) needed to complete an interview. LOE is typically measured using the number of times needed to contact a respondent and/or whether the respondent had initially refused to do the interview. LOE analyses are dependent on the assumption that non-respondents are similar to those who respond with more effort. A second method to assess nonresponse bias is to compare estimates to an external benchmark. In this paper, we compare the results of a nonresponse bias analysis for an random digit dialing (RDD) survey using a LOE approach to one that compares estimates to an external benchmark of a survey with a higher response rate. Important differences are found in the results between the two approaches. One conclusion is that the assumptions behind the level of effort model may not be correct for this survey. Keywords: Response rate, refusal conversion, nonresponse adjustment. 1. Introduction Assessing nonresponse bias requires making inferences about the characteristics of those persons for whom no observations have been made. The methods to make this assessment are based on inferences using assumptions about the nonrespondents that are difficult to validate. In standard nonresponse bias analysis, two common methods are: (1) comparing estimates to a benchmark an external data source with better and/or different error properties; and (2) examining variation in measures by the level of effort (LOE) used to complete an interview, where LOE is typically measured in two ways: the difficulty to contact a potential respondent, and the respondent s willingness to cooperate, once contacted. Each of these methods has strengths and weaknesses. In this paper we compare the results of a nonresponse bias analysis for an RDD survey using a LOE approach to one that compares estimates to an external benchmark of a survey with a higher response rate. The findings from each method will be evaluated and discussed in more detail in the next two sections. The final section will compare the results across different methods in an effort to emphasize the necessity to take multi-method approach for comprehensive nonresponse bias study. 1.1 Three Analyses for Evaluating Nonresponse Bias We conducted three analyses to assess nonresponse bias. One was a benchmark analysis. The other two were based on LOE: 1) level of effort to contact potential respondents, and 2) level of effort to earn cooperation. Benchmarking analysis compares survey estimates to an external data source with different, preferably better, error properties, such as higher coverage and higher response rate. The usefulness of benchmarking depends on the availability of external data source. It is usually difficult to find directly comparable data on the key measures of the survey. Quite often, the only available external data is demographic information in such cases benchmarking analysis only provides an indication of nonresponse bias to the extent the auxiliary demographic variables are related to the outcome statistics of the survey. Another method for assessing bias due to nonresponse is through an analysis by the LOE spent to complete an interview. The basis of this type of analysis is the assumption that those respondents that are interviewed with the most effort resemble those that are not interviewed at all. Using this logic, LOE analyses compare respondents who are interviewed with less effort to those interviewed with a greater amount of effort. If there is a difference, then under the above assumption, there is evidence of bias in the estimates. There are two sources of nonresponse, each of which is a function of different types of effort. One source is whether or not the respondent was contacted by the interviewer. An interview cannot occur if the respondent is not approached by an interviewer. In 3154

3 has a response rate of 69 percent for the sample adult questionnaire. Since the NHIS data are collected through a personal household interview, the survey also has better coverage relative to HINTS, especially of younger age group, minority group, and those without landline telephones. The common variables between the two surveys are mainly related to general health and health services. The NHIS results were used as the gold standard to examine the quality of the HINTS data. Table 1 shows the major differences between HINTS and NHIS estimates on health and health service measures. The HINTS standard errors on listed in the parentheses. The estimates from the two surveys are significantly different in all the variables except % having access to health insurance. Compared to NHIS, the proportion of people who self-reported to have good or excellent health in HINTS is approximately 11 percent points lower. In all the 6 variables that measure the feelings of negative emotions, the HINTS estimates are significantly higher; Table 1 only listed sadness or nervousness. The HINTS survey also seems to have included more eversmokers and more cancer patients. Table 1. Comparison of HINTS and NHIS estimates on health and health service measures * Health and health service measures HINTS estimates NHIS estimates HINTS - NHIS % in good or excellent health (0.81) % feeling sad some, most, or all of the time (0.91) % feeling nervous some, most, or all the time (0.93) % smoked 100+ cigarettes in life time (0.94) % ever had cancer (0.51) (0.17) % having access to health insurance (0.73) (0.29) % never visited a doctor in the past 12 months 16.4 (0.85) 18.6 (0.29) 2.2 * All the numerical values in the table are in percentage points. To interpret the differences, we should keep in mind that (1) such difference is a combined effect of noncoverage and nonresponse; and (2) mode effect may be a potential confounding factor. Existing literature does not suggest consistent patterns of a mode effect for health-related variables. The difference of such magnitude suggests that respondents on HINTS tend to be less healthy both physically and emotionally. Benchmarking analysis has shed some light on the characteristics of nonrespondents. However, among the approximately 200 HINTS variables, only less than 30 were also measured in NHIS and these variables were mainly on demographics and general health and health services. The key outcome variables in HINTS were cancer-related information seeking measures, for which no external data can be used as reliable comparable base. Due to the limitation of benchmarking method, we used LOE analysis to evaluate key outcome measures. 3. LOE Analysis Effort to Contact Respondent During the field period of HINTS 2005, the cases were released by batch groups. Each batch group can be considered a cross section of the total sample. As the field period progressed, yield from the initial sample turned out significantly lower than expected. This resulted in the release of additional batch groups relatively late in the field period in order to increase the number of completes for the survey. However, batch groups released late in the field period were not subject to as many callbacks as those released early. This had a significant effect on the screener level response rate. 3156

4 Three batch groups were used for the purpose of nonresponse bias analysis, referred to as long, middle, and short groups, corresponding to longest, medium, and shortest length of period respectively. Table 2 shows that a case in the short field period group received an average of 5 calls at the screener level, while the long field period group received approximately 7 calls on average. As a result, the screener level response rates are positively associated with the length of field period, as indicated by the difference of 8.6 percentage points between the long and the short groups. On the other hand, at the extended interview level, the response rates across groups were not noticeably different (data not shown). Table 2. Three batch groups for LOE analysis effort to contact target respondents Length of field period Full sample Long Median Short Size of screener sample 35,802 10,951 15,065 9,875 Average number of call attempts at the screener level Unweighted screener response rate 34.8% 39.7% 33.7% 31.1% Final completed cases 5,586 1,823 2,319 1,444 In the analysis of LOE to contact respondents, estimates from three groups were compared, first using base weights, and then using final adjusted weights. No substantial differences were found between base weight and final weight estimates except on several demographics variables. In this paper we present base weight results because it gives better information on who the nonrespondents are. A list of 49 variables were selected for the analysis, including demographics, health and health service measures, and communication measures. For simplicity we focused on examining differences between the two extreme groups the longest and the shortest field period groups displayed statistically significant differences in 8 variables. Table 3 shows the estimates by batch groups using base weights. It indicates that greater effort increases the yield on those who are typically difficult to contact on RDD surveys, such as male, low income population, and immigrants. The differences observed in health and communication variables may be due to the demographic differences across the groups. When final adjusted weights were applied, the differences in demographic characteristics across groups were almost eliminated. This is mainly due to the calibration mechanism which adjusted for demographics. Applying final weights did not eliminate the differences in health and communication variables. This suggests that there were other factors that affected response propensity and were not captured through weighting adjustment. Table 3. Results of LOE analysis effort to contact target respondents * Length of field period Longest shortest Long Median Short field period % male % household income <=20, % born outside US % feeling nervous % had mammogram in the past 12 months % read newspaper every day % found cancer info too hard to understand % use health provider as primary source of cancer information * All the numerical values in the table are in percentage points. All differences between short and long are statistically significant at p<

6 respondents. If we applied this assumption to our age and cancer variables, we would conclude that the survey under-represented older population and underestimated the cancer population. Is this a correct conclusion? Reexamining the results from benchmarking analysis will help answer this question. HINTS estimate for % ever had cancer was about 4 percent point higher than NHIS, which suggests that HINTS has overrepresented cancer population (see Table 2). The assumption also runs counter to the calibration that was done to create the final weights. In Table 6, by comparing the estimates using base weights and final weights, we see that the calibration brought younger age group up and older age group down, which is an indication that the survey overestimated the proportion of people in the older age group, rather than underestimated it. The same is true with the estimate of cancer population. One hypothesis for this pattern may be related to the way the topic of the survey is presented. At the screener, the survey is presented as a nationwide survey about health issues related to cancer. The mentioning of cancer may be more salient to older individuals, and thus helped bring in more older or even cancer people during refusal conversion. In this sense, converted refusers increased bias of demographics and health related characteristics. This result poses a methodological question: with all the time and money it incurs, is it actually worth to do refusal conversion? Will refusal conversion in any way help us? To gain more comprehensive understanding, we compared converted refusers to initial cooperators in terms of communication variables. As shown in Table 7, the initial refusers seem less likely to use Internet and more likely to read newspaper and use health provider as primary source of cancer info. These differences can be attributed mainly to demographic differences described above. What cannot be explained by the demographic difference is the likelihood for seeking cancer information. Since the converted refusers consisted more older people and cancer patients, we would expect them to be more likely to seek cancer information. On the contrary, we see a negative difference between the converted refusers and the initial cooperators, and the result holds once controlling for age and ever having cancer. This reflect some characteristics of the refusers that are independent of demographic and health variables. We suspect that the estimate for seeking cancer info was biased in an upward direction. Since the major interest of the HINTS survey was to measure health or cancer related information seeking, we suspect that the converted refusers were more similar to the final refusers in this aspect, so the refusal conversion effort has helped reduce nonresponse bias in key survey measures. Table 6. Calibration results Overall estimates Using base weights Using final adjusted weights % age % age Table 7. Results of LOE analysis effort to covert initial refusers: communication variables * Initial cooperators Converted refusers Converted refusers initial cooperators % read newspaper everyday % use internet % use health provider as primary source of cancer information % seek cancer information by self * All the numerical values in the table are in percentage points. 3159

7 5. Conclusion In conclusion, HINTS 2005 has over-represented (1) those most concerned about their health, including elderly and less healthy (more likely to have cancer; negative self-assessment) refusal conversion tends to increase this bias; and (2) those most likely to seek information about cancer refusal conversion tends to reduce this bias. Table 8 is a summary of comparison between LOE and benchmarking methods. Important differences are found in the results between the two approaches. Since the common assumption behind the LOE model does not apply to some measures of the survey, we could have drawn a wrong conclusion in nonresponse bias if we had used LOE as the single approach. The lessons we learned through this study is that LOE methods can provide useful information on how survey operations affect sample composition. However, to assess bias, it is important to use LOE in conjunction with other indicators of the non-respondent population such as calibration and benchmarking to other surveys. Table 8. Comparison between benchmarking method and LOE analysis Benchmarking: significant difference between HINTS and NHIS? LOE analysis: p-value for difference between two groups Two methods reaching consistent conclusions? Examples of common variables on between HINTS and NHIS % in good or excellent health Y 0.99 N % feeling sad some, most, or all of the time Y 0.85 N % feeling nervous some, most, or all the time Y 0.44 N % smoked 100+ cigarettes Y 0.54 N % ever had cancer Y 0.07 Y/N % having health insurance N 0.02 N % never visited a doctor in the past 12 months Y 0.28 N References Biemer, P. and Lyberg, L. (2003). Introduction to Survey Quality. New York: Wiley. Groves, R.M., Singer, E., and Corning, A. (2000). Leverage-Saliency Theory of Survey Participation. Public Opinion Quarterly, 64, Lin, I-F., and Schaeffer, N. (1995). Using Survey Participants to Estimate the Impact of Nonparticipation. Public Opinion Quarterly, 59: Olson, K. (2006). Survey Participation, Nonresponse Bias, Measurement Error Bias, and Total Bias. Public Opinion Quarterly, 70,

### APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS

APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS The purpose of the 2011 Health Insurance Surveys is a: Study of the insured, underinsured and uninsured Montanans Study of Montana s current

### Ownership and Usage Patterns of Cell Phones: 2000-2005

Ownership and Usage Patterns of Cell Phones: 2000-2005 Peter Tuckel, Department of Sociology, Hunter College Harry O Neill, Roper Public Affairs, NOP World Key Words: Cell phones, telephone surveys, Nonresponse

### USING DATABASE OVERLAYS TO DETECT NON- RESPONSE SURVEY BIAS

Worldwide Readership Research Symposium 2005 USING DATABASE OVERLAYS TO DETECT NON- RESPONSE SURVEY BIAS Roland Soong, Lindsey Draves & Hugh White, KMR/MARS Synopsis Using a database overlay on a sample

### A Review of the Cell Phone Sample Component of the California Health Interview Survey

A Review of the Cell Phone Sample Component of the California Health Interview Survey J. Michael Brick 1, Ismael Flores Cervantes 1, Jennifer Kali 1 Greg Norman 1, David Grant 2 1 Westat, 1600 Research

### Mississippi Residents Speak Out on Public Health Research

Mississippi Residents Speak Out on Public Health Research A Public Opinion Survey for Research!America 2004 Finding better ways to protect and promote your health Prevention and Public Health Research

### Keywords: time use, nonresponse

Response Analysis Survey: A Qualitative look at Response and Nonresponse in the American Time Use Survey Grace E. O Neill and Jessica R. Sincavage Bureau of Labor Statistics, 2 Massachusetts Ave, N.E.

### Leah Ericson Computing Services, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213, lericson@andrew.cmu.edu

A Comparison of Forced-Choice and Mark-All-That-Apply Formats for Gathering Information on Health Insurance in the 2006 American Community Survey Content Test 1 Leah Ericson Computing Services, Carnegie

### Community Life Survey Summary of web experiment findings November 2013

Community Life Survey Summary of web experiment findings November 2013 BMRB Crown copyright 2013 Contents 1. Introduction 3 2. What did we want to find out from the experiment? 4 3. What response rate

### An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates

An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates Ashley Bowers 1, Jeffrey Gonzalez 2 1 University of Michigan 2 Bureau of Labor Statistics Abstract Random-digit

### HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES

Public Opinion Quarterly, Vol. 71, No. 1, Spring 2007, pp. 3 22 HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES CLYDE TUCKER J. MICHAEL BRICK

### Internet Access and Use: Does Cell Phone Interviewing make a difference?

Internet Access and Use: Does Cell Phone Interviewing make a difference? By Evans Witt, Jonathan Best and Lee Rainie A paper prepared for The 2008 Conference of The American Association for Public Opinion

### Random Digit National Sample: Telephone Sampling and Within-household Selection

Sample Design Introduction The Princeton Affect and Time Survey, based on the American Time Use Survey (ATUS) and Day Reconstruction Method, employed a questionnaire designed to imitate key aspects of

### RDD Telephone Surveys: Reducing Bias and Increasing Operational Efficiency

RDD Telephone Surveys: Reducing Bias and Increasing Operational Efficiency David Ferraro 1, Tom Krenzke 1, Jill Montaquila 1 1 Westat, 1650 Research Blvd., Rockville, MD 20850 Abstract It is well documented

### AN EXPERIMENT IN CALL SCHEDULING. Patricia Cunningham, David Martin, J. Michael Brick Westat, 1650 Research Boulevard, Rockville, MD 20850-3195

AN EXPERIMENT IN CALL SCHEDULING Patricia Cunningham, David Martin, J. Michael Brick Westat, 1650 Research Boulevard, Rockville, MD 20850-3195 Keywords: Random Digit Dial, Telephone Survey, Calling Protocol

### 2014 Washington State Health Care Authority Adult Medicaid Health Plan CAHPS Report

State of Washington Health Care Authority 2014 Washington State Health Care Authority Adult Medicaid Health Plan CAHPS Report December 2014 3133 East Camelback Road, Suite 300 Phoenix, AZ 85016 Phone 602.264.6382

### 71% of online adults now use video sharing sites

71% of online adults now use video sharing sites Kathleen Moore, Pew Internet Project 7/25/2011 http://pewinternet.org/reports/2011/video sharing sites.aspx Pew Research Center 1615 L St., NW Suite 700

### 2003 National Survey of College Graduates Nonresponse Bias Analysis 1

2003 National Survey of College Graduates Nonresponse Bias Analysis 1 Michael White U.S. Census Bureau, Washington, DC 20233 Abstract The National Survey of College Graduates (NSCG) is a longitudinal survey

### The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes

The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes Douglas Currivan 1, David Roe 2, and Jason Stockdale 1 1 RTI International 2 Survey Sciences Group, LLC May

### Americans and their cell phones

Americans and their cell phones Mobile devices help people solve problems and stave off boredom, but create some new challenges and annoyances Aaron Smith, Senior Research Specialist 8/15/2011 http://pewinternet.org/reports/2011/cell-phones.aspx

### Practical Tools for Nonresponse Bias Studies

Practical Tools for Nonresponse Bias Studies Jill M. Montaquila Westat and the Joint Program in Survey Methodology Kristen M. Olson University of Nebraska-Lincoln SRMS/AAPOR Webinar April 24, 2012 Materials

### Multimode Strategies for Designing Establishment Surveys

Multimode Strategies for Designing Establishment Surveys Shelton M. Jones RTI International, 3 Cornwallis Rd., Research Triangle Park, NC, 2779 Abstract Design strategies are discussed that improve participation

### Non-response bias in a lifestyle survey

Journal of Public Health Medicine Vol. 19, No. 2, pp. 203-207 Printed in Great Britain Non-response bias in a lifestyle survey Anthony Hill, Julian Roberts, Paul Ewings and David Gunnell Summary Background

### WESTAT S PROPOSED WEIGHTING PLAN FOR THE PDII RDD SURVEY. DRAFT June 26, 2008

WESTAT S PROPOSED WEIGHTING PLAN FOR THE PDII RDD SURVEY DRAFT June 26, 2008 In order to obtain a sample of 2,000-2,500 respondents for the RDD survey, a self-weighting sample of 91,000 telephone numbers

### Merrile Sing. Agency for Healthcare Research and Quality Working Paper No. 08011. December 2008

Benchmarking Medicare Managed Care Plan Enrollment Estimates from the Medical Expenditure Panel Survey and Administrative Enrollment Files from the Centers for Medicare & Medicaid Services Merrile Sing

### AN INDICATION OF THE EFFECT OF NONINTERVIEW ADJUSTMENT AND POST -STRATIFICATION ON ESTIMATES FROM A SAMPLE SURVEY

AN INDICATION OF THE EFFECT OF NONINTERVIEW AND POST -STRATIFICATION ON ESTIMATES FROM A SAMPLE SURVEY Martha J. Banks, University of Chicago This paper is based on research done at the Center for Health

### THE EFFECT OF AGE AND TYPE OF ADVERTISING MEDIA EXPOSURE ON THE LIKELIHOOD OF RETURNING A CENSUS FORM IN THE 1998 CENSUS DRESS REHEARSAL

THE EFFECT OF AGE AND TYPE OF ADVERTISING MEDIA EXPOSURE ON THE LIKELIHOOD OF RETURNING A CENSUS FORM IN THE 1998 CENSUS DRESS REHEARSAL James Poyer, U.S. Bureau of the Census James Poyer, U.S. Bureau

### Incentives in Random Digit Dial Telephone Surveys: A Replication and Extension

Journal of Official Statistics, Vol. 23, No. 1, 2007, pp. 91 105 Incentives in Random Digit Dial Telephone Surveys: A Replication and Extension Richard Curtin 1, Eleanor Singer 2, and Stanley Presser 3

### National Findings on Access to Health Care and Service Use for Non-elderly Adults Enrolled in Medicaid

National Findings on Access to Health Care and Service Use for Non-elderly Adults Enrolled in Medicaid By Sharon K. Long Karen Stockley Elaine Grimm Christine Coyer Urban Institute MACPAC Contractor Report

### Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts

Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts David Dutwin, PhD State Health Research Policy Interest Group

### Research Overview: Telephone versus Online Research Advantages and Pitfalls

. 26030 Highway 74, Suite A, P.O. Box 1094 Kittredge, CO 80457 P: 303.679.6300 F: 303.679.6680 info@praxigroup.net www.praxigroup.net Research Overview: Telephone versus Online Research Advantages and

### The Risk of Losing Health Insurance Over a Decade: New Findings from Longitudinal Data. Executive Summary

The Risk of Losing Health Insurance Over a Decade: New Findings from Longitudinal Data Executive Summary It is often assumed that policies to make health insurance more affordable to the uninsured would

### Americans and text messaging

Americans and text messaging 31% of text message users prefer texting to voice calls, and young adults stand out in their use of text messaging Aaron Smith, Senior Research Specialist 9/19/2011 http://pewinternet.org/reports/2011/cell-phone-texting-2011.aspx

### 24% of internet users have made phone calls online

24% of internet users have made phone calls online Lee Rainie, Director May 30, 2011 http://pewinternet.org/reports/2011/13--internet-phone-calls--skype.aspx Pew Research Center s Internet & American Life

### HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004 Clyde Tucker, Bureau of Labor Statistics; J. Michael Brick, Westat; Brian Meekins, Bureau of Labor Statistics; and David Morganstein,

### Danny R. Childers and Howard Hogan, Bureau of the,census

MATCHING IRS RECORDS TO CENSUS RECORDS: SOME PROBLEMS AND RESULTS Danny R. Childers and Howard Hogan, Bureau of the,census A. INTRODUCTION This project has two principal aims: to investigate the feasibility

### 17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device

JUNE 26, 2012 17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device Most do so for convenience, but for some their phone is their only option

### Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS)

Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS) April 30, 2008 Abstract A randomized Mode Experiment of 27,229 discharges from 45 hospitals was used to develop adjustments for the

### Evaluating Mode Effects in the Medicare CAHPS Fee-For-Service Survey

Evaluating Mode Effects in the Medicare Fee-For-Service Survey Norma Pugh, MS, Vincent Iannacchione, MS, Trang Lance, MPH, Linda Dimitropoulos, PhD RTI International, Research Triangle Park, NC 27709 Key

### Cell Phone Activities 2013

www.pewresearch.org SEPTEMBER 16, 2013 Cell Phone Activities 2013 50% of cell owners download apps to their phones; 48% listen to music services; video calling has tripled since 2011; texting remains a

### Lessons Learned From an NHIS Web- Followback Feasibility Study

Lessons Learned From an NHIS Web- Followback Feasibility Study Sarah S. Joestl, DrPH Senior Service Fellow Survey Planning and Special Surveys Branch Division of Health Interview Statistics Division of

### Online Product Research

Online Product Research 58% of Americans have researched a product or service online September 29, 2010 Jim Jansen, Senior Fellow, Pew Internet Project http://pewinternet.org/reports/2010/online-product-research.aspx

### Quitline Tax Increase. Survey NEW ZEALAND POLICE CITIZENS SATISFACTION RESEARCH (TN/10/19) Six Month Follow Up. Contents

Market Research Proposal Proposal Prepared For New Zealand Police Quitline Tax Increase Research Report Prepared for The Quit Group January 2011 Survey Six Month Follow Up NEW ZEALAND POLICE CITIZENS SATISFACTION

### An Evaluation of Ported Telephone Numbers in the 2013 California Health Interview Survey

An Evaluation of Ported Telephone Numbers in the 2013 California Health Interview Survey Greg Norman 1 and Ismael Flores Cervantes 1 1 Westat, 1600 Research Boulevard, Rockville, MD 20850 Abstract In telephone

### Dual Frame Estimation in the 2003 National Survey of College Graduates

Dual Frame Estimation in the 2003 National Survey College Graduates John M. Finamore and David W. Hall, U.S. Census Bureau * Ronald S. Fecso, National Science Foundation * Abstract The Scientists and Engineers

### EXECUTIVE SUMMARY M S I - S T A T E W I D E S U R V E Y O N I N S U R A N C E F R A U D

EXECUTIVE SUMMARY The majority of Virginians believe you should obey the law without question but also suggest that right and wrong are ultimately determined by the individual. Nearly three out of five

### Converting the National Household Education Survey from RDD/CATI to ABS/Mail

Converting the National Household Education Survey from RDD/CATI to ABS/Mail Andrew Zukerberg Survey Redesign Seminar Presented July 17, 2012 This discussion is intended to promote the exchange of ideas

### Types of Error in Surveys

2 Types of Error in Surveys Surveys are designed to produce statistics about a target population. The process by which this is done rests on inferring the characteristics of the target population from

### CITY OF MILWAUKEE POLICE SATISFACTION SURVEY

RESEARCH BRIEF Joseph Cera, PhD Survey Center Director UW-Milwaukee Atiera Coleman, MA Project Assistant UW-Milwaukee CITY OF MILWAUKEE POLICE SATISFACTION SURVEY At the request of and in cooperation with

### Search and email still top the list of most popular online activities

Search and email still top the list of most popular online activities Two activities nearly universal among adult internet users Kristen Purcell, Associate Director for Research, Pew Internet Project August

### The Demographics of Social Media Users 2012

www.pewresearch.org FEBRUARY 14, 2013 The Demographics of Social Media Users 2012 Young adults are more likely than older adults to use social media. Women, African- Americans, and Latinos show high interest

### ADJUSTING FOR NONRESPONSE IN A TELEPHONE SUBSCRIBER SURVEY

DJUSTING FOR NONRESPONSE IN TELEPHONE SUBSCRIBER SURVEY James H. Drew, GTE Laboratories and Robert M. Groves, University of Michigan James H. Drew, GTE Laboratories, 40 Sylvan Rd. Waltham, M 02254 Keywords:

### FINDINGS FROM THE 2014 MASSACHUSETTS HEALTH INSURANCE SURVEY

CENTER FOR HEALTH INFORMATION AND ANALYSIS FINDINGS FROM THE MASSACHUSETTS HEALTH INSURANCE SURVEY MAY 2015 Prepared by: Laura Skopec and Sharon K. Long, Urban Institute Susan Sherr, David Dutwin, and

### HEALTH CARE COST CONCERNS AMONG OLDER RESIDENTS OF MASSACHUSETTS

HEALTH CARE COST CONCERNS AMONG OLDER RESIDENTS OF MASSACHUSETTS SEPTEMBER 2015 Robert J. Blendon, Sc. D. John M. Benson Caitlin L. McMurtry Justin M. Sayde EXECUTIVE SUMMARY Fifty years after Medicare

### Altarum Comparison of Physician and Consumer Health Care Opinions. September 20, 2012

Altarum Comparison of Physician and Consumer Health Care Opinions September 20, 2012 W. Lynch C. Ciucci Altarum Comparison of Physician and Consumer Health Care Opinions September 20, 2012 Table of Contents

### The Use and Effects of Incentives in Surveys

The Use and Effects of Incentives in Surveys Eleanor Singer Survey Research Center, University of Michigan Prepared for presentation to the National Science Foundation, Washington DC, October 3-4, 2012.

### 35% of those ages 16 and older own tablet computers

www.pewresearch.org OCTOBER 18, 2013 35% of those ages 16 and older own tablet computers Up from 25% last year more than half of those in households earning \$75,000 or more now have tablets 24% own e-book

### Financial Planning Profiles of American Households:

Financial Planning Profiles of American Households: The 2013 Household Financial Planning Survey and Index A Summary of Key Findings September 18, 2013 Prepared for: Certified Financial Planner Board of

### Tablet Ownership 2013

www.pewresearch.org JUNE 10, 2013 Tablet Ownership 2013 Tablet adoption has almost doubled over the past year. For the first time a third (34%) of American adults now own a tablet computer, including almost

### Getting Better Information from Country Consumers for Better Rural Health Service Responses

Getting Better Information from Country Consumers for Better Rural Health Service Responses Tony Woollacott, Anne Taylor, Kay Anastassiadis, Di Hetzel, Eleonora Dal Grande 5th National Rural Health Conference

### Epi Research Report New York City Department of Health and Mental Hygiene May 2010

Epi Research Report New York City Department of Health and Mental Hygiene May 2010 Including New Yorkers Who Can Be Reached Only by Cell Phone in the Community Health Survey: Results from the 2008 Cell

### Information Bulletin AN INCREASING INDIGENOUS POPULATION? IMPLICATIONS FOR THE CRIMINAL JUSTICE SYSTEM. Justine Doherty

Information Bulletin OFFICE OF CRIME STATISTICS No 32 April 2002 AN INCREASING INDIGENOUS POPULATION? IMPLICATIONS FOR THE CRIMINAL JUSTICE SYSTEM Justine Doherty ATTORNEY-GENERAL S DEPARTMENT Introduction

### NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE JANUARY 16, 2014 FOR FURTHER INFORMATION ON THIS REPORT:

NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE JANUARY 16, 2014 FOR FURTHER INFORMATION ON THIS REPORT: Kathryn Zickuhr, Research Associate, Pew Internet Lee Rainie, Director, Pew Internet 202.419.4500

### Western New England University Polling Institute

WESTERN NEW ENGLAND UNIVERSITY Western New England University Polling Institute Massachusetts Statewide Telephone Survey September 20-28, 2014 Dr. Tim Vercellotti For Immediate Release TABLES Next, please

### Adults and Cell Phone Distractions

Adults and Cell Phone Distractions Mary Madden, Senior Research Specialist Lee Rainie, Director June 18, 2010 http://pewinternet.org/reports/2010/cell-phone-distractions.aspx Pew Internet & American Life

### Wendy M. Ahmed and William D. Kalsbeek Survey Research Unit UNC-CH, 730 Airport Rd. Suite 103, Chapel Hill, NC 27514

AN ANALYSIS OF TELEPHONE CALL HISTORY DATA FROM THE BEHAVIORAL RISK FACTOR SURVEILLANCE SYSTEM Wendy M. Ahmed and William D. Kalsbeek Survey Research Unit UNC-CH, 730 Airport Rd. Suite 103, Chapel Hill,

### Identifying and Reducing Nonresponse Bias throughout the Survey Process

Identifying and Reducing Nonresponse Bias throughout the Survey Process Thomas Krenzke, Wendy Van de Kerckhove, and Leyla Mohadjer Westat Keywords: Weighting, Data Collection, Assessments. Introduction

### Jessica S. Banthin and Thomas M. Selden. Agency for Healthcare Research and Quality Working Paper No. 06005. July 2006

Income Measurement in the Medical Expenditure Panel Survey Jessica S. Banthin and Thomas M. Selden Agency for Healthcare Research and Quality Working Paper No. 06005 July 2006 Suggested citation: Banthin

### What does \$50,000 buy in a population survey?

What does \$50,000 buy in a population survey? Characteristics of internet survey participants compared with a random telephone sample Technical Brief No.4 October 2009 ISSN 1836-9014 Hilary Bambrick, Josh

### Experiment on Web based recruitment of Cell Phone Only respondents

Experiment on Web based recruitment of Cell Phone Only respondents 2008 AAPOR Annual Conference, New Orleans By: Chintan Turakhia, Abt SRBI Inc. Mark A. Schulman, Abt SRBI Inc. Seth Brohinsky, Abt SRBI

### Survey on Person-to-person Direct Marketing Calls

RESEARCH REPORT SUBMITTED TO THE OFFICE OF THE PRIVACY COMMISSIONER FOR PERSONAL DATA Survey on Person-to-person Direct Marketing Calls Social Sciences Research Centre The University of Hong Kong August

### THE NO MÁS STUDY: Domestic Violence and Sexual Assault in the Latin@ Community METHODOLOGY LIMITATIONS

THE NO MÁS STUDY: Domestic Violence and Sexual Assault in the Latin@ Community Commissioned by the Avon Foundation for Women for Casa de Esperanza: National Latin@ Network and NO MORE METHODOLOGY Lake

### Privacy and Data Management on Mobile Devices

SEPTEMBER 5, 2012 Privacy and Data Management on Mobile Devices More than half of app users have uninstalled or avoided an app due to concerns about personal information Jan Lauren Boyles Research Intern,

### Welcome & Introductions. A Study of Population Health. What Is the Family Health Survey?

The Bureau of Health Information and Policy presents Welcome & Introductions The Wisconsin Family Health Survey: A Study of Population Health Christine Hill, Section Chief Bureau of Health Information

### KNOWLEDGEPANEL DESIGN SUMMARY

KNOWLEDGEPANEL DESIGN SUMMARY This document was prepared at the effort and expense of GfK. No part of it may be circulated, copied, or reproduced for distribution without prior written consent. KnowledgePanel

### Mode Adjustment of the CAHPS Hospital Survey

of the CAHPS Hospital Survey December 21, 2006 Abstract A randomized Experiment of 27,229 discharges from 45 hospitals was used to develop adjustments for the effects of survey mode (Mail Only, Telephone

### 2009 Mississippi Youth Tobacco Survey. Office of Health Data and Research Office of Tobacco Control Mississippi State Department of Health

9 Mississippi Youth Tobacco Survey Office of Health Data and Research Office of Tobacco Control Mississippi State Department of Health Acknowledgements... 1 Glossary... 2 Introduction... 3 Sample Design

### Women s Way Direct Mail Project:

Women s Way Direct Mail Project: 2005 Summary Issued: December 2005 Prepared for: Women s Way, the North Dakota Breast and Cervical Cancer Early Detection Program Prepared by: North Dakota State Data Center

### NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE DECEMBER 30, 2013

NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE DECEMBER 30, 2013 Maeve Duggan, Research Assistant Aaron Smith, Senior Researcher 202.419.4500 RECOMMENDED CITATION: Maeve Duggan and Aaron Smith,

### Video calling and video chat

Video calling and video chat 19% of Americans have tried video calls or video chat or teleconferencing online and on cell phones By Lee Rainie, Director Kathryn Zickuhr, Web Coordinator 10/13/2010 http://pewinternet.org/reports/2010/video-chat.aspx

### 2015 Michigan Department of Health and Human Services Adult Medicaid Health Plan CAHPS Report

2015 State of Michigan Department of Health and Human Services 2015 Michigan Department of Health and Human Services Adult Medicaid Health Plan CAHPS Report September 2015 Draft Draft 3133 East Camelback

### Sources of Health Insurance Coverage in Georgia 2007-2008

Sources of Health Insurance Coverage in Georgia 2007-2008 Tabulations of the March 2008 Annual Social and Economic Supplement to the Current Population Survey and The 2008 Georgia Population Survey William

### NUMBERS, FACTS AND TRENDS SHAPING THE WORLD APRIL 3, 2014 FOR FURTHER INFORMATION ON THIS REPORT: Aaron Smith, Senior Researcher

NUMBERS, FACTS AND TRENDS SHAPING THE WORLD APRIL 3, 2014 FOR FURTHER INFORMATION ON THIS REPORT: Aaron Smith, Senior Researcher 202.419.4500 RECOMMENDED CITATION: Pew Research Center, April 2014, Older

### CDPHP CAHPS 4.0 Adult Medicaid Health Plan Survey

CAHPS 4.0 Adult Medicaid Health Plan Survey Continuous Quality Improvement Report February 2012 3975 Research Park Drive Ann Arbor, MI 48108 Table of Contents Background 1 Executive Summary 2 CAHPS 4.0

### Society of Actuaries Middle Market Life Insurance Segmentation Program (Phase 1: Young Families)

Society of Actuaries Middle Market Life Insurance Segmentation Program (Phase 1: Young Families) September 2012 Sponsored By: SOA Marketing and Distribution Section SOA Product Development Section SOA

### Health Care Expenditures for Uncomplicated Pregnancies

Health Care Expenditures for Uncomplicated Pregnancies Agency for Healthcare Research and Quality U.S. Department of Health & Human Services August 2007 ABSTRACT This report uses data pooled from three

### Improving Survey Response in Spanish-speaking Households. 126 College Street, Burlington, VT 05401,

Improving Survey Response in Spanish-speaking Households James Dayton 1*, Michael W. Link 2, Robert Pels 1, and Kirsten Ivie 1 1 ORC-Macro International, 2 Centers for Disease Control and Prevention *James.J.Dayton@orcmacro.com,

Generations and their gadgets By Kathryn Zickuhr, Web Coordinator February 3, 2011 http://pewinternet.org/reports/2011/generations-and-gadgets.aspx Pew Research Center s Internet & American Life Project

### LONG-TERM CARE IN AMERICA: AMERICANS OUTLOOK AND PLANNING FOR FUTURE CARE

Research Highlights LONG-TERM CARE IN AMERICA: AMERICANS OUTLOOK AND PLANNING FOR FUTURE CARE INTRODUCTION In the next 25 years, the U.S. population is expected to include 82 million Americans over the

### THE CAYMAN ISLANDS LABOUR FORCE SURVEY REPORT SPRING 2015

THE CAYMAN ISLANDS LABOUR FORCE SURVEY REPORT SPRING 2015 Published September 2015 Economics and Statistics Office i CONTENTS SUMMARY TABLE 1. KEY LABOUR FORCE INDICATORS BY STATUS... 1 SUMMARY TABLE 2.

### Chapter 5: Analysis of The National Education Longitudinal Study (NELS:88)

Chapter 5: Analysis of The National Education Longitudinal Study (NELS:88) Introduction The National Educational Longitudinal Survey (NELS:88) followed students from 8 th grade in 1988 to 10 th grade in

### Population Aging Research Center. University of Pennsylvania. Mexican Migration to the US and Access to Health Care

Population Aging Research Center University of Pennsylvania Mexican Migration to the US and Access to Health Care Sara Ross, José Pagán, and Daniel Polsky PARC Working Paper Series WPS 05-12 "The authors

### The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys

The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys Barbara Lepidus Carlson 1 and Richard Strouse 2 Mathematica Policy Research, Inc. 1 Battelle 2 Abstract Declining

### Contacting respondents for survey research

Contacting respondents for survey research Is email a useful method? Joanna d Ardenne and Margaret Blake November 2012 Background The question Is email a useful way of contacting potential respondents

### Cell Phones and Nonsampling Error in the American Time Use Survey October 2012

Cell Phones and Nonsampling Error in the American Time Use Survey October 2012 Brian Meekins, Stephanie Denton 1,i 1 Bureau of Labor Statistics Abstract Recent research on the impact of cell phones has

### Health Status, Health Insurance, and Medical Services Utilization: 2010 Household Economic Studies

Health Status, Health Insurance, and Medical Services Utilization: 2010 Household Economic Studies Current Population Reports By Brett O Hara and Kyle Caswell Issued July 2013 P70-133RV INTRODUCTION The

### Executive summary. Participation in gambling activities (Chapter 2)

Executive summary This report presents results from the British Gambling Prevalence Survey (BGPS) 2010. This is the third nationally representative survey of its kind; previous studies were conducted in

### Generations 2010. Kathryn Zickuhr, Web Coordinator. http://pewinternet.org/reports/2010/generations-2010.aspx 12/16/2010

s 2010 Kathryn Zickuhr, Web Coordinator 12/16/2010 http://pewinternet.org/reports/2010/s-2010.aspx Pew Research Center 1615 L St., NW Suite 700 Washington, D.C. 20036 202-419-4500 pewinternet.org Page