Telephone and Mail Surveys: Advantages and Disadvantages of Each

Size: px
Start display at page:

Download "Telephone and Mail Surveys: Advantages and Disadvantages of Each"

Transcription

1 Telephone and Mail Surveys: Advantages and Disadvantages of Each Dan Zahs and Reg Baker Market Strategies, Inc. May 29, 2007 Selecting the mode of administration for a survey requires that one evaluate a number of factors and understand clearly the tradeoffs involved in choosing one mode over another. While telephone and mail research share some similar qualities, there are major differences. We group these differences into four main categories: (1) sample frame; (2) non-response bias; (3) measurement error; and (4) time and money. The Sample Frame A sample frame is essentially a list used to select the sample of persons to be interviewed. A high quality frame is one that contains a complete or nearly complete list of the target population. If the frame is sufficiently complete and accurate, a random sample selected from this frame will be non-biased and representative of the target population. One major reason for the adoption of telephone as the gold standard for much of commercial research over the last 25 years has been the quality of the sample frame. The standard frame for RDD telephone surveys uses information from the telephone companies about the assignment status of groups of telephone numbers (known as groups and blocks). Until recently, this frame was considered largely complete and accurate, with a relatively small number of missing non-telephone households. More recently, the widespread adoption of cell phones has threatened the integrity of the RDD frame. For various reasons, cell phones are not part of this standard frame. Until recently, most people with a cell phone also had a traditional telephone or landline. However, the proportion households having a cell phone but no landline has been growing rapidly and is now around 13 percent (Blumberg and Luke (2007)). Using the standard RDD frame, these cell-only households have no chance of being selected into any sample. When cases are excluded from the possibility of being selected, bias might be introduced into the results. This is known as coverage error. It is possible to include the cell-only population into the sample selection process through the use of a dual frame approach, but this can add significant cost and poses some difficult weighting challenges. Fortunately, recent research suggests that at this point the exclusion of cell-only households does not lead to substantial levels of bias, that is, there are few significant differences in overall estimates produced from samples with and without cell-only households (See, for example, Keeter (2006) and Brick et al. (2006)). That said, cell-only households are disproportionately comprised of year olds and so there is some demographic bias in samples that exclude those households. Research has found that persons in cell-only households differ from landline households on some important health measures such as smoking and binge drinking (Blumberg and Luke 1

2 (2007)). Studies with a special focus on that age group can have substantial bias if cellonly households are not included. Thus far, this group is not so large as to significantly affect estimates for the overall population. The availability of sample frames for mail studies is also an unfolding story. Historically, mail studies of the general population have tended to use a so-called listed frame compiled from various sources including the directories of people listed in the white pages of telephone books. These listed frames are generally incomplete and have high levels of coverage error. For example, a match of an RDD sample to a list of people in a telephone book typically yields about a 50 percent match rate. More recently, a new frame option has emerged and is still being evaluated by survey methodologists. This frame is maintained by the U.S. Postal Service and is called the Delivery Sequence File (DSF). As the name suggests, this is the database the USPS uses for their deliveries. This frame appears to be much more comprehensive and complete than previous mail frames, although early studies suggest that there still may be significant coverage error, specifically of non-urban and lower income areas (Link and Mokdad (2005)). Non-Response Bias The potential for bias in a survey results is not just a function of coverage (i.e. the quality of the frame). Non-response also is a major factor. One hundred percent response may be an admirable goal, but it is almost never achieved. Large proportions of any selected sample, regardless of mode, typically do not respond. This non-response tends to be systematic rather than random, and therefore bias is introduced. Put in simple terms, the non-response bias question is: Would the estimates from my survey be different if everyone had responded? Put another way: Are those who did not respond different from those who did in some systematic way that biases my results? Historically, telephone surveys have yielded significantly higher response rates (and therefore lower non-response) than mail, although the achieved response rate in any mode is a function of the implementation techniques used (e.g., advance letters, incentives, refusal conversion, etc.) Telephone surveys typically under represent people who are difficult to catch at home (e.g., younger, single, or non-family households) while mail surveys have tended to under represent lower SES households. In both instances, post stratification weights are used to bring the achieved sample back in line with population demographics. Over about the last decade we have seen significant declines in response rates to all modes. The impact on the validity of telephone research has been particularly studied because of its widespread use. Recent studies have demonstrated that response rate is not as important a measure of survey data quality as was once thought (See, for example, Holbrook et al. (in press) and Keeter et al. (2006)). Surveys with low response rates (as low as four percent in one comparison) can yield results that are statistically equivalent to those from surveys with much higher response rates, although a high response rate is usually better than a low one. This is interpreted as a validation of the power or probability sampling from good quality sample frames. 2

3 Measurement Error Measurement error can have any number of sources including the interviewer, the questionnaire, the mode of administration, and the respondent. Interviewers may be poorly trained and administer the questionnaire incorrectly. The questionnaire may be flawed in some important way that leads to comprehension problems or misunderstanding. The presence or absence of an interviewer may influence how a respondent answers. The respondent may lack the cognitive ability to comprehend and answer the questions or simply lack the motivation to answer carefully. Generally speaking, the advantage of well-trained interviewers has led researchers to prefer telephone over mail. Interviewers ensure that the correct target respondent within a household completes the survey; administer the survey so all questions are asked; help respondents understand questions or concepts that might be ambiguous or difficult; and keep the respondent engaged over the duration of the survey. They also can probe respondents to get fuller and more accurate answers in open ended questions. One classic statistic here is item non-response, that is, the proportion of missing data in individual questions (skipped questions, don t know or refusal to answer). Telephone studies typically have significantly lower item non-response than self-administered modes such as mail and Web. The combination of well-trained interviewers and computer-assisted interviewing technologies (such as CATI) have made it possible to design and administer very complex questionnaires that tailor the interview to each respondent, prevent routing errors, reduce data recording errors, maintain consistency in respondent answers, and use design techniques such as randomization and rotation to reduce order and context effects. Telephone surveys also lend themselves to interviewing in multiple languages. Mail surveys by virtue of their self-administered, paper-and-pencil format are much more limited. Questionnaires are generally shorter, simpler, have few skips, and must be crafted to be as clear and unambiguous as possible. The researcher has less control over who completes a mail survey and the order in which questions are answered. Recording errors are much more common than with interviewer administration. Commonly used techniques such as randomization, rotation, and multiple language interviewing are extremely difficult to implement. Despite their many benefits, interviewers can also be a source of error if they are not well-trained and monitored. Further, the presence of the interviewer can sometimes lead to under reporting of socially sensitive behaviors (e.g., drug use or binge drinking), a phenomenon know as social desirability bias. The principal advantage of mail surveys in terms of measurement error is the potential reduction in social desirability bias. Time and Money Arguably the simplest comparison of the two modes focuses on cost and length of field period. Mail surveys are almost always less expensive than telephone because the cost of 3

4 mailing and data conversion (keying or scanning) are significantly less than the cost of interviewer labor. Mail surveys typically require an incentive to achieve any sort of reasonable response rate, and depending on the magnitude of that incentive mail surveys can sometimes approach the cost of telephone surveys where incentives are often not offered except where very high response rates are necessary. On the other hand, mail surveys generally require much longer field periods, and the researcher s ability to either predict or control this key condition is very limited when compared to telephone. Summary As we hope the foregoing makes clear, telephone surveys have a broad set of advantages over mail. Further, these advantages are generally recognized by researchers and comprise the main reason why RDD telephone surveys have been consistently viewed as the methodology of choice for high quality, general population surveys. Generally speaking, mail surveys have tended to be viewed as a compromise that sacrifices data quality in return for lower costs. At present, however, there are dynamics at work relative to sample frames that could undermine the dominant role that telephone surveys have traditionally enjoyed. The rapid rise of cell-only households has begun to introduce coverage problems into the traditional RDD sample frame. The most likely remedies will increase the cost of telephone research significantly. At the same time, the emergence of a new and possibly more complete mail sample frame in the form of the DFS may create a significant coverage advantage for mail surveys. Issues of coverage area and cost aside, the combination of interviewer administration and modern CATI technology continue to present and extremely powerful data quality argument for telephone research. References Blumberg, S.J., and Luke, J.V Wireless Substitution: Early Release Estiamtes Based on Data from the National Health Interview Survey, July December 2006, National Center for Health Statistics. Brick, M. J., Dipko, S., Presser, S., Tucker, C., andyuan, Y Nonresponse Bias in a Dual Frame Sample of Cell and Landline Numbers, Public Opinion Quarterly, 70: Holbrook, A. L., Krosnick, J.A., and Pfent, A.M. In Press. Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms, in J. 4

5 Lepkowski, B. Harris-Kojetin, P.J. Lavrakas, C. Tucker, E. de Leeuw, M. Link, M. Brick, L. Japec, and R. Sangster (eds.), Telephone Survey Methodology. New York: Wiley. Keeter, S The Impact of Cell Phone Noncoverage Bias on Polling in the 2004 Presidential Election, Public Opinion Quarterly, 70: Keeter, S., Kennedy, C., Dimock, M., Best, J., and Craighill, P Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey, Public Opinion Quarterly, 70: Link, M.W., Mokdad, A.H Use of Alternative Modes for Health Surveillance surveys: Results from a Web/Mail/Telephone Experiment, Epidemiology, 16:

Integrating Cell Phone Numbers into Random Digit-Dialed (RDD) Landline Surveys

Integrating Cell Phone Numbers into Random Digit-Dialed (RDD) Landline Surveys Integrating Cell Phone Numbers into Random Digit-Dialed (RDD) Landline Surveys Martin R. Frankel 1, Michael P. Battaglia 2, Michael Link 3, and Ali H. Mokdad 3 1 Baruch College, City University of New

More information

Cell Phone Sampling Summit II Proposed Statements on Accounting for Cell Phones in Telephone Survey Research in the U.S.

Cell Phone Sampling Summit II Proposed Statements on Accounting for Cell Phones in Telephone Survey Research in the U.S. Cell Phone Sampling Summit II Proposed Statements on Accounting for Cell Phones in Telephone Survey Research in the U.S. Prepared by Paul J. Lavrakas and Charles D. Shuttles In February 2005, a meeting

More information

Cell Phones and Nonsampling Error in the American Time Use Survey

Cell Phones and Nonsampling Error in the American Time Use Survey Cell Phones and Nonsampling Error in the American Time Use Survey October 2012 Brian Meekins, Stephanie Denton 1,i 1 Bureau of Labor Statistics Abstract Recent research on the impact of cell phones has

More information

Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts

Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts David Dutwin, PhD State Health Research Policy Interest Group

More information

E-reader Ownership Doubles in Six Months

E-reader Ownership Doubles in Six Months E-reader Ownership Doubles in Six Months Adoption rate of e-readers surges ahead of tablet computers Kristen Purcell, Associate Director for Research, Pew Internet Project June 27, 2011 Pew Research Center

More information

Addressing the Cell Phone-Only Problem: Cell Phone Sampling Versus Address Based Sampling

Addressing the Cell Phone-Only Problem: Cell Phone Sampling Versus Address Based Sampling Addressing the Cell Phone-Only Problem: Cell Phone Sampling Versus Address Based Sampling Michael W. Link 1, Gail Daily 2, Charles D. Shuttles 3, H. Christine Bourquin 2, and L. Tracie Yancey 2 1 The Nielsen

More information

Types of Error in Surveys

Types of Error in Surveys 2 Types of Error in Surveys Surveys are designed to produce statistics about a target population. The process by which this is done rests on inferring the characteristics of the target population from

More information

24% of internet users have made phone calls online

24% of internet users have made phone calls online 24% of internet users have made phone calls online Lee Rainie, Director May 30, 2011 http://pewinternet.org/reports/2011/13--internet-phone-calls--skype.aspx Pew Research Center s Internet & American Life

More information

Recruiting Probability-Based Web Panel Members Using an Address-Based Sample Frame: Results from a Pilot Study Conducted by Knowledge Networks

Recruiting Probability-Based Web Panel Members Using an Address-Based Sample Frame: Results from a Pilot Study Conducted by Knowledge Networks Recruiting Probability-Based Web Panel Members Using an Address-Based Sample Frame: Results from a Pilot Study Conducted by Knowledge Networks Charles DiSogra, Mario Callegaro, Erlina Hendarwan Knowledge

More information

The Rise in Download Rate of Search & Email Use by Adults

The Rise in Download Rate of Search & Email Use by Adults Search and email still top the list of most popular online activities Two activities nearly universal among adult internet users Kristen Purcell, Associate Director for Research, Pew Internet Project August

More information

Introduction... 3. Qualitative Data Collection Methods... 7 In depth interviews... 7 Observation methods... 8 Document review... 8 Focus groups...

Introduction... 3. Qualitative Data Collection Methods... 7 In depth interviews... 7 Observation methods... 8 Document review... 8 Focus groups... 1 Table of Contents Introduction... 3 Quantitative Data Collection Methods... 4 Interviews... 4 Telephone interviews... 5 Face to face interviews... 5 Computer Assisted Personal Interviewing (CAPI)...

More information

Self-administered Questionnaire Survey

Self-administered Questionnaire Survey Primary methods of survey data collection Self-administered questionnaire (SAQ) Mail Face-to-face interview Telephone interview Self-administered Questionnaire Survey 1 Steps are: a. Mail distribution

More information

JSM 2013 - Survey Research Methods Section

JSM 2013 - Survey Research Methods Section How Representative are Google Consumer Surveys?: Results from an Analysis of a Google Consumer Survey Question Relative National Level Benchmarks with Different Survey Modes and Sample Characteristics

More information

Address-Based Sampling (ABS) Merits, Design, and Implementation

Address-Based Sampling (ABS) Merits, Design, and Implementation Address-Based Sampling (ABS) Merits, Design, and Implementation Mansour Fahimi, Ph.D. VP, Statistical Research Services National Conference on Health Statistics National Center for Health Statistics (NCHS)

More information

Cell Phone Reverse Directories Promise or Peril?

Cell Phone Reverse Directories Promise or Peril? Cell Phone Reverse Directories Promise or Peril? Sarah Dipko 1 and Krista Freedman 1 1 Westat Key Words: Cell phones, reverse directories, advance notification, sample purging 1. Introduction Surveys of

More information

Ensuring Data Quality: Monitoring Accuracy and Consistency Among Telephone Interview Monitors

Ensuring Data Quality: Monitoring Accuracy and Consistency Among Telephone Interview Monitors Ensuring Data Quality: Monitoring Accuracy and Consistency Among Telephone Interview Monitors Joseph Baker, Claudia Gentile, Jason Markesich, Shawn Marsh, and Rebecca Weiner Joseph Baker, Mathematica Policy

More information

Developing a Sustainable and Flexible Long-Distance Travel Survey

Developing a Sustainable and Flexible Long-Distance Travel Survey Developing a Sustainable and Flexible Long-Distance Travel Survey Lee Giesbrecht and Jonaki Bose Bureau of Transportation Statistics, 400 Seventh Street, SW, Washington, DC 20590 Abstract The Bureau of

More information

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES Public Opinion Quarterly, Vol. 71, No. 1, Spring 2007, pp. 3 22 HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES CLYDE TUCKER J. MICHAEL BRICK

More information

Older Adults and Social Media Social networking use among those ages 50 and older nearly doubled over the past year

Older Adults and Social Media Social networking use among those ages 50 and older nearly doubled over the past year Older Adults and Social Media Social networking use among those ages 50 and older nearly doubled over the past year Mary Madden, Senior Research Specialist August 27, 2010 Report URL: http://pewinternet.org/reports/2010/older-adults-and-social-media.aspx

More information

The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes

The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes Douglas Currivan 1, David Roe 2, and Jason Stockdale 1 1 RTI International 2 Survey Sciences Group, LLC May

More information

Inter-mode comparability in telephone surveys via landline and mobile phones

Inter-mode comparability in telephone surveys via landline and mobile phones Inter-mode comparability in telephone surveys via landline and mobile phones Working paper June 2010 Authors Mike Kühne Dresden University of Technology mike.kuehne@tu-dresden.de Alexander Wentland Social

More information

Descriptive Methods Ch. 6 and 7

Descriptive Methods Ch. 6 and 7 Descriptive Methods Ch. 6 and 7 Purpose of Descriptive Research Purely descriptive research describes the characteristics or behaviors of a given population in a systematic and accurate fashion. Correlational

More information

A Review of the Cell Phone Sample Component of the California Health Interview Survey

A Review of the Cell Phone Sample Component of the California Health Interview Survey A Review of the Cell Phone Sample Component of the California Health Interview Survey J. Michael Brick 1, Ismael Flores Cervantes 1, Jennifer Kali 1 Greg Norman 1, David Grant 2 1 Westat, 1600 Research

More information

Missing Data. A Typology Of Missing Data. Missing At Random Or Not Missing At Random

Missing Data. A Typology Of Missing Data. Missing At Random Or Not Missing At Random [Leeuw, Edith D. de, and Joop Hox. (2008). Missing Data. Encyclopedia of Survey Research Methods. Retrieved from http://sage-ereference.com/survey/article_n298.html] Missing Data An important indicator

More information

Evaluation: Designs and Approaches

Evaluation: Designs and Approaches Evaluation: Designs and Approaches Publication Year: 2004 The choice of a design for an outcome evaluation is often influenced by the need to compromise between cost and certainty. Generally, the more

More information

Survey Nonresponse and Nonresponse Bias. Survey Research Laboratory University of Illinois at Chicago

Survey Nonresponse and Nonresponse Bias. Survey Research Laboratory University of Illinois at Chicago Survey Nonresponse and Nonresponse Bias Survey Research Laboratory University of Illinois at Chicago Outline Overview of response rate issue Overview of non-response bias Review of available methods for

More information

Best Practices: Understanding and Reducing Bias in Your Surveys

Best Practices: Understanding and Reducing Bias in Your Surveys Best Practices: Understanding and Reducing Bias in Your Surveys Sampling Bias - Who You Survey Sampling Bias: When some members of your survey population are less likely to be invited to be surveyed than

More information

Using Proxy Measures of the Survey Variables in Post-Survey Adjustments in a Transportation Survey

Using Proxy Measures of the Survey Variables in Post-Survey Adjustments in a Transportation Survey Using Proxy Measures of the Survey Variables in Post-Survey Adjustments in a Transportation Survey Ting Yan 1, Trivellore Raghunathan 2 1 NORC, 1155 East 60th Street, Chicago, IL, 60634 2 Institute for

More information

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004 HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004 Clyde Tucker, Bureau of Labor Statistics; J. Michael Brick, Westat; Brian Meekins, Bureau of Labor Statistics; and David Morganstein,

More information

IRS Oversight Board 2014 Taxpayer Attitude Survey DECEMBER 2014

IRS Oversight Board 2014 Taxpayer Attitude Survey DECEMBER 2014 IRS Oversight Board 14 Taxpayer Attitude Survey DECEMBER 14 The Internal Revenue Service (IRS) Oversight Board was created by Congress under the IRS Restructuring and Reform Act of 1998. The Oversight

More information

BACKGROUND ON THE SURVEY PROCESS

BACKGROUND ON THE SURVEY PROCESS Chapter Two BACKGROUND ON THE SURVEY PROCESS In this chapter, we present an overview of the various aspects of the research survey process. 1 We emphasize that surveying should first be thought of as a

More information

The Demographics of Social Media Users 2012

The Demographics of Social Media Users 2012 www.pewresearch.org FEBRUARY 14, 2013 The Demographics of Social Media Users 2012 Young adults are more likely than older adults to use social media. Women, African- Americans, and Latinos show high interest

More information

Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS)

Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS) Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS) April 30, 2008 Abstract A randomized Mode Experiment of 27,229 discharges from 45 hospitals was used to develop adjustments for the

More information

Chapter 8: Quantitative Sampling

Chapter 8: Quantitative Sampling Chapter 8: Quantitative Sampling I. Introduction to Sampling a. The primary goal of sampling is to get a representative sample, or a small collection of units or cases from a much larger collection or

More information

Ownership and Usage Patterns of Cell Phones: 2000-2005

Ownership and Usage Patterns of Cell Phones: 2000-2005 Ownership and Usage Patterns of Cell Phones: 2000-2005 Peter Tuckel, Department of Sociology, Hunter College Harry O Neill, Roper Public Affairs, NOP World Key Words: Cell phones, telephone surveys, Nonresponse

More information

APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS

APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS The purpose of the 2011 Health Insurance Surveys is a: Study of the insured, underinsured and uninsured Montanans Study of Montana s current

More information

Cell Phone Coverage in an ABS Multi-Mode Design

Cell Phone Coverage in an ABS Multi-Mode Design Cell Phone Coverage in an ABS Multi-Mode Design Kanru Xia 1, Ashley Amaya 1, Whitney Murphy 1 1 NORC at the University of Chicago, 55 E. Monroe St., Chicago, IL 60603 Abstract By the end of 2011, 32.3

More information

Research Online. University of Wollongong. Margo Barr University of Wollongong, mlb593@uowmail.edu.au

Research Online. University of Wollongong. Margo Barr University of Wollongong, mlb593@uowmail.edu.au University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Information Sciences 2012 Inclusion of mobile phone numbers into an

More information

Sampling and Sampling Distributions

Sampling and Sampling Distributions Sampling and Sampling Distributions Random Sampling A sample is a group of objects or readings taken from a population for counting or measurement. We shall distinguish between two kinds of populations

More information

see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys

see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys SURVEYS Survey = ask the same questions of a large sample see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys Basic Survey Process: 1. Design Questionnaire 2. Design Sample

More information

An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates

An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates Ashley Bowers 1, Jeffrey Gonzalez 2 1 University of Michigan 2 Bureau of Labor Statistics Abstract Random-digit

More information

U.S. Household Food Security Survey Module: Six-Item Short Form Economic Research Service, USDA September 2012

U.S. Household Food Security Survey Module: Six-Item Short Form Economic Research Service, USDA September 2012 U.S. Household Food Security Survey Module: Six-Item Short Form Economic Research Service, USDA September 2012 Revision Notes: The food security questions in the 6-item module are essentially unchanged

More information

REACHING THE U.S. CELL PHONE GENERATION COMPARISON OF CELL PHONE SURVEY RESULTS WITH AN ONGOING LANDLINE TELEPHONE SURVEY

REACHING THE U.S. CELL PHONE GENERATION COMPARISON OF CELL PHONE SURVEY RESULTS WITH AN ONGOING LANDLINE TELEPHONE SURVEY Public Opinion Quarterly, Vol. 71, No. 5 2007, pp. 814 839 REACHING THE U.S. CELL PHONE GENERATION COMPARISON OF CELL PHONE SURVEY RESULTS WITH AN ONGOING LANDLINE TELEPHONE SURVEY MICHAEL W. LINK MICHAEL

More information

Research Overview: Telephone versus Online Research Advantages and Pitfalls

Research Overview: Telephone versus Online Research Advantages and Pitfalls . 26030 Highway 74, Suite A, P.O. Box 1094 Kittredge, CO 80457 P: 303.679.6300 F: 303.679.6680 info@praxigroup.net www.praxigroup.net Research Overview: Telephone versus Online Research Advantages and

More information

Chapter 8: Survey Research

Chapter 8: Survey Research Chapter Outlines for: Frey, L., Botan, C., & Kreps, G. (1999). Investigating communication: An introduction to research methods. (2nd ed.) Boston: Allyn & Bacon. Chapter 8: Survey Research I. Introduction

More information

SURVEY RESEARCH RESEARCH METHODOLOGY CLASS. Lecturer : RIRI SATRIA Date : November 10, 2009

SURVEY RESEARCH RESEARCH METHODOLOGY CLASS. Lecturer : RIRI SATRIA Date : November 10, 2009 SURVEY RESEARCH RESEARCH METHODOLOGY CLASS Lecturer : RIRI SATRIA Date : November 10, 2009 DEFINITION OF SURVEY RESEARCH Survey: A method of primary data collection based on communication with a representative

More information

Asking the Right Questions: Designing a Survey That Works Presentation Notes

Asking the Right Questions: Designing a Survey That Works Presentation Notes Asking the Right Questions: Designing a Survey That Works Presentation Notes Slide 4: Before you even begin to design a survey, you need to determine whether or not a survey is the appropriate methodology

More information

71% of online adults now use video sharing sites

71% of online adults now use video sharing sites 71% of online adults now use video sharing sites Kathleen Moore, Pew Internet Project 7/25/2011 http://pewinternet.org/reports/2011/video sharing sites.aspx Pew Research Center 1615 L St., NW Suite 700

More information

Oregon Health Insurance Survey

Oregon Health Insurance Survey Oregon Health Insurance Survey A Summary of Developing, Fielding, and Analyzing the Oregon Health Insurance Survey For An In-depth Analysis Please Review The Oregon Health Survey 2011 Methodology Report

More information

Do Introductory Sentences Cause Acquiescence Response Bias in Survey Questions? Jon A. Krosnick. Ana Villar. and. Bo MacInnis. Stanford University

Do Introductory Sentences Cause Acquiescence Response Bias in Survey Questions? Jon A. Krosnick. Ana Villar. and. Bo MacInnis. Stanford University Do Introductory Sentences Cause Acquiescence Response Bias in Survey Questions? Jon A. Krosnick Ana Villar and Bo MacInnis Stanford University August 2011 Jon Krosnick is University Fellow at Resources

More information

Practicability of Including Cell Phone Numbers in Random Digit Dialed Surveys: Pilot Study Results from the Behavioral Risk Factor Surveillance System

Practicability of Including Cell Phone Numbers in Random Digit Dialed Surveys: Pilot Study Results from the Behavioral Risk Factor Surveillance System Practicability of Including Cell Phone Numbers in Random Digit Dialed Surveys: Pilot Study Results from the Behavioral Risk Factor Surveillance System Michael W. Link 1, Michael P. Battaglia 2, Martin

More information

The Impact of Cell Phone Non-Coverage Bias on Polling in the 2004 Presidential Election

The Impact of Cell Phone Non-Coverage Bias on Polling in the 2004 Presidential Election The Impact of Cell Phone Non-Coverage Bias on Polling in the 2004 Presidential Election Scott Keeter Pew Research Center August 1, 2005 Abstract Despite concerns that the accuracy of pre-election telephone

More information

The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys

The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys Barbara Lepidus Carlson 1 and Richard Strouse 2 Mathematica Policy Research, Inc. 1 Battelle 2 Abstract Declining

More information

Pre-Call Validation of RDD Cell Phone Numbers A Field Experiment

Pre-Call Validation of RDD Cell Phone Numbers A Field Experiment Pre-Call Validation of RDD Cell Phone Numbers A Field Experiment Tanja Kunz 1, Marek Fuchs 1 1 Institute of Sociology, Darmstadt University of Technology Residenzschloss, 64283 Darmstadt (Germany) Abstract

More information

35% of those ages 16 and older own tablet computers

35% of those ages 16 and older own tablet computers www.pewresearch.org OCTOBER 18, 2013 35% of those ages 16 and older own tablet computers Up from 25% last year more than half of those in households earning $75,000 or more now have tablets 24% own e-book

More information

Overview of Proposed Changes to BRFSS Cell Phone Sampling Frame

Overview of Proposed Changes to BRFSS Cell Phone Sampling Frame Overview of Proposed Changes to BRFSS Cell Phone Sampling Frame When collecting data using RDD, there are several options for sampling that include landline and cell phone users. The first option is to

More information

Politics on Social Networking Sites

Politics on Social Networking Sites SEPTEMBER 4, 2012 Politics on Social Networking Sites Campaign and policy-related material on SNS plays a modest role in influencing most users views and political activities. Democrats and liberals are

More information

USES OF CONSUMER PRICE INDICES

USES OF CONSUMER PRICE INDICES USES OF CONSUMER PRICE INDICES 2 2.1 The consumer price index (CPI) is treated as a key indicator of economic performance in most countries. The purpose of this chapter is to explain why CPIs are compiled

More information

A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey

A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey Daifeng Han and David Cantor Westat, 1650 Research Blvd., Rockville, MD 20850 Abstract One method

More information

THE JOINT HARMONISED EU PROGRAMME OF BUSINESS AND CONSUMER SURVEYS

THE JOINT HARMONISED EU PROGRAMME OF BUSINESS AND CONSUMER SURVEYS THE JOINT HARMONISED EU PROGRAMME OF BUSINESS AND CONSUMER SURVEYS List of best practice for the conduct of business and consumer surveys 21 March 2014 Economic and Financial Affairs This document is written

More information

Survey Research. Classifying surveys on the basis of their scope and their focus gives four categories:

Survey Research. Classifying surveys on the basis of their scope and their focus gives four categories: Survey Research Types of Surveys Surveys are classified according to their focus and scope (census and sample surveys) or according to the time frame for data collection (longitudinal and cross-sectional

More information

NON-PROBABILITY SAMPLING TECHNIQUES

NON-PROBABILITY SAMPLING TECHNIQUES NON-PROBABILITY SAMPLING TECHNIQUES PRESENTED BY Name: WINNIE MUGERA Reg No: L50/62004/2013 RESEARCH METHODS LDP 603 UNIVERSITY OF NAIROBI Date: APRIL 2013 SAMPLING Sampling is the use of a subset of the

More information

How to do a Survey (A 9-Step Process) Mack C. Shelley, II Fall 2001 LC Assessment Workshop

How to do a Survey (A 9-Step Process) Mack C. Shelley, II Fall 2001 LC Assessment Workshop How to do a Survey (A 9-Step Process) Mack C. Shelley, II Fall 2001 LC Assessment Workshop 1. Formulate the survey keeping in mind your overall substantive and analytical needs. Define the problem you

More information

MEMORANDUM FOR THE PRESIDENT S MANAGEMENT COUNCIL. Guidance on Agency Survey and Statistical Information Collections

MEMORANDUM FOR THE PRESIDENT S MANAGEMENT COUNCIL. Guidance on Agency Survey and Statistical Information Collections EXECUTIVE OFFICE OF THE PRESIDENT OFFICE OF MANAGEMENT AND BUDGET WASHINGTON, D.C. 20503 ADMINISTRATOR OFFICE OF INFORMATION AND REGULATORY AFFAIR January 20, 2006 MEMORANDUM FOR THE PRESIDENT S MANAGEMENT

More information

Sampling. COUN 695 Experimental Design

Sampling. COUN 695 Experimental Design Sampling COUN 695 Experimental Design Principles of Sampling Procedures are different for quantitative and qualitative research Sampling in quantitative research focuses on representativeness Sampling

More information

Kaiser Family Foundation/New York Times Survey of Chicago Residents

Kaiser Family Foundation/New York Times Survey of Chicago Residents Kaiser Family Foundation/New York Times Survey of Chicago Residents Selected Findings May 2016 Figure 1 Majority of Chicago Residents Feel City is on the Wrong Track Do you feel things in Chicago are generally

More information

Self-Check and Review Chapter 1 Sections 1.1-1.2

Self-Check and Review Chapter 1 Sections 1.1-1.2 Self-Check and Review Chapter 1 Sections 1.1-1.2 Practice True/False 1. The entire collection of individuals or objects about which information is desired is called a sample. 2. A study is an observational

More information

Americans and text messaging

Americans and text messaging Americans and text messaging 31% of text message users prefer texting to voice calls, and young adults stand out in their use of text messaging Aaron Smith, Senior Research Specialist 9/19/2011 http://pewinternet.org/reports/2011/cell-phone-texting-2011.aspx

More information

Guidelines & Considerations: Telephone Surveys of Respondents Reached via Cell Phone

Guidelines & Considerations: Telephone Surveys of Respondents Reached via Cell Phone Guidelines & Considerations: Telephone Surveys of Respondents Reached via Cell Phone AAPOR Cell Phone Task Force: Paul J. Lavrakas Chair, Charlotte Steeh, Stephen Blumberg, John Boyle, Michael Brick, Mario

More information

surveys that is rarely discussed: the polls reported in the lead stories of the nation s leading

surveys that is rarely discussed: the polls reported in the lead stories of the nation s leading The Economist/YouGov Internet Presidential Poll Morris Fiorina and Jon Krosnick * In 1936, George Gallup correctly forecast Franklin Roosevelt s presidential landslide at a time when one of the country

More information

Meeting the Challenges of Designing the Kauffman Firm Survey: Sampling Frame, Definitions, Questionnaire Development, and Respondent Burden

Meeting the Challenges of Designing the Kauffman Firm Survey: Sampling Frame, Definitions, Questionnaire Development, and Respondent Burden Meeting the Challenges of Designing the Kauffman Firm Survey: Sampling Frame, Definitions, Questionnaire Development, and Respondent Burden Janice Ballou 1, David DesRoches 1, Zhanyun Zhao 1, and Frank

More information

Missing data in randomized controlled trials (RCTs) can

Missing data in randomized controlled trials (RCTs) can EVALUATION TECHNICAL ASSISTANCE BRIEF for OAH & ACYF Teenage Pregnancy Prevention Grantees May 2013 Brief 3 Coping with Missing Data in Randomized Controlled Trials Missing data in randomized controlled

More information

POLLING STANDARDS. The following standards were developed by a committee of editors and reporters and should be adhered to when using poll results.

POLLING STANDARDS. The following standards were developed by a committee of editors and reporters and should be adhered to when using poll results. ! POLLING STANDARDS June, 2006 The following standards were developed by a committee of editors and reporters and should be adhered to when using poll results. OVERVIEW Reporting on polls is no different

More information

Questionnaire Design in Telephone Surveys: Interviewers and Research Call Center Managers Experience

Questionnaire Design in Telephone Surveys: Interviewers and Research Call Center Managers Experience Questionnaire Design in Telephone Surveys: Interviewers and Research Call Center Managers Experience Wojciech Jablonski University of Lodz Poland wjablonski@uni.lodz.pl AAPOR 69th Annual Conference Anaheim,

More information

Telephone Surveys and the Mobile Phone Only Population Workshop 17, July, 2012.

Telephone Surveys and the Mobile Phone Only Population Workshop 17, July, 2012. Telephone Surveys and the Mobile Phone Only Population Workshop 17, July, 2012. The 2011 Workshop Day 1 Integrated Public Number Database (Michel Ryan) Is the exclusion of mobile phones from telephone

More information

COMMENT GARY LANGER* Journal of Survey Statistics and Methodology

COMMENT GARY LANGER* Journal of Survey Statistics and Methodology Journal of Survey Statistics and Methodology COMMENT GARY LANGER* The debates of another era echo mightily in the report of the AAPOR Task Force on Non-Probability Sampling. Were they with us, Hansen and

More information

Alcohol, CAPI, drugs, methodology, non-response, online access panel, substance-use prevalence,

Alcohol, CAPI, drugs, methodology, non-response, online access panel, substance-use prevalence, METHODS AND TECHNIQUES doi:10.1111/j.1360-0443.2009.02642.x The utility of online panel surveys versus computerassisted interviews in obtaining substance-use prevalence estimates in the Netherlandsadd_2642

More information

2014 MASSACHUSETTS HEALTH INSURANCE SURVEY METHODOLOGY REPORT

2014 MASSACHUSETTS HEALTH INSURANCE SURVEY METHODOLOGY REPORT CENTER FOR HEALTH INFORMATION AND ANALYSIS 2014 MASSACHUSETTS HEALTH INSURANCE SURVEY METHODOLOGY REPORT MAY 2015 Prepared by: David Dutwin and Susan Sherr, SSRS Sharon K. Long, Urban Institute Contents

More information

THE EFFECTIVENESS OF SHORT-TERM STUDY ABROAD PROGRAMS IN THE DEVELOPMENT OF INTERCULTURAL COMPETENCE AMONG PARTICIPATING CSU STUDENTS.

THE EFFECTIVENESS OF SHORT-TERM STUDY ABROAD PROGRAMS IN THE DEVELOPMENT OF INTERCULTURAL COMPETENCE AMONG PARTICIPATING CSU STUDENTS. THE EFFECTIVENESS OF SHORT-TERM STUDY ABROAD PROGRAMS IN THE DEVELOPMENT OF INTERCULTURAL COMPETENCE AMONG PARTICIPATING CSU STUDENTS Rafael Carrillo Abstract Study abroad programs have recently been shortening

More information

The Relationship between the Fundamental Attribution Bias, Relationship Quality, and Performance Appraisal

The Relationship between the Fundamental Attribution Bias, Relationship Quality, and Performance Appraisal The Relationship between the Fundamental Attribution Bias, Relationship Quality, and Performance Appraisal Executive Summary Abstract The ability to make quality decisions that influence people to exemplary

More information

DEPARTMENT OF AGRICULTURE. Food and Nutrition Service. Agency Information Collection Activities: Proposed Collection; Comment Request

DEPARTMENT OF AGRICULTURE. Food and Nutrition Service. Agency Information Collection Activities: Proposed Collection; Comment Request This document is scheduled to be published in the Federal Register on 07/15/2014 and available online at http://federalregister.gov/a/2014-16586, and on FDsys.gov BILLING CODE: 3410-30-P DEPARTMENT OF

More information

Introduction to Quantitative Research Contact: tel 01296 680374

Introduction to Quantitative Research Contact: tel 01296 680374 Introduction to Quantitative Research Quantitative Research Quantification - i.e. numbers e.g 51% of the population is female 74% of households have a washing machine 33% strongly agree with the statement.

More information

THE FUTURE OF INTERNET-BASED SURVEY METHODS

THE FUTURE OF INTERNET-BASED SURVEY METHODS Chapter Seven CONCLUSIONS In this chapter, we offer some concluding thoughts on the future of Internet-based surveys, the issues surrounding the use of e-mail and the Web for research surveys, and certain

More information

Use of Text Messaging to Increase Response Rates

Use of Text Messaging to Increase Response Rates Use of Text Messaging to Increase Response Rates Key words: BRFSS, response rate, cell phones, smart phones, text message Prepared by: Piper DuBray, Kristie Healey ICF International 0 Introduction Survey

More information

SURVEY RESEARCH AND RESPONSE BIAS

SURVEY RESEARCH AND RESPONSE BIAS SURVEY RESEARCH AND RESPONSE BIAS Anne G. Scott, Lee Sechrest, University of Arizona Anne G. Scott, CFR, Educ. Bldg. Box 513, University of Arizona, Tucson, AZ 85721 KEY WORDS: Missing data, higher education

More information

Sampling: What is it? Quantitative Research Methods ENGL 5377 Spring 2007

Sampling: What is it? Quantitative Research Methods ENGL 5377 Spring 2007 Sampling: What is it? Quantitative Research Methods ENGL 5377 Spring 2007 Bobbie Latham March 8, 2007 Introduction In any research conducted, people, places, and things are studied. The opportunity to

More information

SAMPLING INDIVIDUALS WITHIN HOUSEHOLDS IN TELEPHONE SURVEYS

SAMPLING INDIVIDUALS WITHIN HOUSEHOLDS IN TELEPHONE SURVEYS SAMPLING INDIVIDUALS WITHIN HOUSEHOLDS IN TELEPHONE SURVEYS Gtista Forsman, University of Linktiping Department of Mathematics, University of Linktiping, S-58183 Link6ping, Sweden Key Words: Sampling,

More information

Welcome & Introductions. A Study of Population Health. What Is the Family Health Survey?

Welcome & Introductions. A Study of Population Health. What Is the Family Health Survey? The Bureau of Health Information and Policy presents Welcome & Introductions The Wisconsin Family Health Survey: A Study of Population Health Christine Hill, Section Chief Bureau of Health Information

More information

Can post-stratification adjustments correct bias in traditional RDD estimates

Can post-stratification adjustments correct bias in traditional RDD estimates Can post-stratification adjustments correct bias in traditional RDD estimates Kathleen Thiede Call SHADAC State Survey Workshop, Washington DC January 13, 2009 Funded by a grant from the Robert Wood Johnson

More information

Research into Issues Surrounding Human Bones in Museums Prepared for

Research into Issues Surrounding Human Bones in Museums Prepared for Research into Issues Surrounding Human Bones in Museums Prepared for 1 CONTENTS 1. OBJECTIVES & RESEARCH APPROACH 2. FINDINGS a. Visits to Museums and Archaeological Sites b. Interest in Archaeology c.

More information

Insurance Markets Ready or Not: Consumers Face New Health Insurance Choices. Employer-based. Insurance Premium. Contribution.

Insurance Markets Ready or Not: Consumers Face New Health Insurance Choices. Employer-based. Insurance Premium. Contribution. Insurance Markets Ready or Not: Consumers Face New Health Insurance Choices Introduction Not long ago, most working Californians, at least those working for large or midsize companies, could expect a standard

More information

Tablet Ownership 2013

Tablet Ownership 2013 www.pewresearch.org JUNE 10, 2013 Tablet Ownership 2013 Tablet adoption has almost doubled over the past year. For the first time a third (34%) of American adults now own a tablet computer, including almost

More information

17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device

17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device JUNE 26, 2012 17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device Most do so for convenience, but for some their phone is their only option

More information

Non-response bias in a lifestyle survey

Non-response bias in a lifestyle survey Journal of Public Health Medicine Vol. 19, No. 2, pp. 203-207 Printed in Great Britain Non-response bias in a lifestyle survey Anthony Hill, Julian Roberts, Paul Ewings and David Gunnell Summary Background

More information

The AmeriSpeak ADVANTAGE

The AmeriSpeak ADVANTAGE OVERVIEW Funded and operated by NORC at the University of Chicago, AmeriSpeak TM is a probabilitybased panel (in contrast to a non-probability panel). Randomly selected households are sampled with a known,

More information

9. Data-Collection Tools

9. Data-Collection Tools 9. Data-Collection Tools Depending on the nature of the information to be gathered, different instruments are used to conduct the assessment: forms for gathering data from official sources such as police

More information

Optimizing Call Patterns for Landline and Cell Phone Surveys

Optimizing Call Patterns for Landline and Cell Phone Surveys Optimizing Call Patterns for Landline and Cell Phone Surveys Becky Reimer 1, Veronica Roth 1, and Robert Montgomery 1 1 NORC at the University of Chicago, 55 East Monroe Street, Suite 3000, Chicago, IL

More information

Fitting a Probability-Based Online Panel to Interview Questions of American Jews

Fitting a Probability-Based Online Panel to Interview Questions of American Jews Vol. 5, no 3, 2012 www.surveypractice.org The premier e-journal resource for the public opinion and survey research community Surveying Rare Populations Using a Probabilitybased Online Panel Graham Wright

More information

Question Bank FACTSHEET 2. Methods of Data Collection in Social Surveys

Question Bank FACTSHEET 2. Methods of Data Collection in Social Surveys Question Bank FACTSHEET 2 Methods of Data Collection in Social Surveys Keywords: Data, Structured Interview, Self completion, Web collection. Introduction This fact sheet will discuss the main methods

More information

Lessons Learned From an NHIS Web- Followback Feasibility Study

Lessons Learned From an NHIS Web- Followback Feasibility Study Lessons Learned From an NHIS Web- Followback Feasibility Study Sarah S. Joestl, DrPH Senior Service Fellow Survey Planning and Special Surveys Branch Division of Health Interview Statistics Division of

More information