Telephone and Mail Surveys: Advantages and Disadvantages of Each

Size: px
Start display at page:

Download "Telephone and Mail Surveys: Advantages and Disadvantages of Each"

Transcription

1 Telephone and Mail Surveys: Advantages and Disadvantages of Each Dan Zahs and Reg Baker Market Strategies, Inc. May 29, 2007 Selecting the mode of administration for a survey requires that one evaluate a number of factors and understand clearly the tradeoffs involved in choosing one mode over another. While telephone and mail research share some similar qualities, there are major differences. We group these differences into four main categories: (1) sample frame; (2) non-response bias; (3) measurement error; and (4) time and money. The Sample Frame A sample frame is essentially a list used to select the sample of persons to be interviewed. A high quality frame is one that contains a complete or nearly complete list of the target population. If the frame is sufficiently complete and accurate, a random sample selected from this frame will be non-biased and representative of the target population. One major reason for the adoption of telephone as the gold standard for much of commercial research over the last 25 years has been the quality of the sample frame. The standard frame for RDD telephone surveys uses information from the telephone companies about the assignment status of groups of telephone numbers (known as groups and blocks). Until recently, this frame was considered largely complete and accurate, with a relatively small number of missing non-telephone households. More recently, the widespread adoption of cell phones has threatened the integrity of the RDD frame. For various reasons, cell phones are not part of this standard frame. Until recently, most people with a cell phone also had a traditional telephone or landline. However, the proportion households having a cell phone but no landline has been growing rapidly and is now around 13 percent (Blumberg and Luke (2007)). Using the standard RDD frame, these cell-only households have no chance of being selected into any sample. When cases are excluded from the possibility of being selected, bias might be introduced into the results. This is known as coverage error. It is possible to include the cell-only population into the sample selection process through the use of a dual frame approach, but this can add significant cost and poses some difficult weighting challenges. Fortunately, recent research suggests that at this point the exclusion of cell-only households does not lead to substantial levels of bias, that is, there are few significant differences in overall estimates produced from samples with and without cell-only households (See, for example, Keeter (2006) and Brick et al. (2006)). That said, cell-only households are disproportionately comprised of year olds and so there is some demographic bias in samples that exclude those households. Research has found that persons in cell-only households differ from landline households on some important health measures such as smoking and binge drinking (Blumberg and Luke 1

2 (2007)). Studies with a special focus on that age group can have substantial bias if cellonly households are not included. Thus far, this group is not so large as to significantly affect estimates for the overall population. The availability of sample frames for mail studies is also an unfolding story. Historically, mail studies of the general population have tended to use a so-called listed frame compiled from various sources including the directories of people listed in the white pages of telephone books. These listed frames are generally incomplete and have high levels of coverage error. For example, a match of an RDD sample to a list of people in a telephone book typically yields about a 50 percent match rate. More recently, a new frame option has emerged and is still being evaluated by survey methodologists. This frame is maintained by the U.S. Postal Service and is called the Delivery Sequence File (DSF). As the name suggests, this is the database the USPS uses for their deliveries. This frame appears to be much more comprehensive and complete than previous mail frames, although early studies suggest that there still may be significant coverage error, specifically of non-urban and lower income areas (Link and Mokdad (2005)). Non-Response Bias The potential for bias in a survey results is not just a function of coverage (i.e. the quality of the frame). Non-response also is a major factor. One hundred percent response may be an admirable goal, but it is almost never achieved. Large proportions of any selected sample, regardless of mode, typically do not respond. This non-response tends to be systematic rather than random, and therefore bias is introduced. Put in simple terms, the non-response bias question is: Would the estimates from my survey be different if everyone had responded? Put another way: Are those who did not respond different from those who did in some systematic way that biases my results? Historically, telephone surveys have yielded significantly higher response rates (and therefore lower non-response) than mail, although the achieved response rate in any mode is a function of the implementation techniques used (e.g., advance letters, incentives, refusal conversion, etc.) Telephone surveys typically under represent people who are difficult to catch at home (e.g., younger, single, or non-family households) while mail surveys have tended to under represent lower SES households. In both instances, post stratification weights are used to bring the achieved sample back in line with population demographics. Over about the last decade we have seen significant declines in response rates to all modes. The impact on the validity of telephone research has been particularly studied because of its widespread use. Recent studies have demonstrated that response rate is not as important a measure of survey data quality as was once thought (See, for example, Holbrook et al. (in press) and Keeter et al. (2006)). Surveys with low response rates (as low as four percent in one comparison) can yield results that are statistically equivalent to those from surveys with much higher response rates, although a high response rate is usually better than a low one. This is interpreted as a validation of the power or probability sampling from good quality sample frames. 2

3 Measurement Error Measurement error can have any number of sources including the interviewer, the questionnaire, the mode of administration, and the respondent. Interviewers may be poorly trained and administer the questionnaire incorrectly. The questionnaire may be flawed in some important way that leads to comprehension problems or misunderstanding. The presence or absence of an interviewer may influence how a respondent answers. The respondent may lack the cognitive ability to comprehend and answer the questions or simply lack the motivation to answer carefully. Generally speaking, the advantage of well-trained interviewers has led researchers to prefer telephone over mail. Interviewers ensure that the correct target respondent within a household completes the survey; administer the survey so all questions are asked; help respondents understand questions or concepts that might be ambiguous or difficult; and keep the respondent engaged over the duration of the survey. They also can probe respondents to get fuller and more accurate answers in open ended questions. One classic statistic here is item non-response, that is, the proportion of missing data in individual questions (skipped questions, don t know or refusal to answer). Telephone studies typically have significantly lower item non-response than self-administered modes such as mail and Web. The combination of well-trained interviewers and computer-assisted interviewing technologies (such as CATI) have made it possible to design and administer very complex questionnaires that tailor the interview to each respondent, prevent routing errors, reduce data recording errors, maintain consistency in respondent answers, and use design techniques such as randomization and rotation to reduce order and context effects. Telephone surveys also lend themselves to interviewing in multiple languages. Mail surveys by virtue of their self-administered, paper-and-pencil format are much more limited. Questionnaires are generally shorter, simpler, have few skips, and must be crafted to be as clear and unambiguous as possible. The researcher has less control over who completes a mail survey and the order in which questions are answered. Recording errors are much more common than with interviewer administration. Commonly used techniques such as randomization, rotation, and multiple language interviewing are extremely difficult to implement. Despite their many benefits, interviewers can also be a source of error if they are not well-trained and monitored. Further, the presence of the interviewer can sometimes lead to under reporting of socially sensitive behaviors (e.g., drug use or binge drinking), a phenomenon know as social desirability bias. The principal advantage of mail surveys in terms of measurement error is the potential reduction in social desirability bias. Time and Money Arguably the simplest comparison of the two modes focuses on cost and length of field period. Mail surveys are almost always less expensive than telephone because the cost of 3

4 mailing and data conversion (keying or scanning) are significantly less than the cost of interviewer labor. Mail surveys typically require an incentive to achieve any sort of reasonable response rate, and depending on the magnitude of that incentive mail surveys can sometimes approach the cost of telephone surveys where incentives are often not offered except where very high response rates are necessary. On the other hand, mail surveys generally require much longer field periods, and the researcher s ability to either predict or control this key condition is very limited when compared to telephone. Summary As we hope the foregoing makes clear, telephone surveys have a broad set of advantages over mail. Further, these advantages are generally recognized by researchers and comprise the main reason why RDD telephone surveys have been consistently viewed as the methodology of choice for high quality, general population surveys. Generally speaking, mail surveys have tended to be viewed as a compromise that sacrifices data quality in return for lower costs. At present, however, there are dynamics at work relative to sample frames that could undermine the dominant role that telephone surveys have traditionally enjoyed. The rapid rise of cell-only households has begun to introduce coverage problems into the traditional RDD sample frame. The most likely remedies will increase the cost of telephone research significantly. At the same time, the emergence of a new and possibly more complete mail sample frame in the form of the DFS may create a significant coverage advantage for mail surveys. Issues of coverage area and cost aside, the combination of interviewer administration and modern CATI technology continue to present and extremely powerful data quality argument for telephone research. References Blumberg, S.J., and Luke, J.V Wireless Substitution: Early Release Estiamtes Based on Data from the National Health Interview Survey, July December 2006, National Center for Health Statistics. Brick, M. J., Dipko, S., Presser, S., Tucker, C., andyuan, Y Nonresponse Bias in a Dual Frame Sample of Cell and Landline Numbers, Public Opinion Quarterly, 70: Holbrook, A. L., Krosnick, J.A., and Pfent, A.M. In Press. Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms, in J. 4

5 Lepkowski, B. Harris-Kojetin, P.J. Lavrakas, C. Tucker, E. de Leeuw, M. Link, M. Brick, L. Japec, and R. Sangster (eds.), Telephone Survey Methodology. New York: Wiley. Keeter, S The Impact of Cell Phone Noncoverage Bias on Polling in the 2004 Presidential Election, Public Opinion Quarterly, 70: Keeter, S., Kennedy, C., Dimock, M., Best, J., and Craighill, P Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey, Public Opinion Quarterly, 70: Link, M.W., Mokdad, A.H Use of Alternative Modes for Health Surveillance surveys: Results from a Web/Mail/Telephone Experiment, Epidemiology, 16:

Jeff Jackson Sarah Likshis Devon Sheppard

Jeff Jackson Sarah Likshis Devon Sheppard James Hedges James Hedges Jeff Jackson Sarah Likshis Devon Sheppard Introduction Telephone surveying is defined as a systematic collection of data from a sample population using a standardized questionnaire.

More information

APPENDIX A: SURVEY METHODOLOGY

APPENDIX A: SURVEY METHODOLOGY 55 APPENDIX A: SURVEY METHODOLOGY The survey was conducted by telephone with a national sample of adults (18 years of age or older) living in all 50 U.S. states and the District of Columbia. The results

More information

The Impact of Wireless-only Households on State Surveys of Health Insurance Coverage

The Impact of Wireless-only Households on State Surveys of Health Insurance Coverage The Impact of Wireless-only Households on State Surveys of Health Insurance Coverage March 2009 Introduction Survey estimates of the distribution of health insurance coverage are critical for informing

More information

Integrating Cell Phone Numbers into Random Digit-Dialed (RDD) Landline Surveys

Integrating Cell Phone Numbers into Random Digit-Dialed (RDD) Landline Surveys Integrating Cell Phone Numbers into Random Digit-Dialed (RDD) Landline Surveys Martin R. Frankel 1, Michael P. Battaglia 2, Michael Link 3, and Ali H. Mokdad 3 1 Baruch College, City University of New

More information

Cell Phone Sampling Summit II Proposed Statements on Accounting for Cell Phones in Telephone Survey Research in the U.S.

Cell Phone Sampling Summit II Proposed Statements on Accounting for Cell Phones in Telephone Survey Research in the U.S. Cell Phone Sampling Summit II Proposed Statements on Accounting for Cell Phones in Telephone Survey Research in the U.S. Prepared by Paul J. Lavrakas and Charles D. Shuttles In February 2005, a meeting

More information

Cell Phones and Nonsampling Error in the American Time Use Survey October 2012

Cell Phones and Nonsampling Error in the American Time Use Survey October 2012 Cell Phones and Nonsampling Error in the American Time Use Survey October 2012 Brian Meekins, Stephanie Denton 1,i 1 Bureau of Labor Statistics Abstract Recent research on the impact of cell phones has

More information

Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts

Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts Alternate Strategies for Including Cell Phone Only Households in Surveys: Comparing the Use of ABS and Cell Phone RDD Samples in Massachusetts David Dutwin, PhD State Health Research Policy Interest Group

More information

NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE DECEMBER 30, 2013

NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE DECEMBER 30, 2013 NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FOR RELEASE DECEMBER 30, 2013 Maeve Duggan, Research Assistant Aaron Smith, Senior Researcher 202.419.4500 RECOMMENDED CITATION: Maeve Duggan and Aaron Smith,

More information

Addressing the Cell Phone-Only Problem: Cell Phone Sampling Versus Address Based Sampling

Addressing the Cell Phone-Only Problem: Cell Phone Sampling Versus Address Based Sampling Addressing the Cell Phone-Only Problem: Cell Phone Sampling Versus Address Based Sampling Michael W. Link 1, Gail Daily 2, Charles D. Shuttles 3, H. Christine Bourquin 2, and L. Tracie Yancey 2 1 The Nielsen

More information

24% of internet users have made phone calls online

24% of internet users have made phone calls online 24% of internet users have made phone calls online Lee Rainie, Director May 30, 2011 http://pewinternet.org/reports/2011/13--internet-phone-calls--skype.aspx Pew Research Center s Internet & American Life

More information

Recruiting Probability-Based Web Panel Members Using an Address-Based Sample Frame: Results from a Pilot Study Conducted by Knowledge Networks

Recruiting Probability-Based Web Panel Members Using an Address-Based Sample Frame: Results from a Pilot Study Conducted by Knowledge Networks Recruiting Probability-Based Web Panel Members Using an Address-Based Sample Frame: Results from a Pilot Study Conducted by Knowledge Networks Charles DiSogra, Mario Callegaro, Erlina Hendarwan Knowledge

More information

Self-administered Questionnaire Survey

Self-administered Questionnaire Survey Primary methods of survey data collection Self-administered questionnaire (SAQ) Mail Face-to-face interview Telephone interview Self-administered Questionnaire Survey 1 Steps are: a. Mail distribution

More information

An Evaluation of Ported Telephone Numbers in the 2013 California Health Interview Survey

An Evaluation of Ported Telephone Numbers in the 2013 California Health Interview Survey An Evaluation of Ported Telephone Numbers in the 2013 California Health Interview Survey Greg Norman 1 and Ismael Flores Cervantes 1 1 Westat, 1600 Research Boulevard, Rockville, MD 20850 Abstract In telephone

More information

E-reader Ownership Doubles in Six Months

E-reader Ownership Doubles in Six Months E-reader Ownership Doubles in Six Months Adoption rate of e-readers surges ahead of tablet computers Kristen Purcell, Associate Director for Research, Pew Internet Project June 27, 2011 Pew Research Center

More information

Search and email still top the list of most popular online activities

Search and email still top the list of most popular online activities Search and email still top the list of most popular online activities Two activities nearly universal among adult internet users Kristen Purcell, Associate Director for Research, Pew Internet Project August

More information

Cell Phone Reverse Directories Promise or Peril?

Cell Phone Reverse Directories Promise or Peril? Cell Phone Reverse Directories Promise or Peril? Sarah Dipko 1 and Krista Freedman 1 1 Westat Key Words: Cell phones, reverse directories, advance notification, sample purging 1. Introduction Surveys of

More information

Developing a Sustainable and Flexible Long-Distance Travel Survey

Developing a Sustainable and Flexible Long-Distance Travel Survey Developing a Sustainable and Flexible Long-Distance Travel Survey Lee Giesbrecht and Jonaki Bose Bureau of Transportation Statistics, 400 Seventh Street, SW, Washington, DC 20590 Abstract The Bureau of

More information

Mobile Phone Problems

Mobile Phone Problems AUGUST 2, 2012 Mobile Phone Problems Telemarketing calls and spam texts are realities for most cell phone users. Smartphone owners are particularly likely to report dropped phone calls and slow download

More information

Generations and their gadgets

Generations and their gadgets Generations and their gadgets By Kathryn Zickuhr, Web Coordinator February 3, 2011 http://pewinternet.org/reports/2011/generations-and-gadgets.aspx Pew Research Center s Internet & American Life Project

More information

Survey Research. Writing Questionnaire Items. Likert-type Scales. Label the endpoints only Label endpoints and middle Label every point on the scale

Survey Research. Writing Questionnaire Items. Likert-type Scales. Label the endpoints only Label endpoints and middle Label every point on the scale Survey Research Claudia J. Stanny PSY 6217 Research Design Writing Questionnaire Items Open-ended items Partially open-ended items Closed (Restricted) items Provide a choice for all potential responses

More information

COSTS AND BENEFITS OF FULL DUAL-FRAME TELEPHONE SURVEY DESIGNS

COSTS AND BENEFITS OF FULL DUAL-FRAME TELEPHONE SURVEY DESIGNS COSTS AND BENEFITS OF FULL DUAL-FRAME TELEPHONE SURVEY DESIGNS Prepared for the annual meeting of the American Association for Public Opinion Research New Orleans, Louisiana May 15-18, 2008 Scott Keeter,

More information

Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse. Andy Peytchev, Lisa Carley-Baxter, and Michele Lynberg Black

Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse. Andy Peytchev, Lisa Carley-Baxter, and Michele Lynberg Black Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse Andy Peytchev, Lisa Carley-Baxter, and Michele Lynberg Black RTI, International Centers for Disease Control and Prevention

More information

The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes

The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes The Impact of Landline and Cell Phone Usage Patterns among Young Adults on RDD Survey Outcomes Douglas Currivan 1, David Roe 2, and Jason Stockdale 1 1 RTI International 2 Survey Sciences Group, LLC May

More information

Inter-mode comparability in telephone surveys via landline and mobile phones

Inter-mode comparability in telephone surveys via landline and mobile phones Inter-mode comparability in telephone surveys via landline and mobile phones Working paper June 2010 Authors Mike Kühne Dresden University of Technology mike.kuehne@tu-dresden.de Alexander Wentland Social

More information

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES Public Opinion Quarterly, Vol. 71, No. 1, Spring 2007, pp. 3 22 HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE UNITED STATES IN 2004: IMPLICATIONS FOR TELEPHONE SAMPLES CLYDE TUCKER J. MICHAEL BRICK

More information

Improving Survey Response in Spanish-speaking Households. 126 College Street, Burlington, VT 05401,

Improving Survey Response in Spanish-speaking Households. 126 College Street, Burlington, VT 05401, Improving Survey Response in Spanish-speaking Households James Dayton 1*, Michael W. Link 2, Robert Pels 1, and Kirsten Ivie 1 1 ORC-Macro International, 2 Centers for Disease Control and Prevention *James.J.Dayton@orcmacro.com,

More information

Sample: n=6,224 people age 16 or older nationwide, including 3,102 cell phone interviews Interviewing dates:

Sample: n=6,224 people age 16 or older nationwide, including 3,102 cell phone interviews Interviewing dates: Survey questions Gates Libraries Typology Survey Final Topline 10/2/2013 Data for July 18 September 30, 2013 Princeton Survey Research Associates International for the Pew Research Center s Internet &

More information

Address-Based Sampling (ABS) Merits, Design, and Implementation

Address-Based Sampling (ABS) Merits, Design, and Implementation Address-Based Sampling (ABS) Merits, Design, and Implementation Mansour Fahimi, Ph.D. VP, Statistical Research Services National Conference on Health Statistics National Center for Health Statistics (NCHS)

More information

JSM 2013 - Survey Research Methods Section

JSM 2013 - Survey Research Methods Section How Representative are Google Consumer Surveys?: Results from an Analysis of a Google Consumer Survey Question Relative National Level Benchmarks with Different Survey Modes and Sample Characteristics

More information

A Review of the Cell Phone Sample Component of the California Health Interview Survey

A Review of the Cell Phone Sample Component of the California Health Interview Survey A Review of the Cell Phone Sample Component of the California Health Interview Survey J. Michael Brick 1, Ismael Flores Cervantes 1, Jennifer Kali 1 Greg Norman 1, David Grant 2 1 Westat, 1600 Research

More information

Appendix Methodology and Statistics

Appendix Methodology and Statistics Appendix Methodology and Statistics Introduction Science and Engineering Indicators (SEI) contains data compiled from a variety of sources. This appendix explains the methodological and statistical criteria

More information

Ownership and Usage Patterns of Cell Phones: 2000-2005

Ownership and Usage Patterns of Cell Phones: 2000-2005 Ownership and Usage Patterns of Cell Phones: 2000-2005 Peter Tuckel, Department of Sociology, Hunter College Harry O Neill, Roper Public Affairs, NOP World Key Words: Cell phones, telephone surveys, Nonresponse

More information

Sample: n=2,252 national adults, age 18 and older, including 1,127 cell phone interviews Interviewing dates: 04.17.2013 05.19.2013

Sample: n=2,252 national adults, age 18 and older, including 1,127 cell phone interviews Interviewing dates: 04.17.2013 05.19.2013 Survey Questions Spring 2013 Tracking Survey Final Topline 5/21/2013 Data for April 17-May 19, 2013 Princeton Survey Research Associates International for the Pew Research Center s Internet & American

More information

Americans and their gadgets

Americans and their gadgets Americans and their gadgets By Aaron Smith, Senior Research Specialist October 14, 2010 http://pewinternet.org/reports/2010/gadgets.aspx Pew Research Center s Internet & American Life Project 1615 L St.,

More information

Types of Error in Surveys

Types of Error in Surveys 2 Types of Error in Surveys Surveys are designed to produce statistics about a target population. The process by which this is done rests on inferring the characteristics of the target population from

More information

The Demographics of Social Media Users 2012

The Demographics of Social Media Users 2012 www.pewresearch.org FEBRUARY 14, 2013 The Demographics of Social Media Users 2012 Young adults are more likely than older adults to use social media. Women, African- Americans, and Latinos show high interest

More information

Internet, broadband, and cell phone statistics

Internet, broadband, and cell phone statistics Internet, broadband, and cell phone statistics As of December 2009, 74% of American adults (ages 18 and older) use the internet. January 2010 Lee Rainie Director View Report Online: http://www.pewinternet.org/reports/2010/internet-broadband-and-cell-phone-statistics.aspx

More information

APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS

APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS APPENDIX V METHODOLOGY OF 2011 MONTANA HEALTH INSURANCE SURVEYS The purpose of the 2011 Health Insurance Surveys is a: Study of the insured, underinsured and uninsured Montanans Study of Montana s current

More information

Within-Household Respondent Selection An MRC Staff Point of View

Within-Household Respondent Selection An MRC Staff Point of View Within-Household Respondent Selection An MRC Staff Point of View In 2008, the MRC retained noted survey researcher Dr. Paul J. Lavrakas to review the state of knowledge concerning within-household respondent

More information

BACKGROUND ON THE SURVEY PROCESS

BACKGROUND ON THE SURVEY PROCESS Chapter Two BACKGROUND ON THE SURVEY PROCESS In this chapter, we present an overview of the various aspects of the research survey process. 1 We emphasize that surveying should first be thought of as a

More information

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004

HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004 HOUSEHOLD TELEPHONE SERVICE AND USAGE PATTERNS IN THE U.S. IN 2004 Clyde Tucker, Bureau of Labor Statistics; J. Michael Brick, Westat; Brian Meekins, Bureau of Labor Statistics; and David Morganstein,

More information

Types of Mixed Mode Designs

Types of Mixed Mode Designs Types of Mixed Mode Designs Edith D. de Leeuw De Aanstaande Revolutie Binnen Het Marktonderzoek MOAbout, Amsterdam 22 January 2009 Copyright Edith D. de Leeuw Mixed Mode Multi Mode Multiple Mode Terminology

More information

Margo L Barr 1,2*, Raymond A Ferguson 1 and David G Steel 2

Margo L Barr 1,2*, Raymond A Ferguson 1 and David G Steel 2 Barr et al. BMC Research Notes 2014, 7:517 RESEARCH ARTICLE Open Access Inclusion of mobile telephone numbers into an ongoing population health survey in New South Wales, Australia, using an overlapping

More information

The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys

The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys The Value of the Increasing Effort To Maintain High Response Rates in Telephone Surveys Barbara Lepidus Carlson 1 and Richard Strouse 2 Mathematica Policy Research, Inc. 1 Battelle 2 Abstract Declining

More information

The American Panel Survey

The American Panel Survey Draft: To be updated in June Guide for Investigators The American Panel Survey Weidenbaum Center Washington University James L. Gibson and Steven S. Smith, Directors About The American Panel Survey (TAPS)

More information

TOTAL HAVE EVER DONE THIS DID YESTERDAY. Does anyone in your household have a Facebook account, or not?

TOTAL HAVE EVER DONE THIS DID YESTERDAY. Does anyone in your household have a Facebook account, or not? Survey Questions WEB1-A Please tell me if you ever use the internet to do any of the following things. Do you ever use the internet to...[insert ITEM; RANDOMIZE; ALWAYS ASK ABOUT FACEBOOK LAST]? Based

More information

Research Overview: Telephone versus Online Research Advantages and Pitfalls

Research Overview: Telephone versus Online Research Advantages and Pitfalls . 26030 Highway 74, Suite A, P.O. Box 1094 Kittredge, CO 80457 P: 303.679.6300 F: 303.679.6680 info@praxigroup.net www.praxigroup.net Research Overview: Telephone versus Online Research Advantages and

More information

Descriptive Methods Ch. 6 and 7

Descriptive Methods Ch. 6 and 7 Descriptive Methods Ch. 6 and 7 Purpose of Descriptive Research Purely descriptive research describes the characteristics or behaviors of a given population in a systematic and accurate fashion. Correlational

More information

A GUIDE TO SAMPLING & STATISTICAL RELIABILITY

A GUIDE TO SAMPLING & STATISTICAL RELIABILITY July 2009 Technical Note 5 A GUIDE TO SAMPLING & STATISTICAL RELIABILITY The word survey is used most often to describe a method of gathering information from a sample of individuals. This sample is usually

More information

Cell Phone Coverage in an ABS Multi-Mode Design

Cell Phone Coverage in an ABS Multi-Mode Design Cell Phone Coverage in an ABS Multi-Mode Design Kanru Xia 1, Ashley Amaya 1, Whitney Murphy 1 1 NORC at the University of Chicago, 55 E. Monroe St., Chicago, IL 60603 Abstract By the end of 2011, 32.3

More information

71% of online adults now use video sharing sites

71% of online adults now use video sharing sites 71% of online adults now use video sharing sites Kathleen Moore, Pew Internet Project 7/25/2011 http://pewinternet.org/reports/2011/video sharing sites.aspx Pew Research Center 1615 L St., NW Suite 700

More information

mobilesampling new challenges and developments in mobile research methodology

mobilesampling new challenges and developments in mobile research methodology mobilesampling new challenges and developments in mobile research methodology Mobile Telephone Methodology: Present and Future The growth of the percent of the U.S. population that owns a mobile telephone

More information

Research Online. University of Wollongong. Margo Barr University of Wollongong, mlb593@uowmail.edu.au

Research Online. University of Wollongong. Margo Barr University of Wollongong, mlb593@uowmail.edu.au University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Information Sciences 2012 Inclusion of mobile phone numbers into an

More information

Oregon Health Insurance Survey

Oregon Health Insurance Survey Oregon Health Insurance Survey A Summary of Developing, Fielding, and Analyzing the Oregon Health Insurance Survey For An In-depth Analysis Please Review The Oregon Health Survey 2011 Methodology Report

More information

Older Adults and Social Media Social networking use among those ages 50 and older nearly doubled over the past year

Older Adults and Social Media Social networking use among those ages 50 and older nearly doubled over the past year Older Adults and Social Media Social networking use among those ages 50 and older nearly doubled over the past year Mary Madden, Senior Research Specialist August 27, 2010 Report URL: http://pewinternet.org/reports/2010/older-adults-and-social-media.aspx

More information

Overview of BLS s Work on Occupational Safety and Health. Kristen Monaco Office of Compensation and Working Conditions

Overview of BLS s Work on Occupational Safety and Health. Kristen Monaco Office of Compensation and Working Conditions Overview of BLS s Work on Occupational Safety and Health Kristen Monaco Office of Compensation and Working Conditions 1 U.S. BUREAU OF LABOR STATISTICS bls.gov Outline Brief Description of SOII Evidence

More information

Making Surveys Work for You: Some Important Design Considerations

Making Surveys Work for You: Some Important Design Considerations Making Surveys Work for You: Some Important Design Considerations By Don A. Dillman* For Net-Conference on Survey Methods and Measurement June 15,11 *Regents Professor in the Department of Sociology and

More information

Sample Surveys. The two most typical sources of bias are selection bias and non-response bias.

Sample Surveys. The two most typical sources of bias are selection bias and non-response bias. Sample Surveys In this class we will consider the problem of sampling from a finite population. This is usually referred to as a survey. The goal of the survey is to learn about some parameters of a population,

More information

Introduction... 3. Qualitative Data Collection Methods... 7 In depth interviews... 7 Observation methods... 8 Document review... 8 Focus groups...

Introduction... 3. Qualitative Data Collection Methods... 7 In depth interviews... 7 Observation methods... 8 Document review... 8 Focus groups... 1 Table of Contents Introduction... 3 Quantitative Data Collection Methods... 4 Interviews... 4 Telephone interviews... 5 Face to face interviews... 5 Computer Assisted Personal Interviewing (CAPI)...

More information

An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates

An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates An Assessment of the Effect of Misreporting of Phone Line Information on Key Weighted RDD Estimates Ashley Bowers 1, Jeffrey Gonzalez 2 1 University of Michigan 2 Bureau of Labor Statistics Abstract Random-digit

More information

Topology of the Landline Telephone Sampling Frame. Mansour Fahimi, Dale Kulp, and David Malarek (Marketing Systems Group)

Topology of the Landline Telephone Sampling Frame. Mansour Fahimi, Dale Kulp, and David Malarek (Marketing Systems Group) Marketing Systems Group GENESYS S A M P L I N G Topology of the Landline Telephone Sampling Frame Mansour Fahimi, Dale Kulp, and David Malarek (Marketing Systems Group) Published in the December 2009 Issue

More information

REACHING THE U.S. CELL PHONE GENERATION COMPARISON OF CELL PHONE SURVEY RESULTS WITH AN ONGOING LANDLINE TELEPHONE SURVEY

REACHING THE U.S. CELL PHONE GENERATION COMPARISON OF CELL PHONE SURVEY RESULTS WITH AN ONGOING LANDLINE TELEPHONE SURVEY Public Opinion Quarterly, Vol. 71, No. 5 2007, pp. 814 839 REACHING THE U.S. CELL PHONE GENERATION COMPARISON OF CELL PHONE SURVEY RESULTS WITH AN ONGOING LANDLINE TELEPHONE SURVEY MICHAEL W. LINK MICHAEL

More information

The Use and Effects of Incentives in Surveys

The Use and Effects of Incentives in Surveys The Use and Effects of Incentives in Surveys Eleanor Singer Survey Research Center, University of Michigan Prepared for presentation to the National Science Foundation, Washington DC, October 3-4, 2012.

More information

Ensuring Data Quality: Monitoring Accuracy and Consistency Among Telephone Interview Monitors

Ensuring Data Quality: Monitoring Accuracy and Consistency Among Telephone Interview Monitors Ensuring Data Quality: Monitoring Accuracy and Consistency Among Telephone Interview Monitors Joseph Baker, Claudia Gentile, Jason Markesich, Shawn Marsh, and Rebecca Weiner Joseph Baker, Mathematica Policy

More information

Assessing the Representativeness of Public Opinion Surveys

Assessing the Representativeness of Public Opinion Surveys TUESDAY, MAY 15, 2012 Assessing the Representativeness of Public Opinion Surveys FOR FURTHER INFORMATION CONTACT: Andrew Kohut President, Pew Research Center Scott Keeter Director of Survey Research Carroll

More information

This report addresses three main questions that researchers face with regard to Internet surveys:

This report addresses three main questions that researchers face with regard to Internet surveys: SUMMARY The Internet is profoundly changing the way we communicate with one another. One of the most recent new uses of the World Wide Web is as a survey platform. Internet-based surveys, although still

More information

Factors Affecting Questionnaire Design. INTRODUCTION TO QUESTIONNAIRE DESIGN Fall, 2015 Allyson L. Holbrook. Open vs.

Factors Affecting Questionnaire Design. INTRODUCTION TO QUESTIONNAIRE DESIGN Fall, 2015 Allyson L. Holbrook. Open vs. INTRODUCTION TO QUESTIONNAIRE DESIGN Fall, 2015 Allyson L. Holbrook www.srl.uic.edu Factors Affecting Questionnaire Design Goals and purpose of research Types of questions being asked Objective (e.g.,

More information

Practicability of Including Cell Phone Numbers in Random Digit Dialed Surveys: Pilot Study Results from the Behavioral Risk Factor Surveillance System

Practicability of Including Cell Phone Numbers in Random Digit Dialed Surveys: Pilot Study Results from the Behavioral Risk Factor Surveillance System Practicability of Including Cell Phone Numbers in Random Digit Dialed Surveys: Pilot Study Results from the Behavioral Risk Factor Surveillance System Michael W. Link 1, Michael P. Battaglia 2, Martin

More information

Overview of Proposed Changes to BRFSS Cell Phone Sampling Frame

Overview of Proposed Changes to BRFSS Cell Phone Sampling Frame Overview of Proposed Changes to BRFSS Cell Phone Sampling Frame When collecting data using RDD, there are several options for sampling that include landline and cell phone users. The first option is to

More information

The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms 1

The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms 1 CHAPTER 23 The Causes and Consequences of Response Rates in Surveys by the News Media and Government Contractor Survey Research Firms 1 Allyson L. Holbrook Survey Research Laboratory, University of Illinois

More information

Sampling & Sample Size Estimation

Sampling & Sample Size Estimation Sampling & Sample Size Estimation Moazzam Ali MD, PhD, MPH Department of Reproductive Health and Research World Health Organization Geneva, Switzerland Presented at: GFMER September 16, 2014 Topics to

More information

Politics on Social Networking Sites

Politics on Social Networking Sites SEPTEMBER 4, 2012 Politics on Social Networking Sites Campaign and policy-related material on SNS plays a modest role in influencing most users views and political activities. Democrats and liberals are

More information

CHOOSING AMONG THE VARIOUS TYPES OF INTERNET SURVEYS

CHOOSING AMONG THE VARIOUS TYPES OF INTERNET SURVEYS Chapter Four CHOOSING AMONG THE VARIOUS TYPES OF INTERNET SURVEYS Here, we examine the various types of Internet surveys and the differences among them that factor into deciding what sort of survey is

More information

Pre-Call Validation of RDD Cell Phone Numbers A Field Experiment

Pre-Call Validation of RDD Cell Phone Numbers A Field Experiment Pre-Call Validation of RDD Cell Phone Numbers A Field Experiment Tanja Kunz 1, Marek Fuchs 1 1 Institute of Sociology, Darmstadt University of Technology Residenzschloss, 64283 Darmstadt (Germany) Abstract

More information

A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey

A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey A Comparison of Level of Effort and Benchmarking Approaches for Nonresponse Bias Analysis of an RDD Survey Daifeng Han and David Cantor Westat, 1650 Research Blvd., Rockville, MD 20850 Abstract One method

More information

Best Practices: Understanding and Reducing Bias in Your Surveys

Best Practices: Understanding and Reducing Bias in Your Surveys Best Practices: Understanding and Reducing Bias in Your Surveys Sampling Bias - Who You Survey Sampling Bias: When some members of your survey population are less likely to be invited to be surveyed than

More information

Quick Reference Guide MRIA CODE OF CONDUCT FOR MARKET AND SOCIAL RESEARCH

Quick Reference Guide MRIA CODE OF CONDUCT FOR MARKET AND SOCIAL RESEARCH Quick Reference Guide MRIA CODE OF CONDUCT FOR MARKET AND SOCIAL RESEARCH APPENDIX "L" Polling Standards This document contains the following information: 1. Mandatory disclosures when releasing polling

More information

Sample: n=2,252 national adults, age 18 and older, including 1,127 cell phone interviews Interviewing dates:

Sample: n=2,252 national adults, age 18 and older, including 1,127 cell phone interviews Interviewing dates: Survey Questions Spring 2013 Tracking Survey Final Topline 5/21/2013 Data for April 17-May 19, 2013 Princeton Survey Research Associates International for the Pew Research Center s Internet & American

More information

Sources of Variation in Response Rates to the Behavioral Risk Factor Surveillance System

Sources of Variation in Response Rates to the Behavioral Risk Factor Surveillance System Sources of Variation in Response Rates to the Behavioral Risk Factor Surveillance System David W. Smith, Ph.D., M.P.H. University of Texas School of Public Health Abstract The BRFSS is an RDD telephone

More information

Web panel surveys can they be designed and used in a scientifically sound way?

Web panel surveys can they be designed and used in a scientifically sound way? Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session IPS064) p.567 Web panel surveys can they be designed and used in a scientifically sound way? Jörgen Svensson Statistics

More information

Americans and text messaging

Americans and text messaging Americans and text messaging 31% of text message users prefer texting to voice calls, and young adults stand out in their use of text messaging Aaron Smith, Senior Research Specialist 9/19/2011 http://pewinternet.org/reports/2011/cell-phone-texting-2011.aspx

More information

Statistics Notes Revision in Maths Week

Statistics Notes Revision in Maths Week Statistics Notes Revision in Maths Week 1 Section - Producing Data 1.1 Introduction Statistics is the science that studies the collection and interpretation of numerical data. Statistics divides the study

More information

7/2/2014. Text Message Follow-up for Cell-Phone Respondents to a National Survey. Introduction. Research Questions. National Immunization Survey (NIS)

7/2/2014. Text Message Follow-up for Cell-Phone Respondents to a National Survey. Introduction. Research Questions. National Immunization Survey (NIS) Introduction Text Message Follow-up for Cell-Phone Respondents to a National Survey American Association for Public Opinion Research (AAPOR) Annual Conference, Anaheim, CA May 17, 2014 Jennifer Vanicek,

More information

The Impact of Cell Phone Non-Coverage Bias on Polling in the 2004 Presidential Election

The Impact of Cell Phone Non-Coverage Bias on Polling in the 2004 Presidential Election The Impact of Cell Phone Non-Coverage Bias on Polling in the 2004 Presidential Election Scott Keeter Pew Research Center August 1, 2005 Abstract Despite concerns that the accuracy of pre-election telephone

More information

POLLING STANDARDS. The following standards were developed by a committee of editors and reporters and should be adhered to when using poll results.

POLLING STANDARDS. The following standards were developed by a committee of editors and reporters and should be adhered to when using poll results. ! POLLING STANDARDS June, 2006 The following standards were developed by a committee of editors and reporters and should be adhered to when using poll results. OVERVIEW Reporting on polls is no different

More information

Questionnaire Design in Telephone Surveys: Interviewers and Research Call Center Managers Experience

Questionnaire Design in Telephone Surveys: Interviewers and Research Call Center Managers Experience Questionnaire Design in Telephone Surveys: Interviewers and Research Call Center Managers Experience Wojciech Jablonski University of Lodz Poland wjablonski@uni.lodz.pl AAPOR 69th Annual Conference Anaheim,

More information

65% of online adults use social networking sites

65% of online adults use social networking sites 65% of online adults use social networking sites Women maintain their foothold on SNS use and older Americans are still coming aboard Mary Madden, Senior Research Specialist Kathryn Zickuhr, Research Specialist

More information

A survey of civil court users: a feasibility study

A survey of civil court users: a feasibility study 2015 Analytical Services exists to improve policy making, decision taking and practice by the Ministry of Justice. It does this by providing robust, timely and relevant data and advice drawn from research

More information

35% of those ages 16 and older own tablet computers

35% of those ages 16 and older own tablet computers www.pewresearch.org OCTOBER 18, 2013 35% of those ages 16 and older own tablet computers Up from 25% last year more than half of those in households earning $75,000 or more now have tablets 24% own e-book

More information

surveys that is rarely discussed: the polls reported in the lead stories of the nation s leading

surveys that is rarely discussed: the polls reported in the lead stories of the nation s leading The Economist/YouGov Internet Presidential Poll Morris Fiorina and Jon Krosnick * In 1936, George Gallup correctly forecast Franklin Roosevelt s presidential landslide at a time when one of the country

More information

Telephone Surveys and the Mobile Phone Only Population Workshop 17, July, 2012.

Telephone Surveys and the Mobile Phone Only Population Workshop 17, July, 2012. Telephone Surveys and the Mobile Phone Only Population Workshop 17, July, 2012. The 2011 Workshop Day 1 Integrated Public Number Database (Michel Ryan) Is the exclusion of mobile phones from telephone

More information

Address correspondence to Richard Curtin;

Address correspondence to Richard Curtin; Public Opinion Quarterly, Vol. 69, No. 1, Spring 2005, pp. 87 98 CHANGES IN TELEPHONE SURVEY NONRESPONSE OVER THE PAST QUARTER CENTURY RICHARD CURTIN Survey Research Center, University of Michigan STANLEY

More information

THE FUTURE OF INTERNET-BASED SURVEY METHODS

THE FUTURE OF INTERNET-BASED SURVEY METHODS Chapter Seven CONCLUSIONS In this chapter, we offer some concluding thoughts on the future of Internet-based surveys, the issues surrounding the use of e-mail and the Web for research surveys, and certain

More information

Survey Nonresponse and Nonresponse Bias. Survey Research Laboratory University of Illinois at Chicago

Survey Nonresponse and Nonresponse Bias. Survey Research Laboratory University of Illinois at Chicago Survey Nonresponse and Nonresponse Bias Survey Research Laboratory University of Illinois at Chicago Outline Overview of response rate issue Overview of non-response bias Review of available methods for

More information

Comparative Study of Electoral Systems (CSES) Module 3: Sample Design and Data Collection Report

Comparative Study of Electoral Systems (CSES) Module 3: Sample Design and Data Collection Report Comparative Study of Electoral Systems 1 Comparative Study of Electoral Systems (CSES) Module 3: Sample Design and Data Collection Report Country: México. Date of Election: July, 5 2009. Prepared by: CIDE.

More information

Lessons Learned From an NHIS Web- Followback Feasibility Study

Lessons Learned From an NHIS Web- Followback Feasibility Study Lessons Learned From an NHIS Web- Followback Feasibility Study Sarah S. Joestl, DrPH Senior Service Fellow Survey Planning and Special Surveys Branch Division of Health Interview Statistics Division of

More information

Tablet Ownership 2013

Tablet Ownership 2013 www.pewresearch.org JUNE 10, 2013 Tablet Ownership 2013 Tablet adoption has almost doubled over the past year. For the first time a third (34%) of American adults now own a tablet computer, including almost

More information

17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device

17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device JUNE 26, 2012 17% of cell phone owners do most of their online browsing on their phone, rather than a computer or other device Most do so for convenience, but for some their phone is their only option

More information

Experiences with the Criminal Justice System in a Household Survey: Introducing the Survey of Criminal Justice Experience (SCJE)

Experiences with the Criminal Justice System in a Household Survey: Introducing the Survey of Criminal Justice Experience (SCJE) Experiences with the Criminal Justice System in a Household Survey: Introducing the Survey of Criminal Justice Experience (SCJE) August 2013 Susan L. Brown and Wendy D. Manning, Co-Directors National Center

More information

2014 MASSACHUSETTS HEALTH INSURANCE SURVEY METHODOLOGY REPORT

2014 MASSACHUSETTS HEALTH INSURANCE SURVEY METHODOLOGY REPORT CENTER FOR HEALTH INFORMATION AND ANALYSIS 2014 MASSACHUSETTS HEALTH INSURANCE SURVEY METHODOLOGY REPORT MAY 2015 Prepared by: David Dutwin and Susan Sherr, SSRS Sharon K. Long, Urban Institute Contents

More information