Introduction to Survey Methodology. Professor Ron Fricker Naval Postgraduate School Monterey, California

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Introduction to Survey Methodology. Professor Ron Fricker Naval Postgraduate School Monterey, California"

Transcription

1 Introduction to Survey Methodology Professor Ron Fricker Naval Postgraduate School Monterey, California 1

2 Goals for this Lecture Introduce professor and course Define what we mean by the term survey Characteristics of typical surveys Distinction between polls and surveys Lay out the basic steps in conducting a rigorous research survey Discuss various survey modes Pros and cons Trade-offs Considerations in making mode selection 2

3 Contact Information Professor Ron Fricker Phone Commercial: DSN: Fax:

4 Professional Experience Academic credentials Ph.D. and M.A. in Statistics, Yale University M.S. in Ops Research, The George Washington University B.S. in Mathematics from the United States Naval Academy Teaching credentials Started teaching post-graduate courses in mid-80s Have taught at NPS, RAND Graduate School, and USC Real world credentials Former active duty naval officer Senior Statistician specializing in survey and military research at the RAND Corporation Can find out more at 4

5 Class is All About Collecting and Analyzing Survey Data In this class you ll learn: How to design good survey questions and craft an effective survey instrument About the trade-offs between various survey modes The rudiments of sampling for surveys Some statistical methods useful for analyzing survey data 5

6 Course Schedule 6

7 Course Materials Lecture notes: Binder with all slides Textbook: Survey Methodology Also excerpts from Mail and Internet Surveys and Sampling: Design and Analysis Software: JMP version 7.0 All on-line at Readings password protected: MCOTEA 7

8 What is a Survey? A survey is a: systematic method for gathering information from (a sample of) entities for the purposes of constructing quantitative descriptors of the attributes of the larger population of which the entities are members In many ways, a survey is just a form of data collection 8

9 A Special Type of Data Collection Typically surveys: Gather information by asking people questions Collect information by either (1) interviewers asking questions and recording responses or (2) respondents reading and recording their own answers Collect information from a subset of the population, a sample, rather than from all members 9

10 Polls vs. Surveys No clear distinction between the two terms Poll often used for private sector opinion studies Use many of the same design features as studies that would be called surveys Poll rarely used to describe government or scientific surveys To me, the term poll implies either a commercial or less-scientific study, or a quick turn-around survey whose results may be of short-term interest 10

11 Steps in Conducting a Survey (1) Clearly state research objective(s) Decide on survey mode(s) How will you contact potential respondents? Contact mode web, , phone, etc. In what media will the survey be given? Survey or response mode web, , phone, etc. How will you follow up with non-respondents? Follow-up mode web, , phone, etc. Determine fielding strategy How to maximize response rates? Design the survey questions and the survey instrument 11

12 Steps in Conducting a Survey (2) Determine sample size and sampling strategy Obtain Institutional Review Board (IRB) or other approval as necessary Are respondents promised confidentiality? What is the impact if their survey responses become known? Pre-test, pre-test, pre-test Give the survey to some test subjects and get their feedback What works and what doesn t? Are you getting correct data/information? Revise and re-pre-test as necessary Draw sample and field the survey 12

13 Steps in Conducting a Survey (3) Follow up with non-respondents Assemble the survey data Clean data as necessary Analyze the data and report results Summarize the data Weight as necessary and appropriate to infer back to population Calculate and report margin of error Evidence of bias? Unit and item non-response Communicate survey results to respondents if promised 13

14 One View of the Steps 14

15 Another View of the Steps Excerpted from What is a Survey? by Fritz Scheuren 15

16 Examples of Large-scale Surveys (from your textbook) National Crime Victimization Survey National Survey on Drug Use and Health Survey of Consumers National Assessment of Educational Progress Behavioral Risk Factor Surveillance System Current Employment Statistics Program 16

17 National Crime Victimization Survey (NCVS) 17

18 National Survey on Drug Use and Health (NSDUH) 18

19 Survey of Consumers (SOC) 19

20 National Assessment of Educational Progress (NAEP) 20

21 Behavioral Risk Factor Surveillance System (BRFSS) 21

22 Current Employment Statistics (CES) Program 22

23 Examples of DoD Surveys Annual Active Duty Personnel Survey Survey of MWR Services and Family Programs Military Recruiter Quality of Life Survey Adult Influencer Poll (AIP) Youth Attitude Tracking Study (now JAMRS Youth Poll) 23

24 Which Survey Mode Should I Use? You are going to conduct a research survey Goal is to make the best inference possible from sample to population within budget constraints To important questions: What is the most appropriate method to choose for a particular research question? What is the impact of choosing a particular method on survey cost and accuracy? 24

25 Traditional Survey Modes Mail Paper questionnaire sent to respondents Self-administered and mailed back Telephone Interviewers call respondents on telephone Interviewer-assisted In-person Interviewers go to respondent s home or office Interviewer-assisted 25

26 Options Have Multiplied in the Age of Computers and Telecom CATI TDE IVR/T-ACASI CAPI Text- CASI Audio- CASI Video- CASI CAI Methods 26

27 Mode Considerations Modes differ by: Degree of interviewer involvement Degree of interaction with respondent Degree of respondent privacy Channels of communication Technology usage Considerations: Cost Timeliness Accuracy 27

28 Broadly speaking Interviewer-assisted surveys more costly than selfadministered Face-to-face most expensive, then telephone, then mail, then web Interviewer-assisted surveys less prone to various errors and item nonresponse Particularly compared to self-administered paper survey More or less compared to self-administered computerassisted surveys depending on sophistication of program Self-administered less subject to sensitivity bias Can combine some self-administered questions into a predominantly interviewer-assisted mode 28

29 However, timeliness is not just about the length of time a survey is in the field Really, it s from instrument design through analysis Much of that is not affected by survey mode Compared to a mail survey, web-based surveys save on two main things: Mailing time to send out and return surveys Time required to do data entry, coding, and cleaning for paper surveys Yet, it can also take a lot of time to code, test, and de-bug very complicated web (more generally, computer-assisted) survey programs 29

30 And, cost is not just about manhours On large mail surveys, printing and postage costs can be significant 1,500 surveys x 4 mailings at $1.50 each = $9,000! 1,500 $1 incentives = $1,500 On telephone surveys must consider fixed costs of phone and CATI equipment + variable costs On face-to-face interviews, travel costs can be significant (not to mention interviewer time) But in terms of manhours, don t forget: (1) instrument design time; (2) in non-electronic modes, data entry and coding time; (3) non-response follow-up efforts; and (4) analysis time 30

31 And accuracy is not just about response rates and missing items Total survey error: what mode or modes will most decrease all types of error? Lots of considerations Frame coverage Nonresponse rates Sensitivity bias, etc 31

32 32

33 Cost and Frame Availability Often Drive Mode Choice Survey budgets never unlimited Frequently cost will eliminate one or more modes And likely drive other survey design decisions I.e., face-to-face interviews too expensive On the other hand, surveys using area probability frames usually have to start out using face-to-face interviews Similarly, if a complete frame is available, web survey is possible option But general population surveys eliminate web 33

34 Some Modes Logically Group (1) Telephone and face-to-face surveys Telephone often considered an alternative to faceto-face Both interviewer-assisted and can have similar frame coverage issues Many longitudinal surveys start out with a face-toface interview with telephone follow-ups Mail sometimes an alternative to telephone If both telephone and addresses available 34

35 Some Modes Logically Group (2) Web surveys often considered a replacement for mail surveys Coverage is still an issue for web surveys Web superior in terms of automation and speed However, literature consistently finds that nonresponse rates are higher for web surveys 35

36 Using Multiple Modes Reasons to use multiple modes: Maximize response rates Reduce/eliminate selection & coverage bias Allow respondents to use most convenient mode Reduce costs Cross-sectional surveys: start with least costly mode and move to more expensive modes as necessary to get response Longitudinal surveys: Use less expensive modes in follow-on survey rounds 36

37 Considerations When Using Multiple Modes Focus should be on designing instruments to ensure equivalency across modes Rather than optimizing for a particular mode E.g., make the follow-on web survey resemble its counterpart paper survey Ensure operation is set up to track cases across the modes Must avoid duplications and uncoordinated respondent contact Must design and field so that mode effects can be disentangled from other sample characteristics May have to randomize survey mode over a subset 37

38 Other Data Collection Methods Surveys are not the only (nor necessarily the best) way to collect data Other methods include Administrative records Focus groups and qualitative investigations Randomized experiments Which is best depends on the research question(s) and/or the purpose for which the data will be used 38

39 What We Have Covered Defined the term survey, including the Characteristics of types of surveys this class will focus on Distinction between polls and surveys Discussed the basic steps in conducting a rigorous research survey Considerations when choosing a survey mode In various ways, choice likely to affect survey Timeliness Accuracy Cost And cost may drive mode choice possibilities 39

BACKGROUND ON THE SURVEY PROCESS

BACKGROUND ON THE SURVEY PROCESS Chapter Two BACKGROUND ON THE SURVEY PROCESS In this chapter, we present an overview of the various aspects of the research survey process. 1 We emphasize that surveying should first be thought of as a

More information

THE FUTURE OF INTERNET-BASED SURVEY METHODS

THE FUTURE OF INTERNET-BASED SURVEY METHODS Chapter Seven CONCLUSIONS In this chapter, we offer some concluding thoughts on the future of Internet-based surveys, the issues surrounding the use of e-mail and the Web for research surveys, and certain

More information

Statistical Methods for Sample Surveys (140.640)

Statistical Methods for Sample Surveys (140.640) This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this

More information

Self-administered Questionnaire Survey

Self-administered Questionnaire Survey Primary methods of survey data collection Self-administered questionnaire (SAQ) Mail Face-to-face interview Telephone interview Self-administered Questionnaire Survey 1 Steps are: a. Mail distribution

More information

Appendix Methodology and Statistics

Appendix Methodology and Statistics Appendix Methodology and Statistics Introduction Science and Engineering Indicators (SEI) contains data compiled from a variety of sources. This appendix explains the methodological and statistical criteria

More information

Informing Public Policy Through Rigorous Survey Research

Informing Public Policy Through Rigorous Survey Research Informing Public Policy Through Rigorous Survey Research Data / Insight / Impact Abt SRBI is a recognized leader in utilizing survey, opinion, and public policy research to build bridges between decision-makers

More information

Section 2: Ten Tools for Applying Sociology

Section 2: Ten Tools for Applying Sociology Section 2: Ten Tools for Applying Sociology CHAPTER 2.6: DATA COLLECTION METHODS QUICK START: In this chapter, you will learn The basics of data collection methods. To know when to use quantitative and/or

More information

Using Surveys for Data Collection in Continuous Improvement

Using Surveys for Data Collection in Continuous Improvement Innovation Insight Series Number 14 http://www.psu.edu/president/pia/innovation/ Across the University many people use data in assessment and decision-making. Data-based decision-making, an essential element

More information

Internet Surveys. Examples

Internet Surveys. Examples Web material accompanying The International Handbook of Survey Methodology Chapter 14 Internet Surveys Katja Lozar Manfreda Vasja Vehovar University of Ljubljana Examples Below follow examples and more

More information

National Postsecondary Student Aid Study

National Postsecondary Student Aid Study National Postsecondary Student Aid Study Jennifer S. Wine John A. Riccobono Education Studies Division RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org What

More information

Mixed-Mode Methods for Conducting Survey Research

Mixed-Mode Methods for Conducting Survey Research Mixed-Mode Methods for Conducting Survey Research Herbert M. Baum, Ph.D.; Anna Chandonnet M.A.; Jack Fentress M.S., M.B.A.; and Colleen Rasinowich, B.A. www.datarecognitioncorp.com In November 2010 the

More information

The AmeriSpeak ADVANTAGE

The AmeriSpeak ADVANTAGE OVERVIEW Funded and operated by NORC at the University of Chicago, AmeriSpeak TM is a probabilitybased panel (in contrast to a non-probability panel). Randomly selected households are sampled with a known,

More information

ESSNET PROJECT ON DATA COLLECTION FOR SOCIAL SURVEYS USING MULTI MODES (DCSS)

ESSNET PROJECT ON DATA COLLECTION FOR SOCIAL SURVEYS USING MULTI MODES (DCSS) Distr. GENERAL 8 October 2012 WP. 27 ENGLISH ONLY UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Seminar on New Frontiers for Statistical Data Collection (Geneva, Switzerland,

More information

Lessons Learned From an NHIS Web- Followback Feasibility Study

Lessons Learned From an NHIS Web- Followback Feasibility Study Lessons Learned From an NHIS Web- Followback Feasibility Study Sarah S. Joestl, DrPH Senior Service Fellow Survey Planning and Special Surveys Branch Division of Health Interview Statistics Division of

More information

Survey Research. Classifying surveys on the basis of their scope and their focus gives four categories:

Survey Research. Classifying surveys on the basis of their scope and their focus gives four categories: Survey Research Types of Surveys Surveys are classified according to their focus and scope (census and sample surveys) or according to the time frame for data collection (longitudinal and cross-sectional

More information

Use of Text Messaging to Increase Response Rates

Use of Text Messaging to Increase Response Rates Use of Text Messaging to Increase Response Rates Key words: BRFSS, response rate, cell phones, smart phones, text message Prepared by: Piper DuBray, Kristie Healey ICF International 0 Introduction Survey

More information

This report addresses three main questions that researchers face with regard to Internet surveys:

This report addresses three main questions that researchers face with regard to Internet surveys: SUMMARY The Internet is profoundly changing the way we communicate with one another. One of the most recent new uses of the World Wide Web is as a survey platform. Internet-based surveys, although still

More information

QUALITY ASSESSMENT OF DATA COLLECTED FROM NON-ENGLISH SPEAKING HOUSEHOLDS IN THE AMERICAN COMMUNITY SURVEY

QUALITY ASSESSMENT OF DATA COLLECTED FROM NON-ENGLISH SPEAKING HOUSEHOLDS IN THE AMERICAN COMMUNITY SURVEY QUALITY ASSESSMENT OF DATA COLLECTED FROM NON-ENGLISH SPEAKING HOUSEHOLDS IN THE AMERICAN COMMUNITY SURVEY Pamela D. McGovern and Deborah H. Griffin U.S. Census Bureau, 4700 Silver Hill Road, Washington,

More information

www.doncaster.gov.uk

www.doncaster.gov.uk Market Research It is essential that market research is undertaken to establish there is a need for childcare within a specified area and have knowledge of the local community that a business wishes to

More information

Descriptive Methods Ch. 6 and 7

Descriptive Methods Ch. 6 and 7 Descriptive Methods Ch. 6 and 7 Purpose of Descriptive Research Purely descriptive research describes the characteristics or behaviors of a given population in a systematic and accurate fashion. Correlational

More information

Appendix I: Methodology

Appendix I: Methodology Appendix I: Methodology SSRS METHODOLOGY SSRS conducted a survey of Muslims and Jews for the Institute for Social Policy and Understanding from January 18 through January 27, 2016. The study investigated

More information

Data Collection and Sampling OPRE 6301

Data Collection and Sampling OPRE 6301 Data Collection and Sampling OPRE 6301 Recall... Statistics is a tool for converting data into information: Statistics Data Information But where then does data come from? How is it gathered? How do we

More information

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives Fulfilling the core market research educational needs of individuals and companies worldwide Presented through a unique partnership between How to Contact Us: Phone: +1-706-542-3537 or 1-800-811-6640 (USA

More information

Survey Design. Acknowledgements. Teri Peterson The Survey Kit. Survey Design and Sampling Procedures.

Survey Design. Acknowledgements. Teri Peterson The Survey Kit. Survey Design and Sampling Procedures. Survey Design Teri Peterson Peteteri@isu.edu Acknowledgements The Survey Kit By Arlene Fink Survey Design and Sampling Procedures By Tony Babinec Statistics.com 1 What is a survey? A system for collecting

More information

Exploring New Methods of Data Gathering in Long-Distance Passenger Travel Data

Exploring New Methods of Data Gathering in Long-Distance Passenger Travel Data Exploring New Methods of Data Gathering in Long-Distance Passenger Travel Data Ben Pierce, Battelle June 8, 2011 1 Why Are New Methods Needed? Declining Response Rates Survey saturation in US Growing cell-only

More information

Questionnaire Design. Questionnaires in Clinical Research

Questionnaire Design. Questionnaires in Clinical Research Questionnaire Design Theresa A Scott, MS Department of Biostatistics theresa.scott@vanderbilt.edu D-2217 MCN; 3-1713 http://biostat.mc.vanderbilt.edu/theresascott Theresa A Scott, MS (Biostatistics) Questionnaire

More information

Optimising survey costs in a mixed mode environment

Optimising survey costs in a mixed mode environment Optimising survey costs in a mixed mode environment Vasja Vehovar 1, Nejc Berzelak 2, Katja Lozar Manfreda 3, Eva Belak 4 1 University of Ljubljana, vasja.vehovar@fdv.uni-lj.si 2 University of Ljubljana,

More information

How to gather and evaluate information

How to gather and evaluate information 09 May 2016 How to gather and evaluate information Chartered Institute of Internal Auditors Information is central to the role of an internal auditor. Gathering and evaluating information is the basic

More information

LITERATURE REVIEW OF WEB AND E-MAIL SURVEYS

LITERATURE REVIEW OF WEB AND E-MAIL SURVEYS Chapter Three LITERATURE REVIEW OF WEB AND E-MAIL SURVEYS In this chapter, we examine what has been written about Internet surveys in the literature, specifically Web and e-mail surveys. We address the

More information

Chapter 2: Research Methodology

Chapter 2: Research Methodology Chapter 2: Research Methodology 1. Type of Research 2. Sources of Data 3. Instruments for Data Collection 4. Research Methods 5. Sampling 6. Limitations of the Study 6 Chapter 2: Research Methodology Research

More information

Avoiding Bias in the Research Interview

Avoiding Bias in the Research Interview Avoiding Bias in the Research Interview Sonja I. Ziniel, MA Ph.D. Clinical Research Program, Children s Hospital Boston Department of Medicine, Harvard Medical School sonja.ziniel@childrens.harvard.edu

More information

Use of Text Messaging to Increase Response Rates

Use of Text Messaging to Increase Response Rates Introduction Use of Text Messaging to Increase Response Rates Piper DuBray ICF International 126 College St. Burlington, VT 05401 Proceedings of the 2013 Federal Committee on Statistical Methodology (FCSM)

More information

Collecting and Storing Data

Collecting and Storing Data Collecting and Storing Data Outlines Introduction. What do we mean by data collection? General guidelines on collecting data. Data Collection Methods. Data-Collection Formats. Data storage. Confidentiality

More information

see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys

see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys SURVEYS Survey = ask the same questions of a large sample see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys Basic Survey Process: 1. Design Questionnaire 2. Design Sample

More information

How do we know what we know?

How do we know what we know? Research Methods Family in the News Can you identify some main debates (controversies) for your topic? Do you think the authors positions in these debates (i.e., their values) affect their presentation

More information

MEMORANDUM FOR THE PRESIDENT S MANAGEMENT COUNCIL. Guidance on Agency Survey and Statistical Information Collections

MEMORANDUM FOR THE PRESIDENT S MANAGEMENT COUNCIL. Guidance on Agency Survey and Statistical Information Collections EXECUTIVE OFFICE OF THE PRESIDENT OFFICE OF MANAGEMENT AND BUDGET WASHINGTON, D.C. 20503 ADMINISTRATOR OFFICE OF INFORMATION AND REGULATORY AFFAIR January 20, 2006 MEMORANDUM FOR THE PRESIDENT S MANAGEMENT

More information

05/10/2015. Chapter 3 - Marketing Research. Marketing Project Plan

05/10/2015. Chapter 3 - Marketing Research. Marketing Project Plan Chapter 3 - Marketing Research Copyright 2013 Pearson Canada Inc. Marketing Project Plan How do we begin the project? Identify product What do we need? We need to gather information We also need to do

More information

ESOMAR 28 Questions. 1. What experience does your company have in providing online samples for market research?

ESOMAR 28 Questions. 1. What experience does your company have in providing online samples for market research? ESOMAR 28 Questions COMPANY PROFILE 1. What experience does your company have in providing online samples for market research? Since 1996, DoctorDirectory has specialized in deploying healthcare market

More information

Electronic Mail Vs. Mail Survey Response Rates

Electronic Mail Vs. Mail Survey Response Rates Electronic Mail Vs. Mail Survey Response Rates Researchers can no longer ignore the possihilities computer technology hold for the future of our profession. By Barbara A. Schuldt and Jeff W. Totten An

More information

Cash Rents Methodology and Quality Measures

Cash Rents Methodology and Quality Measures ISSN: 2167-129X Cash Rents Methodology and Quality Measures Released August 1, 2014, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States Department of Agriculture

More information

The application of alternative modes of data collection in UK Government social surveys. A report for the Government Statistical Service

The application of alternative modes of data collection in UK Government social surveys. A report for the Government Statistical Service The application of alternative modes of data collection in UK Government social surveys A report for the Government Statistical Service Peter Betts and Charles Lound Office for National Statistics May

More information

Research design and methods Part II. Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT

Research design and methods Part II. Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT Research design and methods Part II Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT From last week Research methodology Quantitative vs. Qualitative vs. Participatory/action research Research methods

More information

Using Online Surveys to Evaluate Distance Education Programs

Using Online Surveys to Evaluate Distance Education Programs 1 21st Annual Conference on Distance Teaching and Learning click here -> Using Online Surveys to Evaluate Distance Education Programs Elaine Strachota, PhD Distance Learning Coordinator and Faculty Milwaukee

More information

Descriptive Inferential. The First Measured Century. Statistics. Statistics. We will focus on two types of statistical applications

Descriptive Inferential. The First Measured Century. Statistics. Statistics. We will focus on two types of statistical applications Introduction: Statistics, Data and Statistical Thinking The First Measured Century FREC 408 Dr. Tom Ilvento 213 Townsend Hall ilvento@udel.edu http://www.udel.edu/frec/ilvento http://www.pbs.org/fmc/index.htm

More information

Sampling & Sample Size Estimation

Sampling & Sample Size Estimation Sampling & Sample Size Estimation Moazzam Ali MD, PhD, MPH Department of Reproductive Health and Research World Health Organization Geneva, Switzerland Presented at: GFMER September 16, 2014 Topics to

More information

Supporting Statement Part A Background

Supporting Statement Part A Background Supporting Statement Part A Surveys of Physicians and Home Health Agencies to Assess Access Issues for Specific Medicare Beneficiaries as Defined in Section 3131(d) of the ACA CMS-10429, OMB 0938-New Background

More information

Putting Reliable Health Care Performance Measurement Systems into Practice

Putting Reliable Health Care Performance Measurement Systems into Practice An NCQA Issue Brief 2000 L Street, NW Washington, DC 20036 888-275-7585 www.ncqa.org Putting Reliable Health Care Performance Measurement Systems into Practice By Joachim Roski, PhD MPH; Vice President,

More information

2. Choose the location and format for focus groups or interviews

2. Choose the location and format for focus groups or interviews Steps for Conducting Focus Groups or Individual In-depth Interviews 1. Plan the study Plan your study with the input of your strategy team, stakeholders, and research experts so that collaboratively you

More information

Flipped Learning Evaluation Institute for Effective Education (University of York) Peter Rudd, PhD

Flipped Learning Evaluation Institute for Effective Education (University of York) Peter Rudd, PhD Evaluation Summary Age range Year 5 (9-10) Number of pupils Average 60 pupils per school Number of schools 12 treatment and 12 control schools Design Randomised Controlled Trial with Delayed Treatment

More information

Methods for Improving Response Rates: Findings from AAPOR 2012. Benjamin L. Messer Washington State University PAPOR Mini-Conference, June 22, 2012

Methods for Improving Response Rates: Findings from AAPOR 2012. Benjamin L. Messer Washington State University PAPOR Mini-Conference, June 22, 2012 Methods for Improving Response Rates: Findings from AAPOR 2012 Benjamin L. Messer Washington State University PAPOR Mini-Conference, June 22, 2012 2 Disclaimer Papers/presentations obtained with author

More information

GSA Federal Supply Schedule

GSA Federal Supply Schedule GSA Federal Supply Schedule GS-23F-8181H Base Year-Option Period 3: May 21, 1998 September 30, 2017 Industrial Group 874; Industrial Class 8742 Mission Oriented Business Integrated Services (MOBIS) Through

More information

DETC DEGREE PROGRAMS GRADUATES AND EMPLOYERS EVALUATE THEIR WORTH

DETC DEGREE PROGRAMS GRADUATES AND EMPLOYERS EVALUATE THEIR WORTH DETC DEGREE PROGRAMS GRADUATES AND EMPLOYERS EVALUATE THEIR WORTH Conducted by the Distance Education and Training Council 2001 Foreword In July 2001 the Distance Education and Training Council surveyed

More information

9. Needed resources and preparation of a survey budget

9. Needed resources and preparation of a survey budget Last updated: 05 March.2008 HT 9. Needed resources and preparation of a survey budget Hanna Tolonen 1, 1 National Public Health Institute, Department of Health Promotion and Chronic Disease Prevention,

More information

NAPCS Product List for NAICS 54191: Marketing Research and Public Opinion Polling

NAPCS Product List for NAICS 54191: Marketing Research and Public Opinion Polling NAPCS List for NAICS 54191: Marketing Research and Public Opinion Polling 1 2 3 4 5 6 7 8 9 54191 1 Marketing research and public opinion polling services Gathering, analyzing, and interpreting information

More information

Texas Commission on Fire Protection Report on Customer Service Surveys, 2014

Texas Commission on Fire Protection Report on Customer Service Surveys, 2014 Texas Commission on Fire Protection Report on Customer Service Surveys, 2014 Submitted June 1, 2014 Table of Contents Identification of customers... 5 Survey development... 6 Survey response analysis...

More information

A First Look at Attitudes Surrounding Telehealth:

A First Look at Attitudes Surrounding Telehealth: A First Look at Attitudes Surrounding Telehealth: Findings from a national survey taking a first look at attitudes, usage, and beliefs of family physicians in the U.S. towards telehealth. OVERVIEW Telehealth

More information

Billing Code: 4163-18-P DEPARTMENT OF HEALTH AND HUMAN SERVICES. Agency for Toxic Substances and Disease Registry. [30Day-16-0047]

Billing Code: 4163-18-P DEPARTMENT OF HEALTH AND HUMAN SERVICES. Agency for Toxic Substances and Disease Registry. [30Day-16-0047] This document is scheduled to be published in the Federal Register on 10/13/2015 and available online at http://federalregister.gov/a/2015-25920, and on FDsys.gov Billing Code: 4163-18-P DEPARTMENT OF

More information

Integrated Direct Marketing Services

Integrated Direct Marketing Services Integrated Direct Marketing Services 5 BASIC COMPONENTS OF A LEAD MANAGEMENT PROCESS: Tips & Techniques for Better Return on Your Marketing and Sales Activities Jana L. Hyde Marketing Director 0104500-003R01

More information

Leah Ericson Computing Services, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213, lericson@andrew.cmu.edu

Leah Ericson Computing Services, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213, lericson@andrew.cmu.edu A Comparison of Forced-Choice and Mark-All-That-Apply Formats for Gathering Information on Health Insurance in the 2006 American Community Survey Content Test 1 Leah Ericson Computing Services, Carnegie

More information

NEW DATA COLLECTIONS USING TOUCHTONE DATA ENTRY

NEW DATA COLLECTIONS USING TOUCHTONE DATA ENTRY NEW DATA COLLECTIONS USING TOUCHTONE DATA ENTRY Sudarshan Jakhu and E. Mark Sauer, Bureau of Labor Statistics Sudarshan Jakhu, Bureau of Labor Statistics, Suite 4840, 2 Massachusetts Avenue, NE Washington,

More information

Data Collection Tools For Evaluation

Data Collection Tools For Evaluation Written Surveys Written surveys are questionnaires that are mailed, faxed, or dropped-off at a residence and returned via mail. Surveys of this type may also be handed-out to respondents, who complete

More information

Survey Research Program. Part of Group Health Research Institute

Survey Research Program. Part of Group Health Research Institute Survey Research Program Part of Group Health Research Institute Leading-edge health care research is your mission. It s ours, too. When you study health and illness, you need data savvy and field expertise

More information

James E. Bartlett, II is Assistant Professor, Department of Business Education and Office Administration, Ball State University, Muncie, Indiana.

James E. Bartlett, II is Assistant Professor, Department of Business Education and Office Administration, Ball State University, Muncie, Indiana. Organizational Research: Determining Appropriate Sample Size in Survey Research James E. Bartlett, II Joe W. Kotrlik Chadwick C. Higgins The determination of sample size is a common task for many organizational

More information

My bias is toward business-to-business research because that s primarily what we do.

My bias is toward business-to-business research because that s primarily what we do. Hi. This is Joe Kalinowski of Trilogy Associates, a management consulting firm based in the Research Triangle region of North Carolina. I founded the firm in 1989 in the Boston area. This guide will be

More information

Developing a Sustainable and Flexible Long-Distance Travel Survey

Developing a Sustainable and Flexible Long-Distance Travel Survey Developing a Sustainable and Flexible Long-Distance Travel Survey Lee Giesbrecht and Jonaki Bose Bureau of Transportation Statistics, 400 Seventh Street, SW, Washington, DC 20590 Abstract The Bureau of

More information

Concept Paper. Improving Training Evaluation in Organizations

Concept Paper. Improving Training Evaluation in Organizations Concept Paper On Improving Training Evaluation in Organizations Prepared by Dr. Paul Squires President 20 Community Place Morristown, NJ 07960 973-631-1607 fax 973-631-8020 www.appliedskills.com According

More information

CUSTOMER SERVICE EXCELLENCE

CUSTOMER SERVICE EXCELLENCE CUSTOMER SERVICE EXCELLENCE STANDARD 2 Table of Contents The starting point 4 Criterion 1 Customer Insight 5 1.1 Customer Identification 5 1.2 Engagement and Consultation 6 1.3 Customer Satisfaction 7

More information

Starting a Volunteer Program

Starting a Volunteer Program Program Planning Instituting a new volunteer program or reenergizing an existing one involves a process that covers a range of management functions from needs assessment, volunteer position development

More information

1.1 What is statistics? Data Collection. Important Definitions. What is data? Descriptive Statistics. Inferential Statistics

1.1 What is statistics? Data Collection. Important Definitions. What is data? Descriptive Statistics. Inferential Statistics 1.1 What is statistics? Data Collection Chapter 1 A science (bet you thought it was a math) of Collecting of data (Chap. 1) Organizing data (Chap. 2) Summarizing data (Chap. 2, 3) Analyzing data (Chap.

More information

Designing Effective Survey Instruments

Designing Effective Survey Instruments Designing Effective Survey Instruments Thomas M. Archer, Ph.D. 2007 Definitions: Handout Some Terms Data Instrumentation Quantitative and Qualitative Survey and a Questionnaire Reliability and Validity

More information

Assessing Interactive Voice Response for the National Crime Victimization Survey

Assessing Interactive Voice Response for the National Crime Victimization Survey Assessing Interactive Voice Response for the National Crime Victimization Survey Final Report Authors David Cantor Douglas Williams Draft: August 29, 2013 Prepared for: Bureau of Justice Statistics 810

More information

WEB-BASED ORIGIN-DESTINATION SURVEYS: AN ANALYSIS OF RESPONDENT BEHAVIOUR

WEB-BASED ORIGIN-DESTINATION SURVEYS: AN ANALYSIS OF RESPONDENT BEHAVIOUR WEB-BASED ORIGIN-DESTINATION SURVEYS: AN ANALYSIS OF RESPONDENT BEHAVIOUR Pierre-Léo Bourbonnais, Ph.D. Candidate, Department of Civil, Geological and Mining Engineering, École Polytechnique de Montréal,

More information

OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH

OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH WWW.CFMC.COM Copyright 2015 CONTENTS IDEA IN ACTION 2 DATA COLLECTION MODES 2 DEMOGRAPHICS MATTER 3 COSTS MATTER

More information

DATA COLLECTION IN THE STATISTICAL OFFICE OF THE REPUBLIC OF SLOVENIA (SURS)

DATA COLLECTION IN THE STATISTICAL OFFICE OF THE REPUBLIC OF SLOVENIA (SURS) Distr. GENERAL 8 October 2012 WP. 28 ENGLISH ONLY UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Seminar on New Frontiers for Statistical Data Collection (Geneva, Switzerland,

More information

Contacting respondents for survey research

Contacting respondents for survey research Contacting respondents for survey research Is email a useful method? Joanna d Ardenne and Margaret Blake November 2012 Background The question Is email a useful way of contacting potential respondents

More information

1a. Awarded Special Item Numbers: SIN 871-1: Consulting Services SIN 871-3:Survey Services

1a. Awarded Special Item Numbers: SIN 871-1: Consulting Services SIN 871-3:Survey Services Contracting with Responsive Management is easier than ever before. U.S. federal agencies and other eligible organizations can procure Responsive Management services through the General Service s Administration

More information

Research & Consultation Guidelines

Research & Consultation Guidelines Questionnaires Research & Consultation Guidelines Introduction A questionnaire is simply a tool for collecting and recording information about a particular issue of interest. It is mainly made up of a

More information

AMERICAN PASSAGES: A Literary Survey Formative Research Survey Results Executive Summary

AMERICAN PASSAGES: A Literary Survey Formative Research Survey Results Executive Summary AMERICAN PASSAGES: A Literary Survey Formative Research Survey Results Executive Summary PREPARED BY Kathleen R. Mohrle, M.A. Sarah A. Geddes, B.A. Irene F. Goodman, Ed.D. SUBMITTED TO Oregon Public Broadcasting

More information

Telephone and Mail Surveys: Advantages and Disadvantages of Each

Telephone and Mail Surveys: Advantages and Disadvantages of Each Telephone and Mail Surveys: Advantages and Disadvantages of Each Dan Zahs and Reg Baker Market Strategies, Inc. May 29, 2007 Selecting the mode of administration for a survey requires that one evaluate

More information

Selecting Research Participants

Selecting Research Participants C H A P T E R 6 Selecting Research Participants OBJECTIVES After studying this chapter, students should be able to Define the term sampling frame Describe the difference between random sampling and random

More information

Research and Research Methods What we will cover: Formal vs. Informal Qualitative vs. Quantitative Primary vs. Secondary Focus Groups In-Depth Interviews Intercept Interviews 1 Research and Research Methods

More information

Measuring Prescription Drug Misuse in the National Survey on Drug Use and Health

Measuring Prescription Drug Misuse in the National Survey on Drug Use and Health Measuring Prescription Drug Misuse in the National Survey on Drug Use and Health Presented by Larry Kroutil, RTI International Joel Kennet, SAMHSA Elizabeth Dean, RTI International Patricia LeBaron, RTI

More information

STRATEGIES FOR GROWTH SM

STRATEGIES FOR GROWTH SM STRATEGIES FOR GROWTH SM "CONSULTANTS TO THE SERVICES INDUSTRY" Comparative Advantages/ Disadvantages of Phone, Mail & E-Mail/Web-Based Surveys Prepared by: William K. Pollock President Strategies For

More information

Designing & Conducting Survey Research

Designing & Conducting Survey Research Designing & Conducting Survey Research Santa Monica College Fall 2011 Presented by: Hannah Alford, Director Office of Institutional Research 1 Workshop Overview Part I: Overview of Survey Method Paper/Pencil

More information

Labour Force Survey s Electronic Collection

Labour Force Survey s Electronic Collection Labour Force Survey s Electronic Collection Collection strategy, test results and transition Anne-Marie Côté and Guy Laflamme, Statistics Canada September 4 th, 2014 Outline A. Canadian LFS particularities

More information

How are polls conducted? by Frank Newport, Lydia Saad, David Moore from Where America Stands, 1997 John Wiley & Sons, Inc.

How are polls conducted? by Frank Newport, Lydia Saad, David Moore from Where America Stands, 1997 John Wiley & Sons, Inc. How are polls conducted? by Frank Newport, Lydia Saad, David Moore from Where America Stands, 1997 John Wiley & Sons, Inc. Public opinion polls would have less value in a democracy if the public -- the

More information

DATA COLLECTION IN THE U.S. BUREAU OF LABOR STATISTICS CURRENT EMPLOYMENT STATISTICS SURVEY

DATA COLLECTION IN THE U.S. BUREAU OF LABOR STATISTICS CURRENT EMPLOYMENT STATISTICS SURVEY Distr. GENERAL 8 October 2012 WP. 20 ENGLISH ONLY UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Seminar on New Frontiers for Statistical Data Collection (Geneva, Switzerland,

More information

THREE BOOKLET. COLLECTING AND ANALYZING EVALUATION DATA 2nd edition. Outreach Evaluation Resource Center

THREE BOOKLET. COLLECTING AND ANALYZING EVALUATION DATA 2nd edition. Outreach Evaluation Resource Center Planning and Evaluating Health Information Outreach Projects THREE BOOKLET COLLECTING AND ANALYZING EVALUATION DATA 2nd edition Outreach Evaluation Resource Center 2013 National Network of Libraries of

More information

Hospital Survey on Patient Safety Culture

Hospital Survey on Patient Safety Culture Hospital Survey on Patient Safety Culture Prepared for: Agency for Healthcare Research and Quality U.S. Department of Health and Human Services 540 Gaither Road Rockville, MD 20850 http://www.ahrq.gov

More information

Students Perceptions of Effective Teaching in Distance Education

Students Perceptions of Effective Teaching in Distance Education Students Perceptions of Effective Teaching in Distance Education Albert Johnson, M.Ed. (IT) Senior Instructional Designer, Distance Education and Learning Technologies Trudi Johnson, PhD Associate Professor,

More information

Developing and implementing surveys

Developing and implementing surveys Developing and implementing surveys This fact sheet points to some key considerations in developing and implementing surveys to ensure you get interesting and useful information and data. What are surveys?

More information

Revisiting the Survey Form: The Effects of Redesigning the Current Employment Statistics Survey s Iconic 1-Page Form with a Booklet Style Form

Revisiting the Survey Form: The Effects of Redesigning the Current Employment Statistics Survey s Iconic 1-Page Form with a Booklet Style Form Revisiting the Survey Form: The Effects of Redesigning the Current Employment Statistics Survey s Iconic 1-Page Form with a Booklet Style Form Louis J. Harrell, Jr. and Edward Park Bureau of Labor Statistics

More information

Continuing Education and Knowledge Retention: A Comparison of Online and Face-to-Face Deliveries

Continuing Education and Knowledge Retention: A Comparison of Online and Face-to-Face Deliveries Grand Valley State University ScholarWorks@GVSU Articles University Libraries 4-1-2007 Continuing Education and Knowledge Retention: A Comparison of Online and Face-to-Face Julie A. Garrison Grand Valley

More information

OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006. Table of Contents

OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006. Table of Contents OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006 Table of Contents LIST OF STANDARDS FOR STATISTICAL SURVEYS... i INTRODUCTION... 1 SECTION 1 DEVELOPMENT

More information

CROATIAN BUREAU OF STATISTICS REPUBLIC OF CROATIA MAIN (STATISTICAL) BUSINESS PROCESSES INSTRUCTIONS FOR FILLING OUT THE TEMPLATE

CROATIAN BUREAU OF STATISTICS REPUBLIC OF CROATIA MAIN (STATISTICAL) BUSINESS PROCESSES INSTRUCTIONS FOR FILLING OUT THE TEMPLATE CROATIAN BUREAU OF STATISTICS REPUBLIC OF CROATIA MAIN (STATISTICAL) BUSINESS PROCESSES INSTRUCTIONS FOR FILLING OUT THE TEMPLATE CONTENTS INTRODUCTION... 3 1. SPECIFY NEEDS... 4 1.1 Determine needs for

More information

SURVEY REPORT DATA SCIENCE SOCIETY 2014

SURVEY REPORT DATA SCIENCE SOCIETY 2014 SURVEY REPORT DATA SCIENCE SOCIETY 2014 TABLE OF CONTENTS Contents About the Initiative 1 Report Summary 2 Participants Info 3 Participants Expertise 6 Suggested Discussion Topics 7 Selected Responses

More information

Resource 6 Workplace travel survey guide

Resource 6 Workplace travel survey guide Resource 6 Workplace travel survey guide Page 1 Resource 6 Workplace travel survey guide Overview Introduction Contents The NZ Transport Agency (NZTA) provides a workplace travel survey (hereafter referred

More information

Research Overview: Telephone versus Online Research Advantages and Pitfalls

Research Overview: Telephone versus Online Research Advantages and Pitfalls . 26030 Highway 74, Suite A, P.O. Box 1094 Kittredge, CO 80457 P: 303.679.6300 F: 303.679.6680 info@praxigroup.net www.praxigroup.net Research Overview: Telephone versus Online Research Advantages and

More information

The Challenges and Potentials of Evaluating Courses Online

The Challenges and Potentials of Evaluating Courses Online The Challenges and Potentials of Evaluating Courses Online Mark Troy Texas A&M University Hossein Hakimzadeh Indiana University February 23, 2009 Trav D. Johnson Brigham Young University Dawn M. Zimmaro

More information

December 2013. Small Business Employee Survey

December 2013. Small Business Employee Survey December 2013 Small Business Employee Survey INTRODUCTION Technology has changed the way we communicate, socialize and do business. Every day, millions of employees in the US use some form of technology

More information