Introduction to Survey Methodology. Professor Ron Fricker Naval Postgraduate School Monterey, California

Size: px
Start display at page:

Download "Introduction to Survey Methodology. Professor Ron Fricker Naval Postgraduate School Monterey, California"

Transcription

1 Introduction to Survey Methodology Professor Ron Fricker Naval Postgraduate School Monterey, California 1

2 Goals for this Lecture Introduce professor and course Define what we mean by the term survey Characteristics of typical surveys Distinction between polls and surveys Lay out the basic steps in conducting a rigorous research survey Discuss various survey modes Pros and cons Trade-offs Considerations in making mode selection 2

3 Contact Information Professor Ron Fricker Phone Commercial: DSN: Fax:

4 Professional Experience Academic credentials Ph.D. and M.A. in Statistics, Yale University M.S. in Ops Research, The George Washington University B.S. in Mathematics from the United States Naval Academy Teaching credentials Started teaching post-graduate courses in mid-80s Have taught at NPS, RAND Graduate School, and USC Real world credentials Former active duty naval officer Senior Statistician specializing in survey and military research at the RAND Corporation Can find out more at 4

5 Class is All About Collecting and Analyzing Survey Data In this class you ll learn: How to design good survey questions and craft an effective survey instrument About the trade-offs between various survey modes The rudiments of sampling for surveys Some statistical methods useful for analyzing survey data 5

6 Course Schedule 6

7 Course Materials Lecture notes: Binder with all slides Textbook: Survey Methodology Also excerpts from Mail and Internet Surveys and Sampling: Design and Analysis Software: JMP version 7.0 All on-line at Readings password protected: MCOTEA 7

8 What is a Survey? A survey is a: systematic method for gathering information from (a sample of) entities for the purposes of constructing quantitative descriptors of the attributes of the larger population of which the entities are members In many ways, a survey is just a form of data collection 8

9 A Special Type of Data Collection Typically surveys: Gather information by asking people questions Collect information by either (1) interviewers asking questions and recording responses or (2) respondents reading and recording their own answers Collect information from a subset of the population, a sample, rather than from all members 9

10 Polls vs. Surveys No clear distinction between the two terms Poll often used for private sector opinion studies Use many of the same design features as studies that would be called surveys Poll rarely used to describe government or scientific surveys To me, the term poll implies either a commercial or less-scientific study, or a quick turn-around survey whose results may be of short-term interest 10

11 Steps in Conducting a Survey (1) Clearly state research objective(s) Decide on survey mode(s) How will you contact potential respondents? Contact mode web, , phone, etc. In what media will the survey be given? Survey or response mode web, , phone, etc. How will you follow up with non-respondents? Follow-up mode web, , phone, etc. Determine fielding strategy How to maximize response rates? Design the survey questions and the survey instrument 11

12 Steps in Conducting a Survey (2) Determine sample size and sampling strategy Obtain Institutional Review Board (IRB) or other approval as necessary Are respondents promised confidentiality? What is the impact if their survey responses become known? Pre-test, pre-test, pre-test Give the survey to some test subjects and get their feedback What works and what doesn t? Are you getting correct data/information? Revise and re-pre-test as necessary Draw sample and field the survey 12

13 Steps in Conducting a Survey (3) Follow up with non-respondents Assemble the survey data Clean data as necessary Analyze the data and report results Summarize the data Weight as necessary and appropriate to infer back to population Calculate and report margin of error Evidence of bias? Unit and item non-response Communicate survey results to respondents if promised 13

14 One View of the Steps 14

15 Another View of the Steps Excerpted from What is a Survey? by Fritz Scheuren 15

16 Examples of Large-scale Surveys (from your textbook) National Crime Victimization Survey National Survey on Drug Use and Health Survey of Consumers National Assessment of Educational Progress Behavioral Risk Factor Surveillance System Current Employment Statistics Program 16

17 National Crime Victimization Survey (NCVS) 17

18 National Survey on Drug Use and Health (NSDUH) 18

19 Survey of Consumers (SOC) 19

20 National Assessment of Educational Progress (NAEP) 20

21 Behavioral Risk Factor Surveillance System (BRFSS) 21

22 Current Employment Statistics (CES) Program 22

23 Examples of DoD Surveys Annual Active Duty Personnel Survey Survey of MWR Services and Family Programs Military Recruiter Quality of Life Survey Adult Influencer Poll (AIP) Youth Attitude Tracking Study (now JAMRS Youth Poll) 23

24 Which Survey Mode Should I Use? You are going to conduct a research survey Goal is to make the best inference possible from sample to population within budget constraints To important questions: What is the most appropriate method to choose for a particular research question? What is the impact of choosing a particular method on survey cost and accuracy? 24

25 Traditional Survey Modes Mail Paper questionnaire sent to respondents Self-administered and mailed back Telephone Interviewers call respondents on telephone Interviewer-assisted In-person Interviewers go to respondent s home or office Interviewer-assisted 25

26 Options Have Multiplied in the Age of Computers and Telecom CATI TDE IVR/T-ACASI CAPI Text- CASI Audio- CASI Video- CASI CAI Methods 26

27 Mode Considerations Modes differ by: Degree of interviewer involvement Degree of interaction with respondent Degree of respondent privacy Channels of communication Technology usage Considerations: Cost Timeliness Accuracy 27

28 Broadly speaking Interviewer-assisted surveys more costly than selfadministered Face-to-face most expensive, then telephone, then mail, then web Interviewer-assisted surveys less prone to various errors and item nonresponse Particularly compared to self-administered paper survey More or less compared to self-administered computerassisted surveys depending on sophistication of program Self-administered less subject to sensitivity bias Can combine some self-administered questions into a predominantly interviewer-assisted mode 28

29 However, timeliness is not just about the length of time a survey is in the field Really, it s from instrument design through analysis Much of that is not affected by survey mode Compared to a mail survey, web-based surveys save on two main things: Mailing time to send out and return surveys Time required to do data entry, coding, and cleaning for paper surveys Yet, it can also take a lot of time to code, test, and de-bug very complicated web (more generally, computer-assisted) survey programs 29

30 And, cost is not just about manhours On large mail surveys, printing and postage costs can be significant 1,500 surveys x 4 mailings at $1.50 each = $9,000! 1,500 $1 incentives = $1,500 On telephone surveys must consider fixed costs of phone and CATI equipment + variable costs On face-to-face interviews, travel costs can be significant (not to mention interviewer time) But in terms of manhours, don t forget: (1) instrument design time; (2) in non-electronic modes, data entry and coding time; (3) non-response follow-up efforts; and (4) analysis time 30

31 And accuracy is not just about response rates and missing items Total survey error: what mode or modes will most decrease all types of error? Lots of considerations Frame coverage Nonresponse rates Sensitivity bias, etc 31

32 32

33 Cost and Frame Availability Often Drive Mode Choice Survey budgets never unlimited Frequently cost will eliminate one or more modes And likely drive other survey design decisions I.e., face-to-face interviews too expensive On the other hand, surveys using area probability frames usually have to start out using face-to-face interviews Similarly, if a complete frame is available, web survey is possible option But general population surveys eliminate web 33

34 Some Modes Logically Group (1) Telephone and face-to-face surveys Telephone often considered an alternative to faceto-face Both interviewer-assisted and can have similar frame coverage issues Many longitudinal surveys start out with a face-toface interview with telephone follow-ups Mail sometimes an alternative to telephone If both telephone and addresses available 34

35 Some Modes Logically Group (2) Web surveys often considered a replacement for mail surveys Coverage is still an issue for web surveys Web superior in terms of automation and speed However, literature consistently finds that nonresponse rates are higher for web surveys 35

36 Using Multiple Modes Reasons to use multiple modes: Maximize response rates Reduce/eliminate selection & coverage bias Allow respondents to use most convenient mode Reduce costs Cross-sectional surveys: start with least costly mode and move to more expensive modes as necessary to get response Longitudinal surveys: Use less expensive modes in follow-on survey rounds 36

37 Considerations When Using Multiple Modes Focus should be on designing instruments to ensure equivalency across modes Rather than optimizing for a particular mode E.g., make the follow-on web survey resemble its counterpart paper survey Ensure operation is set up to track cases across the modes Must avoid duplications and uncoordinated respondent contact Must design and field so that mode effects can be disentangled from other sample characteristics May have to randomize survey mode over a subset 37

38 Other Data Collection Methods Surveys are not the only (nor necessarily the best) way to collect data Other methods include Administrative records Focus groups and qualitative investigations Randomized experiments Which is best depends on the research question(s) and/or the purpose for which the data will be used 38

39 What We Have Covered Defined the term survey, including the Characteristics of types of surveys this class will focus on Distinction between polls and surveys Discussed the basic steps in conducting a rigorous research survey Considerations when choosing a survey mode In various ways, choice likely to affect survey Timeliness Accuracy Cost And cost may drive mode choice possibilities 39

BACKGROUND ON THE SURVEY PROCESS

BACKGROUND ON THE SURVEY PROCESS Chapter Two BACKGROUND ON THE SURVEY PROCESS In this chapter, we present an overview of the various aspects of the research survey process. 1 We emphasize that surveying should first be thought of as a

More information

Statistical Methods for Sample Surveys (140.640)

Statistical Methods for Sample Surveys (140.640) This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this

More information

Self-administered Questionnaire Survey

Self-administered Questionnaire Survey Primary methods of survey data collection Self-administered questionnaire (SAQ) Mail Face-to-face interview Telephone interview Self-administered Questionnaire Survey 1 Steps are: a. Mail distribution

More information

THE FUTURE OF INTERNET-BASED SURVEY METHODS

THE FUTURE OF INTERNET-BASED SURVEY METHODS Chapter Seven CONCLUSIONS In this chapter, we offer some concluding thoughts on the future of Internet-based surveys, the issues surrounding the use of e-mail and the Web for research surveys, and certain

More information

The AmeriSpeak ADVANTAGE

The AmeriSpeak ADVANTAGE OVERVIEW Funded and operated by NORC at the University of Chicago, AmeriSpeak TM is a probabilitybased panel (in contrast to a non-probability panel). Randomly selected households are sampled with a known,

More information

Internet Surveys. Examples

Internet Surveys. Examples Web material accompanying The International Handbook of Survey Methodology Chapter 14 Internet Surveys Katja Lozar Manfreda Vasja Vehovar University of Ljubljana Examples Below follow examples and more

More information

Informing Public Policy Through Rigorous Survey Research

Informing Public Policy Through Rigorous Survey Research Informing Public Policy Through Rigorous Survey Research Data / Insight / Impact Abt SRBI is a recognized leader in utilizing survey, opinion, and public policy research to build bridges between decision-makers

More information

Mixed-Mode Methods for Conducting Survey Research

Mixed-Mode Methods for Conducting Survey Research Mixed-Mode Methods for Conducting Survey Research Herbert M. Baum, Ph.D.; Anna Chandonnet M.A.; Jack Fentress M.S., M.B.A.; and Colleen Rasinowich, B.A. www.datarecognitioncorp.com In November 2010 the

More information

Section 2: Ten Tools for Applying Sociology

Section 2: Ten Tools for Applying Sociology Section 2: Ten Tools for Applying Sociology CHAPTER 2.6: DATA COLLECTION METHODS QUICK START: In this chapter, you will learn The basics of data collection methods. To know when to use quantitative and/or

More information

Using Surveys for Data Collection in Continuous Improvement

Using Surveys for Data Collection in Continuous Improvement Innovation Insight Series Number 14 http://www.psu.edu/president/pia/innovation/ Across the University many people use data in assessment and decision-making. Data-based decision-making, an essential element

More information

National Postsecondary Student Aid Study

National Postsecondary Student Aid Study National Postsecondary Student Aid Study Jennifer S. Wine John A. Riccobono Education Studies Division RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org What

More information

Survey Research. Classifying surveys on the basis of their scope and their focus gives four categories:

Survey Research. Classifying surveys on the basis of their scope and their focus gives four categories: Survey Research Types of Surveys Surveys are classified according to their focus and scope (census and sample surveys) or according to the time frame for data collection (longitudinal and cross-sectional

More information

Use of Text Messaging to Increase Response Rates

Use of Text Messaging to Increase Response Rates Use of Text Messaging to Increase Response Rates Key words: BRFSS, response rate, cell phones, smart phones, text message Prepared by: Piper DuBray, Kristie Healey ICF International 0 Introduction Survey

More information

Lessons Learned From an NHIS Web- Followback Feasibility Study

Lessons Learned From an NHIS Web- Followback Feasibility Study Lessons Learned From an NHIS Web- Followback Feasibility Study Sarah S. Joestl, DrPH Senior Service Fellow Survey Planning and Special Surveys Branch Division of Health Interview Statistics Division of

More information

Descriptive Methods Ch. 6 and 7

Descriptive Methods Ch. 6 and 7 Descriptive Methods Ch. 6 and 7 Purpose of Descriptive Research Purely descriptive research describes the characteristics or behaviors of a given population in a systematic and accurate fashion. Correlational

More information

www.doncaster.gov.uk

www.doncaster.gov.uk Market Research It is essential that market research is undertaken to establish there is a need for childcare within a specified area and have knowledge of the local community that a business wishes to

More information

LITERATURE REVIEW OF WEB AND E-MAIL SURVEYS

LITERATURE REVIEW OF WEB AND E-MAIL SURVEYS Chapter Three LITERATURE REVIEW OF WEB AND E-MAIL SURVEYS In this chapter, we examine what has been written about Internet surveys in the literature, specifically Web and e-mail surveys. We address the

More information

Data Collection and Sampling OPRE 6301

Data Collection and Sampling OPRE 6301 Data Collection and Sampling OPRE 6301 Recall... Statistics is a tool for converting data into information: Statistics Data Information But where then does data come from? How is it gathered? How do we

More information

Appendix I: Methodology

Appendix I: Methodology Appendix I: Methodology SSRS METHODOLOGY SSRS conducted a survey of Muslims and Jews for the Institute for Social Policy and Understanding from January 18 through January 27, 2016. The study investigated

More information

Avoiding Bias in the Research Interview

Avoiding Bias in the Research Interview Avoiding Bias in the Research Interview Sonja I. Ziniel, MA Ph.D. Clinical Research Program, Children s Hospital Boston Department of Medicine, Harvard Medical School sonja.ziniel@childrens.harvard.edu

More information

Optimising survey costs in a mixed mode environment

Optimising survey costs in a mixed mode environment Optimising survey costs in a mixed mode environment Vasja Vehovar 1, Nejc Berzelak 2, Katja Lozar Manfreda 3, Eva Belak 4 1 University of Ljubljana, vasja.vehovar@fdv.uni-lj.si 2 University of Ljubljana,

More information

Exploring New Methods of Data Gathering in Long-Distance Passenger Travel Data

Exploring New Methods of Data Gathering in Long-Distance Passenger Travel Data Exploring New Methods of Data Gathering in Long-Distance Passenger Travel Data Ben Pierce, Battelle June 8, 2011 1 Why Are New Methods Needed? Declining Response Rates Survey saturation in US Growing cell-only

More information

James E. Bartlett, II is Assistant Professor, Department of Business Education and Office Administration, Ball State University, Muncie, Indiana.

James E. Bartlett, II is Assistant Professor, Department of Business Education and Office Administration, Ball State University, Muncie, Indiana. Organizational Research: Determining Appropriate Sample Size in Survey Research James E. Bartlett, II Joe W. Kotrlik Chadwick C. Higgins The determination of sample size is a common task for many organizational

More information

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives Fulfilling the core market research educational needs of individuals and companies worldwide Presented through a unique partnership between How to Contact Us: Phone: +1-706-542-3537 or 1-800-811-6640 (USA

More information

Questionnaire Design. Questionnaires in Clinical Research

Questionnaire Design. Questionnaires in Clinical Research Questionnaire Design Theresa A Scott, MS Department of Biostatistics theresa.scott@vanderbilt.edu D-2217 MCN; 3-1713 http://biostat.mc.vanderbilt.edu/theresascott Theresa A Scott, MS (Biostatistics) Questionnaire

More information

How do we know what we know?

How do we know what we know? Research Methods Family in the News Can you identify some main debates (controversies) for your topic? Do you think the authors positions in these debates (i.e., their values) affect their presentation

More information

see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys

see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys SURVEYS Survey = ask the same questions of a large sample see Designing Surveys by Ron Czaja and Johnny Blair for more information on surveys Basic Survey Process: 1. Design Questionnaire 2. Design Sample

More information

Use of Text Messaging to Increase Response Rates

Use of Text Messaging to Increase Response Rates Introduction Use of Text Messaging to Increase Response Rates Piper DuBray ICF International 126 College St. Burlington, VT 05401 Proceedings of the 2013 Federal Committee on Statistical Methodology (FCSM)

More information

How to gather and evaluate information

How to gather and evaluate information 09 May 2016 How to gather and evaluate information Chartered Institute of Internal Auditors Information is central to the role of an internal auditor. Gathering and evaluating information is the basic

More information

The application of alternative modes of data collection in UK Government social surveys. A report for the Government Statistical Service

The application of alternative modes of data collection in UK Government social surveys. A report for the Government Statistical Service The application of alternative modes of data collection in UK Government social surveys A report for the Government Statistical Service Peter Betts and Charles Lound Office for National Statistics May

More information

2. Choose the location and format for focus groups or interviews

2. Choose the location and format for focus groups or interviews Steps for Conducting Focus Groups or Individual In-depth Interviews 1. Plan the study Plan your study with the input of your strategy team, stakeholders, and research experts so that collaboratively you

More information

Research design and methods Part II. Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT

Research design and methods Part II. Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT Research design and methods Part II Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT From last week Research methodology Quantitative vs. Qualitative vs. Participatory/action research Research methods

More information

Chapter 2: Research Methodology

Chapter 2: Research Methodology Chapter 2: Research Methodology 1. Type of Research 2. Sources of Data 3. Instruments for Data Collection 4. Research Methods 5. Sampling 6. Limitations of the Study 6 Chapter 2: Research Methodology Research

More information

Descriptive Inferential. The First Measured Century. Statistics. Statistics. We will focus on two types of statistical applications

Descriptive Inferential. The First Measured Century. Statistics. Statistics. We will focus on two types of statistical applications Introduction: Statistics, Data and Statistical Thinking The First Measured Century FREC 408 Dr. Tom Ilvento 213 Townsend Hall ilvento@udel.edu http://www.udel.edu/frec/ilvento http://www.pbs.org/fmc/index.htm

More information

OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH

OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH WWW.CFMC.COM Copyright 2015 CONTENTS IDEA IN ACTION 2 DATA COLLECTION MODES 2 DEMOGRAPHICS MATTER 3 COSTS MATTER

More information

Face-to-Face vs. Online Professional Development? Do Both! The Power of the Blended Model

Face-to-Face vs. Online Professional Development? Do Both! The Power of the Blended Model Face-to-Face vs. Online Professional Development? Do Both! The Power of the Blended Model Dr. Paula Hidalgo, Director of Product Development Dr. Paula Hidalgo is the Director of Product Development for

More information

05/10/2015. Chapter 3 - Marketing Research. Marketing Project Plan

05/10/2015. Chapter 3 - Marketing Research. Marketing Project Plan Chapter 3 - Marketing Research Copyright 2013 Pearson Canada Inc. Marketing Project Plan How do we begin the project? Identify product What do we need? We need to gather information We also need to do

More information

Leah Ericson Computing Services, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213, lericson@andrew.cmu.edu

Leah Ericson Computing Services, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213, lericson@andrew.cmu.edu A Comparison of Forced-Choice and Mark-All-That-Apply Formats for Gathering Information on Health Insurance in the 2006 American Community Survey Content Test 1 Leah Ericson Computing Services, Carnegie

More information

THREE BOOKLET. COLLECTING AND ANALYZING EVALUATION DATA 2nd edition. Outreach Evaluation Resource Center

THREE BOOKLET. COLLECTING AND ANALYZING EVALUATION DATA 2nd edition. Outreach Evaluation Resource Center Planning and Evaluating Health Information Outreach Projects THREE BOOKLET COLLECTING AND ANALYZING EVALUATION DATA 2nd edition Outreach Evaluation Resource Center 2013 National Network of Libraries of

More information

Electronic Mail Vs. Mail Survey Response Rates

Electronic Mail Vs. Mail Survey Response Rates Electronic Mail Vs. Mail Survey Response Rates Researchers can no longer ignore the possihilities computer technology hold for the future of our profession. By Barbara A. Schuldt and Jeff W. Totten An

More information

How are polls conducted? by Frank Newport, Lydia Saad, David Moore from Where America Stands, 1997 John Wiley & Sons, Inc.

How are polls conducted? by Frank Newport, Lydia Saad, David Moore from Where America Stands, 1997 John Wiley & Sons, Inc. How are polls conducted? by Frank Newport, Lydia Saad, David Moore from Where America Stands, 1997 John Wiley & Sons, Inc. Public opinion polls would have less value in a democracy if the public -- the

More information

OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006. Table of Contents

OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006. Table of Contents OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006 Table of Contents LIST OF STANDARDS FOR STATISTICAL SURVEYS... i INTRODUCTION... 1 SECTION 1 DEVELOPMENT

More information

Contacting respondents for survey research

Contacting respondents for survey research Contacting respondents for survey research Is email a useful method? Joanna d Ardenne and Margaret Blake November 2012 Background The question Is email a useful way of contacting potential respondents

More information

A First Look at Attitudes Surrounding Telehealth:

A First Look at Attitudes Surrounding Telehealth: A First Look at Attitudes Surrounding Telehealth: Findings from a national survey taking a first look at attitudes, usage, and beliefs of family physicians in the U.S. towards telehealth. OVERVIEW Telehealth

More information

Methods for Improving Response Rates: Findings from AAPOR 2012. Benjamin L. Messer Washington State University PAPOR Mini-Conference, June 22, 2012

Methods for Improving Response Rates: Findings from AAPOR 2012. Benjamin L. Messer Washington State University PAPOR Mini-Conference, June 22, 2012 Methods for Improving Response Rates: Findings from AAPOR 2012 Benjamin L. Messer Washington State University PAPOR Mini-Conference, June 22, 2012 2 Disclaimer Papers/presentations obtained with author

More information

Consumer s Guide to Research on STEM Education. March 2012. Iris R. Weiss

Consumer s Guide to Research on STEM Education. March 2012. Iris R. Weiss Consumer s Guide to Research on STEM Education March 2012 Iris R. Weiss Consumer s Guide to Research on STEM Education was prepared with support from the National Science Foundation under grant number

More information

Cash Rents Methodology and Quality Measures

Cash Rents Methodology and Quality Measures ISSN: 2167-129X Cash Rents Methodology and Quality Measures Released August 1, 2014, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States Department of Agriculture

More information

New! For Association Executives Certificate in Programming Online

New! For Association Executives Certificate in Programming Online New! For Association Executives Certificate in Programming Online Online programming is growing and will become 50% of association professional development. But Gen Y and future generations of association

More information

My bias is toward business-to-business research because that s primarily what we do.

My bias is toward business-to-business research because that s primarily what we do. Hi. This is Joe Kalinowski of Trilogy Associates, a management consulting firm based in the Research Triangle region of North Carolina. I founded the firm in 1989 in the Boston area. This guide will be

More information

Designing & Conducting Survey Research

Designing & Conducting Survey Research Designing & Conducting Survey Research Santa Monica College Fall 2011 Presented by: Hannah Alford, Director Office of Institutional Research 1 Workshop Overview Part I: Overview of Survey Method Paper/Pencil

More information

Assessing Interactive Voice Response for the National Crime Victimization Survey

Assessing Interactive Voice Response for the National Crime Victimization Survey Assessing Interactive Voice Response for the National Crime Victimization Survey Final Report Authors David Cantor Douglas Williams Draft: August 29, 2013 Prepared for: Bureau of Justice Statistics 810

More information

Concept Paper. Improving Training Evaluation in Organizations

Concept Paper. Improving Training Evaluation in Organizations Concept Paper On Improving Training Evaluation in Organizations Prepared by Dr. Paul Squires President 20 Community Place Morristown, NJ 07960 973-631-1607 fax 973-631-8020 www.appliedskills.com According

More information

ESOMAR 28 Questions. 1. What experience does your company have in providing online samples for market research?

ESOMAR 28 Questions. 1. What experience does your company have in providing online samples for market research? ESOMAR 28 Questions COMPANY PROFILE 1. What experience does your company have in providing online samples for market research? Since 1996, DoctorDirectory has specialized in deploying healthcare market

More information

MEMORANDUM FOR THE PRESIDENT S MANAGEMENT COUNCIL. Guidance on Agency Survey and Statistical Information Collections

MEMORANDUM FOR THE PRESIDENT S MANAGEMENT COUNCIL. Guidance on Agency Survey and Statistical Information Collections EXECUTIVE OFFICE OF THE PRESIDENT OFFICE OF MANAGEMENT AND BUDGET WASHINGTON, D.C. 20503 ADMINISTRATOR OFFICE OF INFORMATION AND REGULATORY AFFAIR January 20, 2006 MEMORANDUM FOR THE PRESIDENT S MANAGEMENT

More information

SURVEY REPORT DATA SCIENCE SOCIETY 2014

SURVEY REPORT DATA SCIENCE SOCIETY 2014 SURVEY REPORT DATA SCIENCE SOCIETY 2014 TABLE OF CONTENTS Contents About the Initiative 1 Report Summary 2 Participants Info 3 Participants Expertise 6 Suggested Discussion Topics 7 Selected Responses

More information

CROATIAN BUREAU OF STATISTICS REPUBLIC OF CROATIA MAIN (STATISTICAL) BUSINESS PROCESSES INSTRUCTIONS FOR FILLING OUT THE TEMPLATE

CROATIAN BUREAU OF STATISTICS REPUBLIC OF CROATIA MAIN (STATISTICAL) BUSINESS PROCESSES INSTRUCTIONS FOR FILLING OUT THE TEMPLATE CROATIAN BUREAU OF STATISTICS REPUBLIC OF CROATIA MAIN (STATISTICAL) BUSINESS PROCESSES INSTRUCTIONS FOR FILLING OUT THE TEMPLATE CONTENTS INTRODUCTION... 3 1. SPECIFY NEEDS... 4 1.1 Determine needs for

More information

Clean Slate Program, San Francisco Office of the Public Defender 2007-08 Evaluation Findings. Executive Summary

Clean Slate Program, San Francisco Office of the Public Defender 2007-08 Evaluation Findings. Executive Summary Executive Summary Introduction The Clean Slate Program extends legal advocacy beyond an arrest or disposition by the Court so that clients may avail themselves of opportunities to clean up their criminal

More information

Developing a Sustainable and Flexible Long-Distance Travel Survey

Developing a Sustainable and Flexible Long-Distance Travel Survey Developing a Sustainable and Flexible Long-Distance Travel Survey Lee Giesbrecht and Jonaki Bose Bureau of Transportation Statistics, 400 Seventh Street, SW, Washington, DC 20590 Abstract The Bureau of

More information

Labour Force Survey s Electronic Collection

Labour Force Survey s Electronic Collection Labour Force Survey s Electronic Collection Collection strategy, test results and transition Anne-Marie Côté and Guy Laflamme, Statistics Canada September 4 th, 2014 Outline A. Canadian LFS particularities

More information

WEB-BASED ORIGIN-DESTINATION SURVEYS: AN ANALYSIS OF RESPONDENT BEHAVIOUR

WEB-BASED ORIGIN-DESTINATION SURVEYS: AN ANALYSIS OF RESPONDENT BEHAVIOUR WEB-BASED ORIGIN-DESTINATION SURVEYS: AN ANALYSIS OF RESPONDENT BEHAVIOUR Pierre-Léo Bourbonnais, Ph.D. Candidate, Department of Civil, Geological and Mining Engineering, École Polytechnique de Montréal,

More information

Using Online Surveys to Evaluate Distance Education Programs

Using Online Surveys to Evaluate Distance Education Programs 1 21st Annual Conference on Distance Teaching and Learning click here -> Using Online Surveys to Evaluate Distance Education Programs Elaine Strachota, PhD Distance Learning Coordinator and Faculty Milwaukee

More information

Putting Reliable Health Care Performance Measurement Systems into Practice

Putting Reliable Health Care Performance Measurement Systems into Practice An NCQA Issue Brief 2000 L Street, NW Washington, DC 20036 888-275-7585 www.ncqa.org Putting Reliable Health Care Performance Measurement Systems into Practice By Joachim Roski, PhD MPH; Vice President,

More information

Hospital Survey on Patient Safety Culture

Hospital Survey on Patient Safety Culture Hospital Survey on Patient Safety Culture Prepared for: Agency for Healthcare Research and Quality U.S. Department of Health and Human Services 540 Gaither Road Rockville, MD 20850 http://www.ahrq.gov

More information

CUSTOMER SERVICE EXCELLENCE

CUSTOMER SERVICE EXCELLENCE CUSTOMER SERVICE EXCELLENCE STANDARD 2 Table of Contents The starting point 4 Criterion 1 Customer Insight 5 1.1 Customer Identification 5 1.2 Engagement and Consultation 6 1.3 Customer Satisfaction 7

More information

Survey Research Program. Part of Group Health Research Institute

Survey Research Program. Part of Group Health Research Institute Survey Research Program Part of Group Health Research Institute Leading-edge health care research is your mission. It s ours, too. When you study health and illness, you need data savvy and field expertise

More information

NEW DATA COLLECTIONS USING TOUCHTONE DATA ENTRY

NEW DATA COLLECTIONS USING TOUCHTONE DATA ENTRY NEW DATA COLLECTIONS USING TOUCHTONE DATA ENTRY Sudarshan Jakhu and E. Mark Sauer, Bureau of Labor Statistics Sudarshan Jakhu, Bureau of Labor Statistics, Suite 4840, 2 Massachusetts Avenue, NE Washington,

More information

Flipped Learning Evaluation Institute for Effective Education (University of York) Peter Rudd, PhD

Flipped Learning Evaluation Institute for Effective Education (University of York) Peter Rudd, PhD Evaluation Summary Age range Year 5 (9-10) Number of pupils Average 60 pupils per school Number of schools 12 treatment and 12 control schools Design Randomised Controlled Trial with Delayed Treatment

More information

DETC DEGREE PROGRAMS GRADUATES AND EMPLOYERS EVALUATE THEIR WORTH

DETC DEGREE PROGRAMS GRADUATES AND EMPLOYERS EVALUATE THEIR WORTH DETC DEGREE PROGRAMS GRADUATES AND EMPLOYERS EVALUATE THEIR WORTH Conducted by the Distance Education and Training Council 2001 Foreword In July 2001 the Distance Education and Training Council surveyed

More information

Developing Standardized Paradata and Dashboards for Use Across Multiple Surveys

Developing Standardized Paradata and Dashboards for Use Across Multiple Surveys RTI International Developing Standardized Paradata and Dashboards for Use Across Multiple Surveys Susan Mitchell, Lisa Carley-Baxter, Orin Day, Andy Peytchev, Susan Sadler-Redmond www.rti.org RTI International

More information

A Class Project in Survey Sampling

A Class Project in Survey Sampling A Class Project in Survey Sampling Andrew Gelman and Deborah Nolan July 1, 2001 Courses in quantitative methods typically require students to analyze previously collected data. There is great value in

More information

Assessment Project Management in the Real World. Hour Three: Implementation, Technology, IRB, and Challenges

Assessment Project Management in the Real World. Hour Three: Implementation, Technology, IRB, and Challenges Assessment Project Management in the Real World Hour Three: Implementation, Technology, IRB, and Challenges Implementation Creating a culture of assessment Definitions and clarifications Encourage and

More information

Whether and How to Apply to Ph.D. Programs in Political Science: FAQs

Whether and How to Apply to Ph.D. Programs in Political Science: FAQs Whether and How to Apply to Ph.D. Programs in Political Science: FAQs Matthias Staisch, The University of Chicago This Version: October 4, 2013 1 Do I need a Ph.D. to attain my professional goals? Before

More information

Billing Code: 4163-18-P DEPARTMENT OF HEALTH AND HUMAN SERVICES. Agency for Toxic Substances and Disease Registry. [30Day-16-0047]

Billing Code: 4163-18-P DEPARTMENT OF HEALTH AND HUMAN SERVICES. Agency for Toxic Substances and Disease Registry. [30Day-16-0047] This document is scheduled to be published in the Federal Register on 10/13/2015 and available online at http://federalregister.gov/a/2015-25920, and on FDsys.gov Billing Code: 4163-18-P DEPARTMENT OF

More information

DATA COLLECTION IN THE STATISTICAL OFFICE OF THE REPUBLIC OF SLOVENIA (SURS)

DATA COLLECTION IN THE STATISTICAL OFFICE OF THE REPUBLIC OF SLOVENIA (SURS) Distr. GENERAL 8 October 2012 WP. 28 ENGLISH ONLY UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Seminar on New Frontiers for Statistical Data Collection (Geneva, Switzerland,

More information

TIPSHEET IMPROVING RESPONSE SCALES

TIPSHEET IMPROVING RESPONSE SCALES TIPSHEET IMPROVING RESPONSE SCALES As important as it is to have good question wording in a survey, it is equally important to have well constructed and reliable question response options. Researchers

More information

IAB Evaluation Study of Methods Used to Assess the Effectiveness of Advertising on the Internet

IAB Evaluation Study of Methods Used to Assess the Effectiveness of Advertising on the Internet IAB Evaluation Study of Methods Used to Assess the Effectiveness of Advertising on the Internet ARF Research Quality Council Paul J. Lavrakas, Ph.D. November 15, 2010 IAB Study of IAE The effectiveness

More information

Data quality and metadata

Data quality and metadata Chapter IX. Data quality and metadata This draft is based on the text adopted by the UN Statistical Commission for purposes of international recommendations for industrial and distributive trade statistics.

More information

Measuring Prescription Drug Misuse in the National Survey on Drug Use and Health

Measuring Prescription Drug Misuse in the National Survey on Drug Use and Health Measuring Prescription Drug Misuse in the National Survey on Drug Use and Health Presented by Larry Kroutil, RTI International Joel Kennet, SAMHSA Elizabeth Dean, RTI International Patricia LeBaron, RTI

More information

Selecting Research Participants

Selecting Research Participants C H A P T E R 6 Selecting Research Participants OBJECTIVES After studying this chapter, students should be able to Define the term sampling frame Describe the difference between random sampling and random

More information

Question Bank FACTSHEET 5

Question Bank FACTSHEET 5 Question Bank FACTSHEET 5 Survey Interviewing Introduction: The Survey Interview This factsheet will describe the processes involved in the Survey interview. It will focus only on the face to face interview

More information

December 2013. Small Business Employee Survey

December 2013. Small Business Employee Survey December 2013 Small Business Employee Survey INTRODUCTION Technology has changed the way we communicate, socialize and do business. Every day, millions of employees in the US use some form of technology

More information

Supplemental Materials

Supplemental Materials Supplemental Materials How Can I Use Student Feedback to Improve My Teaching? Presented by: Ken Alford, Ph.D. and Tyler Griffin, Ph.D. 2014 Magna Publications Inc. All rights reserved. It is unlawful to

More information

STRATEGIES FOR GROWTH SM

STRATEGIES FOR GROWTH SM STRATEGIES FOR GROWTH SM "CONSULTANTS TO THE SERVICES INDUSTRY" Comparative Advantages/ Disadvantages of Phone, Mail & E-Mail/Web-Based Surveys Prepared by: William K. Pollock President Strategies For

More information

Non-Researcher s Guide to Evidence-Based Program Evaluation

Non-Researcher s Guide to Evidence-Based Program Evaluation Non-Researcher s Guide to Evidence-Based Program Evaluation July 2012 Table of Contents Table of Contents... 2 Course Overview... 4 About This Course... 4 Intended Audience... 4 Course Topics... 4 Learning

More information

List of Contents. Introduction 1

List of Contents. Introduction 1 I List of Contents Page Foreword III Introduction 1 A. General Principles 2 B. Institutional By-Laws and Regulations 2 C. Specific Guidance 3 Annexes II FOREWORD The purpose of this Code of Practice is

More information

Analyzing Qualitative Data For Assessment October 15, 2014

Analyzing Qualitative Data For Assessment October 15, 2014 Analyzing Qualitative Data For Assessment October 15, 2014 Jeremy Penn, Ph.D. Director Resources Data Analysis, Chapter 6 in Schuh book (pages 158-168) Creswell (2002) or (1998) you can borrow from me

More information

Resource 6 Workplace travel survey guide

Resource 6 Workplace travel survey guide Resource 6 Workplace travel survey guide Page 1 Resource 6 Workplace travel survey guide Overview Introduction Contents The NZ Transport Agency (NZTA) provides a workplace travel survey (hereafter referred

More information

Employee Assistance Research Foundation Announces Winners of Its First Grant Awards Program

Employee Assistance Research Foundation Announces Winners of Its First Grant Awards Program EASNA RESEARCH NOTES Volume 2, Number 2, August 2011 Employee Assistance Research Foundation Announces Winners of Its First Grant Awards Program Employee Assistance Research Foundation Public Awareness

More information

egranary EVALUATION PROJECT: Evaluating the Use, Usability, and Usefulness of the egranary Digital Library

egranary EVALUATION PROJECT: Evaluating the Use, Usability, and Usefulness of the egranary Digital Library egranary EVALUATION PROJECT: Evaluating the Use, Usability, and Usefulness of the egranary Digital Library The WiderNet Project Cliff Missen, MA, Director Thomas Cook, Ph.D. 201 Communications Center The

More information

Research Proposal Guidelines

Research Proposal Guidelines Department of Marketing Management MCOM & DCOM DEGREES Research Proposal Guidelines Compiled by Dr Roberts-Lombard A. RESEARCH PROPOSAL STRUCTURE 1. Introduction and Background to the research 2. Statement

More information

SURVEY RESEARCH RESEARCH METHODOLOGY CLASS. Lecturer : RIRI SATRIA Date : November 10, 2009

SURVEY RESEARCH RESEARCH METHODOLOGY CLASS. Lecturer : RIRI SATRIA Date : November 10, 2009 SURVEY RESEARCH RESEARCH METHODOLOGY CLASS Lecturer : RIRI SATRIA Date : November 10, 2009 DEFINITION OF SURVEY RESEARCH Survey: A method of primary data collection based on communication with a representative

More information

Supporting Statement Part A Background

Supporting Statement Part A Background Supporting Statement Part A Surveys of Physicians and Home Health Agencies to Assess Access Issues for Specific Medicare Beneficiaries as Defined in Section 3131(d) of the ACA CMS-10429, OMB 0938-New Background

More information

Learning at a Distance

Learning at a Distance DTA SU.S. EPARTMENTTS OF EDUCATIONINOCTOBBER 11RNICESE12F-154 Learning at a Distance Undergraduate Enrollment in Distance Education Courses and Degree Programs AUTHOR Alexandria Walton Radford MPR Associates,

More information

Article. Building a Business Case for Census Internet Data Collection. by Tracy Moore, Lorna Bailie and Graeme Gilmour

Article. Building a Business Case for Census Internet Data Collection. by Tracy Moore, Lorna Bailie and Graeme Gilmour Component of Statistics Canada Catalogue no. 11-522-X Statistics Canada s International Symposium Series: Proceedings Article Symposium 2008: Data Collection: Challenges, Achievements and New Directions

More information

Data Collection Tools For Evaluation

Data Collection Tools For Evaluation Written Surveys Written surveys are questionnaires that are mailed, faxed, or dropped-off at a residence and returned via mail. Surveys of this type may also be handed-out to respondents, who complete

More information

Research and Research Methods What we will cover: Formal vs. Informal Qualitative vs. Quantitative Primary vs. Secondary Focus Groups In-Depth Interviews Intercept Interviews 1 Research and Research Methods

More information

CHAPTER 3 RESEARCH METHODOLOGY

CHAPTER 3 RESEARCH METHODOLOGY CHAPTER 3 RESEARCH METHODOLOGY 3.1 Introduction This study is survey research in order to study marketing strategies for sustainable cultural tourism in Nakhon Si Thammarat. In case of primary data were

More information

PROGRAMME SPECIFICATION POSTGRADUATE PROGRAMMES

PROGRAMME SPECIFICATION POSTGRADUATE PROGRAMMES PROGRAMME SPECIFICATION POSTGRADUATE PROGRAMMES KEY FACTS Programme name Clinical Research Award Masters in Research (MRes) School Health Sciences Department or equivalent Division of Health Services Research

More information

The Challenges and Potentials of Evaluating Courses Online

The Challenges and Potentials of Evaluating Courses Online The Challenges and Potentials of Evaluating Courses Online Mark Troy Texas A&M University Hossein Hakimzadeh Indiana University February 23, 2009 Trav D. Johnson Brigham Young University Dawn M. Zimmaro

More information

PROCEDURES RELATING TO THE USE OF THE MASSEY ONLINE SURVEY TOOL (MOST - http://ost.massey.ac.nz/ost)

PROCEDURES RELATING TO THE USE OF THE MASSEY ONLINE SURVEY TOOL (MOST - http://ost.massey.ac.nz/ost) Massey University Policy Guide PROCEDURES RELATING TO THE USE OF THE MASSEY ONLINE SURVEY TOOL (MOST - http://ost.massey.ac.nz/ost) Section Academic Contact Student Survey and Evaluation Unit Last Review

More information