Basic Principles of Survey Design. Thomas Lindsay Andrew Sell

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Basic Principles of Survey Design. Thomas Lindsay Andrew Sell"

Transcription

1 Basic Principles of Survey Design Thomas Lindsay Andrew Sell

2 Introductions Name, Institution, why are you here?

3 Survey Review 5 minutes Political Participation and Attitudes Survey What is the research question being investigated? What is the audience? Is questionnaire appropriate length? Are any questions especially good? Are any questions especially problematic?

4 RESEARCH PURPOSE

5 Step 1: Define Research Purpose DO NOT start writing questions! What are you trying to learn, and why? If it can t be reduced to a single sentence, it s probably not focused enough Source:

6 Step 1: Define Research Purpose Considering Purpose Statement: What specific goals or objectives are there? What questions need to be answered in order to reach actionable outcomes?

7 Step 1: Define Research Purpose Is there a well-defined population? If so, what is the universe of this population? If not, can a segment be identified? What distinguishing characteristics do you expect in your population? Age, gender, attitudes, behaviors Are any of these relevant for research purpose?

8 Step 2: Determine Data Sources Is it necessary to collect new data? What data sources already exist?

9 Step 2: Determine Data Sources Do not assume a survey is the best way to answer the research question Consider strengths and weaknesses of all potential data sources

10 Small Group Exercise 5 min. Consider scenario What is the overarching research question? Develop objectives and desired outcomes What population will this effect? What data sources might be applicable?

11 Small Group Exercise 5 min. Imagine you are a librarian at a large university library. Your library serves faculty, students and staff throughout the university. In any given year approximately 500 faculty, 10,000 students, and 1,000 staff work at or attend the university. Recently your supervisor asked you to create a plan to introduce social media into the mix of services your library provides. She mentioned that she expects you to survey library users to gather their opinions about social media and incorporate these insights into your plan.

12 Discussion Purpose Definition What is the overarching research question? What are objectives of project? What are desired outcomes? What population will this affect? What data sources might be applicable?

13 Data Sources, continued Secondary Research Previous research Census or other public data

14 Data Sources, continued Non-self-report Primary Research Observation Usage Data Website Traffic Physical Traffic

15 Data Sources Self Report Self-Report Primary Research Interviews Focus Groups Surveys

16 Data Sources Self Report May be qualitative or quantitative, or both Is self-report data compatible with objectives? Is it actionable?

17 Population - Surveys Participant Pool: Census attempt to reach entire population Difficult to reach entire population Non-respondents are likely different from respondents Sample Contact small portion of population Difficult to define sample Matching sampling approach to research needs can be challenging

18 Probability Sampling Generally requires that you can define your population very well People are excluded randomly, not categorically Examples: Simple Random Sample Must be truly randomly drawn from entire population Stratified Sample Requires knowledge of key segmenting characteristics of population

19 Nonprobability Sampling Does not require knowledge of population Sampling Error is Unknowable Examples: Convenience Sample Subgroups that are easier to reach than others Quota Sample Participants are selected to meet quotas based on researcher interest rather than known population characteristics

20 Survey Modes Paper mail, lab, face-to-face Online , lab, web Telephone

21 BREAK

22 QUESTIONNAIRE DESIGN

23 Paper and Online Surveys Paper Universal Access Allows RICH responses Limited control over respondent Space is extremely limited Branching is awkward, space-consuming, and visible Online Digital Divide No chance to provide unexpected data Allows greater control over respondent Much more space available Branching can be complex and time-saving

24 Paper Survey

25 Online Survey

26 Question Types Open-Ended Blank space or box for free response in participant s own words Closed-Ended / Scalar List of predefined answer choices from which participants must choose to answer

27 Question Types Open-Ended Response is minimally influenced by researcher Provides rich information Especially good when researcher knows little about question ahead of time For numerical data, allows great precision Closed-Ended Structures and shapes answer choices Reduces error from interpretation Easy for participant to answer Lower incidence of nonresponse Easier to analyze

28 Question Types Open-Ended Fill-in-the-blank Free text Numeric (Other, please specify) Essay Closed-Ended Choose Many Choose One Scale Rank Order

29 Question Types

30 Data Types Ratio Interval Ordinal Nominal

31 Data Types - Ratio Data points are ordered Interval between items is constant Natural zero-point exists Examples Height Age Temperature - Kelvin

32 Data Types - Interval Data points are ordered Interval between items is constant No natural zero-point Examples Date Temperature Celsius, Fahrenheit

33 Data Types - Ordinal Data points are ordered Interval between items is unknown or not constant Examples Rankings (e.g., school rankings, Olympic medals) Academic Degree Attained

34 Data Types Nominal Data points have no order (are equivalent) Categories are mutually exclusive Examples Gender Species Musical genre -? Color -???

35 Small Group Exercise 5 min. Develop and write questions for survey Consider desired data type for answers Consider desired question types Do not write answer choices (that s later!)

36 SCALE ITEMS

37 Scale Items Should you use a scale? If question is clearly dichotomous don t use scale True/false, yes/no Nominal items are not compatible with scales Scales are usually interval or ordinal, can be ratio

38 Scale Items Two-sided Scales (Bipolar) Anchors have equal valence at opposite sides of a neutral mid-point One-sided Scales (Unipolar) Low anchor represents absence of property, zero point, or least valence High anchor represents high point or greatest valence

39 Scale Items Two-Sided Scales Positive neutral negative Typically, sides should be balanced in number of items and degree of labels Rarely, research interests may dictate otherwise Scale should have a neutral midpoint Absence of reasonable midpoint may indicate that scale is single-sided

40 Scale Items Two-Sided Scales Labeled neutral midpoint improves reliability Two or three levels of differentiation on either side of neutral In all, five or seven answer choices

41 Scale Items Two-Sided Scales

42 Scale Items Two-Sided Scales

43 Scale Items One-Sided Scales Zero Maximum Measure levels or gradation, not direction Scale does not have a midpoint ( neutral ) Make sure bottom point represents absence or very lowest presence of measured trait Optimal length is generally four, or sometimes five, categories

44 Scale Items One-Sided Scales

45 Scale Items One-Sided Scales

46 Scale Items One or Two Sides?

47 Scale Items Validity Don t make your participants lie! If acceptable choice is not present, participant must either close survey, skip question, or lie Scales should include full range of choices Anchors should reasonably segment range Labels should be easy for participants to interpret and evaluate

48 Scale Items - Validity Two-sided scales should include neutral midpoint One-sided scales should start at zero point, e.g., none, not at all, never If never is truly unexpected, it can be combined with next-lowest choice, e.g., never or rarely

49 SHAPING RESPONSE

50 Response Construct-Specific Labels Choice labels that directly describe the choice ( very anxious ) are more reliable than indirect labels ( strongly agree ) Before deciding how to answer agreement scales, participants must analyze original statement Direct statements do not require this extra step Direct statements are not always feasible

51 Response Construct-Specific Labels

52 Response Anchoring Effects Two Types: Extremity and/or wording of question labels (especially endpoints) influences choice selection Early questions affect perception of later questions

53 Response Anchoring Effects

54 Response Anchoring Effects

55 Response Anchoring Effects

56 Response Anchoring Effects

57 Response Acquiescence Bias Social Desirability Participants acquiesce toward choices they view as more socially acceptable People also tend away from extremes Leading Questions Question text may indicate researcher s desired or expected answers

58 Response Acquiescence Bias

59 Response Acquiescence Bias

60 Response Acquiescence Bias Leading Questions: If using two-sided scale, make sure both options are in the question How much do you agree should be How much do you agree or disagree Avoid words that pass judgment when possible Do you prefer high art or popular music? could be reframed as Do you prefer opera or rock and roll music?

61 Response Recency Bias Respondents heavily weight recent events and discount less recent events Questions should scope timing whenever possible Avoid Typically, how often do you or In the last five years, how often have you

62 Response Free Response Free response questions need to be specific and direct Respondents are less likely to respond to vague, overarching questions Participants who respond will say more if question is specific Space given implies expected answer length

63 Response Free Response

64 Small Group Exercise 10 min. Refine and edit questions Create answer choices

65 BREAK

66 Demographics Only ask demographics that are needed If you don t need to know income, don t ask it If your population is uniform in level of education, you already know it don t ask it Know the context of your audience Gender / sex and available choices Race / Ethnicity and available choices

67 PARTICIPANT EXPERIENCE

68 Participant Experience Participants are people too! If burdens of time, comprehension, or confidentiality are too much, participant may: Exit the survey Answer dishonestly Lose focus Become distracted by other things

69 Participant Experience Survey Flow Survey is an interview, conversation Avoid placing probing questions early Build and transition smoothly and logically Avoid tricking respondents Consider effects question order may have

70 Participant Experience Time How much of your participant s time are you expecting to take? BE REALISTIC! Temporality Consider what is going on for your population at the time you re collecting data Holidays, work loads, semesters, other similar surveys, etc.

71 Survey Review 5 minutes Small Group Surveys How does it look? How long will it take? Is it asking the right questions? Are any topics missing? Will the results address the research question? Can actions be taken based on responses?

72 Survey Review 5 minutes Political Participation and Attitudes Survey What is the research question being investigated? What is the audience? Is questionnaire appropriate length? Are any questions especially good? Are any questions especially problematic?

73 Resources Dillman, Don, Jolene D. Smyth, and Leah Melani Christian. Internet, Mail, and Mixed Mode Surveys: the Tailored Design Method. Hoboken, NJ: John Wiley & Sons, Inc., Weisberg, Herbert F., Jon A. Krosnick, Bruce D. Bowen. An introduction to survey research, polling, and data analysis. Thousand Oaks, Calif. : Sage Publications,1996. McDaniel, Carl, Jr., Roger Gates. Marketing Research. Hoboken, NJ: John Wiley & Sons, Inc., 2010.

74 Discussion Thank you!

Survey research. Contents. Chaiwoo Lee. Key characteristics of survey research Designing your questionnaire

Survey research. Contents. Chaiwoo Lee. Key characteristics of survey research Designing your questionnaire Survey research Chaiwoo Lee Postdoctoral Associate MIT AgeLab chaiwoo@mit.edu agelab.mit.edu Contents Key characteristics of survey research Designing your questionnaire Writing the questions Putting them

More information

TIPSHEET IMPROVING RESPONSE SCALES

TIPSHEET IMPROVING RESPONSE SCALES TIPSHEET IMPROVING RESPONSE SCALES As important as it is to have good question wording in a survey, it is equally important to have well constructed and reliable question response options. Researchers

More information

Basic Concepts in Research and Data Analysis

Basic Concepts in Research and Data Analysis Basic Concepts in Research and Data Analysis Introduction: A Common Language for Researchers...2 Steps to Follow When Conducting Research...3 The Research Question... 3 The Hypothesis... 4 Defining the

More information

Capstone Suggestions for Survey Development for Research Purposes

Capstone Suggestions for Survey Development for Research Purposes Capstone Suggestions for Survey Development for Research Purposes 1. Begin by listing the questions you would like to answer with the survey. These questions will be relatively broad and should be based

More information

Good Practices in Survey Design Step-by-Step

Good Practices in Survey Design Step-by-Step From: Measuring Regulatory Performance A Practitioner's Guide to Perception Surveys Access the complete publication at: http://dx.doi.org/10.1787/9789264167179-en Good Practices in Survey Design Step-by-Step

More information

Northumberland Knowledge

Northumberland Knowledge Northumberland Knowledge Know Guide How to Analyse Data - November 2012 - This page has been left blank 2 About this guide The Know Guides are a suite of documents that provide useful information about

More information

Designing & Conducting Survey Research

Designing & Conducting Survey Research Designing & Conducting Survey Research Santa Monica College Fall 2011 Presented by: Hannah Alford, Director Office of Institutional Research 1 Workshop Overview Part I: Overview of Survey Method Paper/Pencil

More information

There are three kinds of people in the world those who are good at math and those who are not. PSY 511: Advanced Statistics for Psychological and Behavioral Research 1 Positive Views The record of a month

More information

Factsheet: Market research

Factsheet: Market research Factsheet: Market research A close understanding of the local childcare market and your customers needs is essential in order for your childcare business to succeed. Performing market research on potential

More information

The Savvy Survey #13: Online Surveys 1

The Savvy Survey #13: Online Surveys 1 AEC407 1 Glenn D. Israel and Jessica L. Gouldthorpe 2 As part of the Savvy Survey Series, this publication provides Extension faculty with an overview of the process that uses email to invite participants

More information

Making Surveys Work for You: Some Important Design Considerations

Making Surveys Work for You: Some Important Design Considerations Making Surveys Work for You: Some Important Design Considerations By Don A. Dillman* For Net-Conference on Survey Methods and Measurement June 15,11 *Regents Professor in the Department of Sociology and

More information

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives

Marketing Research Core Body Knowledge (MRCBOK ) Learning Objectives Fulfilling the core market research educational needs of individuals and companies worldwide Presented through a unique partnership between How to Contact Us: Phone: +1-706-542-3537 or 1-800-811-6640 (USA

More information

Introduction to Statistics for the Life Sciences

Introduction to Statistics for the Life Sciences Introduction to Statistics for the Life Sciences Fall 2014 Volunteer Definition A biased sample systematically overestimates or underestimates a characteristic of the population Paid subjects for drug

More information

BACKGROUND ON THE SURVEY PROCESS

BACKGROUND ON THE SURVEY PROCESS Chapter Two BACKGROUND ON THE SURVEY PROCESS In this chapter, we present an overview of the various aspects of the research survey process. 1 We emphasize that surveying should first be thought of as a

More information

DATA IN A RESEARCH. Tran Thi Ut, FEC/HSU October 10, 2013

DATA IN A RESEARCH. Tran Thi Ut, FEC/HSU October 10, 2013 DATA IN A RESEARCH Tran Thi Ut, FEC/HSU October 10, 2013 IMPORTANCE OF DATA In research, it needs data for the stages: Research design Sampling design Data gathering and /or field work techniques Data

More information

Information About Survey Length and Question Types

Information About Survey Length and Question Types Introduction Please note that for the purposes of this survey an information literate individual is some one who is able to recognize when information is needed and is capable of effectively locating,

More information

James E. Bartlett, II is Assistant Professor, Department of Business Education and Office Administration, Ball State University, Muncie, Indiana.

James E. Bartlett, II is Assistant Professor, Department of Business Education and Office Administration, Ball State University, Muncie, Indiana. Organizational Research: Determining Appropriate Sample Size in Survey Research James E. Bartlett, II Joe W. Kotrlik Chadwick C. Higgins The determination of sample size is a common task for many organizational

More information

MARKETING RESEARCH WORKBOOK

MARKETING RESEARCH WORKBOOK MARKETING RESEARCH WORKBOOK Version 1.1 ª Copyright Custom Decision Support, Inc. (2001) Eugene B. Lieb Customer Decision Support, Inc. P.O. Box 998 Chadds Ford, PA 19317 (610) 793-3520 genelieb@lieb.com

More information

Section 2: Ten Tools for Applying Sociology

Section 2: Ten Tools for Applying Sociology Section 2: Ten Tools for Applying Sociology CHAPTER 2.6: DATA COLLECTION METHODS QUICK START: In this chapter, you will learn The basics of data collection methods. To know when to use quantitative and/or

More information

Respond to all four questions, following the Degree Program Assessment Plan Form Instructions. Attach additional pages as needed.

Respond to all four questions, following the Degree Program Assessment Plan Form Instructions. Attach additional pages as needed. Graduate Degree Program Assessment Plan Cover Sheet (rev. 07): UNIVERSITY OF ARKANSAS AT LITTLE ROCK Plan No. Degree Program: MA Mass Communication Department and College: School of Mass Communication,

More information

Paul Harris, PhD. Planning, Collecting and Managing Data For Clinical And Translational Research

Paul Harris, PhD. Planning, Collecting and Managing Data For Clinical And Translational Research Planning, Collecting and Managing Data For Clinical And Translational Research Paul Harris, PhD Associate Professor Department of Biomedical Informatics Vanderbilt University Agenda Data Planning for Clinical

More information

Manual on Training Needs Assessment

Manual on Training Needs Assessment Project on Improvement of Local Administration in Cambodia What is Training Needs Assessment? Five Steps of Training Needs Assessment Step 1: Identify Problem and Needs Step 2: Determine Design of Needs

More information

Conducting Aging Well Resident Surveys

Conducting Aging Well Resident Surveys Conducting Aging Well Resident Surveys a g i n g we l l in c o m m u n i t i e s: a t o o l k i t f o r p l a n n i n g, e n g age m e n t & a c t i o n A survey provides detailed quantitative information

More information

Market research. Chapter 4 Market research. The objectives of market research

Market research. Chapter 4 Market research. The objectives of market research Market research Businesses regard having an understanding of the market place as a major priority. This is because of the following factors: the expense of launching new products; the importance of maintaining

More information

TOOLS AND METHODS FOR MEASURING PUBLIC AWARENESS

TOOLS AND METHODS FOR MEASURING PUBLIC AWARENESS TOOLS AND METHODS FOR MEASURING PUBLIC AWARENESS Project: South-East European Digital Television Acronym: SEE Digi.TV Version A-1.0; Date: 04.04.2012 Page: 1/47 DOCUMENT HISTORY Version Status Date Author

More information

Intro to GIS Winter 2011. Data Visualization Part I

Intro to GIS Winter 2011. Data Visualization Part I Intro to GIS Winter 2011 Data Visualization Part I Cartographer Code of Ethics Always have a straightforward agenda and have a defining purpose or goal for each map Always strive to know your audience

More information

Conducting a Tenant Satisfaction Survey Key Issues

Conducting a Tenant Satisfaction Survey Key Issues Conducting a Tenant Satisfaction Survey Key Issues Produced by PPCR Associates 2 Introduction I have set out below the major key issues which we would normally include in a tender for conducting a resident

More information

Determine whether the data are qualitative or quantitative. 8) the colors of automobiles on a used car lot Answer: qualitative

Determine whether the data are qualitative or quantitative. 8) the colors of automobiles on a used car lot Answer: qualitative Name Score: Math 227 Review Exam 1 Chapter 2& Fall 2011 ********************************************************************************************************************** SHORT ANSWER. Show work on

More information

Consumer s Guide to Research on STEM Education. March 2012. Iris R. Weiss

Consumer s Guide to Research on STEM Education. March 2012. Iris R. Weiss Consumer s Guide to Research on STEM Education March 2012 Iris R. Weiss Consumer s Guide to Research on STEM Education was prepared with support from the National Science Foundation under grant number

More information

Simple Predictive Analytics Curtis Seare

Simple Predictive Analytics Curtis Seare Using Excel to Solve Business Problems: Simple Predictive Analytics Curtis Seare Copyright: Vault Analytics July 2010 Contents Section I: Background Information Why use Predictive Analytics? How to use

More information

Market Research: Friend or Foe to Christian Charities?

Market Research: Friend or Foe to Christian Charities? Market Research: Friend or Foe to Christian Charities? by Redina Kolaneci Senior Fundraising & Stewardship Consultant McConkey Johnston international UK If you are ignorant of yourself, in addition to

More information

Best Practices in Data Visualizations. Vihao Pham January 29, 2014

Best Practices in Data Visualizations. Vihao Pham January 29, 2014 Best Practices in Data Visualizations Vihao Pham January 29, 2014 Agenda Best Practices in Data Visualizations Why We Visualize Understanding Data Visualizations Enhancing Visualizations Visualization

More information

Best Practices in Data Visualizations. Vihao Pham 2014

Best Practices in Data Visualizations. Vihao Pham 2014 Best Practices in Data Visualizations Vihao Pham 2014 Agenda Best Practices in Data Visualizations Why We Visualize Understanding Data Visualizations Enhancing Visualizations Visualization Considerations

More information

Google Analytics and the Modernization of Marketing

Google Analytics and the Modernization of Marketing PEGASUS CREATIVE Google Analytics and the Modernization of Marketing Web Content Development Internship Chandler Anderson 5/12/2013 One of the most important functions of any kind of business is gathering

More information

Appendix 1: Methodology Interviews. A1 Sampling Strategy

Appendix 1: Methodology Interviews. A1 Sampling Strategy Appendix 1: Methodology Interviews A1 Sampling Strategy Creswell noted that in qualitative research, the intent is not to generalize to a population, but to develop an in-depth exploration of a central

More information

5. Survey Samples, Sample Populations and Response Rates

5. Survey Samples, Sample Populations and Response Rates An Analysis of Mode Effects in Three Mixed-Mode Surveys of Veteran and Military Populations Boris Rachev ICF International, 9300 Lee Highway, Fairfax, VA 22031 Abstract: Studies on mixed-mode survey designs

More information

IBM SPSS Direct Marketing 22

IBM SPSS Direct Marketing 22 IBM SPSS Direct Marketing 22 Note Before using this information and the product it supports, read the information in Notices on page 25. Product Information This edition applies to version 22, release

More information

2. Choose the location and format for focus groups or interviews

2. Choose the location and format for focus groups or interviews Steps for Conducting Focus Groups or Individual In-depth Interviews 1. Plan the study Plan your study with the input of your strategy team, stakeholders, and research experts so that collaboratively you

More information

THE OREGON STATE REHABILITATION COUNCIL AND OFFICE OF VOCATIONAL REHABILITATION SERVICES 2008 CLIENT CONSUMER SATISFACTION SURVEY

THE OREGON STATE REHABILITATION COUNCIL AND OFFICE OF VOCATIONAL REHABILITATION SERVICES 2008 CLIENT CONSUMER SATISFACTION SURVEY THE OREGON STATE REHABILITATION COUNCIL AND OFFICE OF VOCATIONAL REHABILITATION SERVICES 2008 CLIENT CONSUMER SATISFACTION SURVEY Assisting Oregonians with Disabilities Achieve And Maintain Employment

More information

Writing better questionnaires getting better data

Writing better questionnaires getting better data Writing better questionnaires getting better data Writing better questionnaires: getting better data January 2014 Charities Evaluation Services Acknowledgements Written by Jean Ellis for Charities Evaluation

More information

Strategy Overview. S.M.A.R.T. Specific, Measurable, Achievable, Results Oriented, Timely

Strategy Overview. S.M.A.R.T. Specific, Measurable, Achievable, Results Oriented, Timely GOAL PLANNER 2 Strategy Overview S.M.A.R.T. Specific, Measurable, Achievable, Results Oriented, Timely 6 Strings of Marketing: Visibility Lead Generation Conversions Average $ Sale Average # Sales / Customer

More information

New Findings on Questionnaire Design from 2012 AAPOR Conference Papers. 2012 AAPOR conference recap mini-conference 6/22/2012 San Francisco, CA Su Li

New Findings on Questionnaire Design from 2012 AAPOR Conference Papers. 2012 AAPOR conference recap mini-conference 6/22/2012 San Francisco, CA Su Li New Findings on Questionnaire Design from 2012 AAPOR Conference Papers 2012 AAPOR conference recap mini-conference 6/22/2012 San Francisco, CA Su Li Disclaimer Everything presented in this talk is adapted

More information

Data Collection and Sampling OPRE 6301

Data Collection and Sampling OPRE 6301 Data Collection and Sampling OPRE 6301 Recall... Statistics is a tool for converting data into information: Statistics Data Information But where then does data come from? How is it gathered? How do we

More information

Chapter 4 Constructing Questionnaires and Indexes 71. Chapter 4 Constructing Questionnaires and Indexes

Chapter 4 Constructing Questionnaires and Indexes 71. Chapter 4 Constructing Questionnaires and Indexes Chapter 4 Constructing Questionnaires and Indexes 71 Chapter 4 Constructing Questionnaires and Indexes When one is interested in conducting primary research, he or she most often needs to either purchase

More information

Question Bank FACTSHEET 2. Methods of Data Collection in Social Surveys

Question Bank FACTSHEET 2. Methods of Data Collection in Social Surveys Question Bank FACTSHEET 2 Methods of Data Collection in Social Surveys Keywords: Data, Structured Interview, Self completion, Web collection. Introduction This fact sheet will discuss the main methods

More information

CITY OF MILWAUKEE POLICE SATISFACTION SURVEY

CITY OF MILWAUKEE POLICE SATISFACTION SURVEY RESEARCH BRIEF Joseph Cera, PhD Survey Center Director UW-Milwaukee Atiera Coleman, MA Project Assistant UW-Milwaukee CITY OF MILWAUKEE POLICE SATISFACTION SURVEY At the request of and in cooperation with

More information

THREE BOOKLET. COLLECTING AND ANALYZING EVALUATION DATA 2nd edition. Outreach Evaluation Resource Center

THREE BOOKLET. COLLECTING AND ANALYZING EVALUATION DATA 2nd edition. Outreach Evaluation Resource Center Planning and Evaluating Health Information Outreach Projects THREE BOOKLET COLLECTING AND ANALYZING EVALUATION DATA 2nd edition Outreach Evaluation Resource Center 2013 National Network of Libraries of

More information

Surveys: Why, When and How. Gavin Henning Pamela Bagley

Surveys: Why, When and How. Gavin Henning Pamela Bagley Surveys: Why, When and How Gavin Henning Pamela Bagley Session Outcomes Articulate appropriate uses for surveys Write effective survey questions Construct an effective questionnaire Choose an appropriate

More information

Antoine J. Alston North Carolina A&T State University. W. Wade Miller Iowa State University. Introduction/ Rationale

Antoine J. Alston North Carolina A&T State University. W. Wade Miller Iowa State University. Introduction/ Rationale Analyzing the Barriers and Benefits Toward Instructional Technology Instruction in North Carolina and Virginia Secondary Agricultural Education Curricula Antoine J. Alston North Carolina A&T State University

More information

Quantitative survey packet

Quantitative survey packet SPSS in a nutshell Quantitative survey packet What is a scale? A scale is something that measures a particular construct or a particular idea o Example: There are scale that try to assess well-being, motivation

More information

Cloud Productivity Is Key to Success: A Survey of Young Businesses

Cloud Productivity Is Key to Success: A Survey of Young Businesses Cloud Productivity Is Key to Success: A Survey of Young Businesses Cloud Productivity Is Key to Success: A Survey of Young Businesses Executive Summary In every market in every region, launching and building

More information

Essentials of Marketing Research

Essentials of Marketing Research Essentials of Marketing Second Edition Joseph F. Hair, Jr. Kennesaw State University Mary F. Wolfinbarger California State University-Long Beach David J. Ortinau University of South Florida Robert P. Bush

More information

SEAN J. GALLAGHER. A paper submitted to the faculty of. The University of North Carolina at Chapel Hill

SEAN J. GALLAGHER. A paper submitted to the faculty of. The University of North Carolina at Chapel Hill Linking Individual Performance Appraisals with Departmental Outcomes: Does the Type of Individual Performance Appraisal System Influence Departmental Performance? By SEAN J. GALLAGHER A paper submitted

More information

Benefits of Electronic Audience Response Systems on Student Participation, Learning, and Emotion

Benefits of Electronic Audience Response Systems on Student Participation, Learning, and Emotion Benefits of Electronic Audience Response Systems on Student Participation, Learning, and Emotion Jeffrey R. Stowell and Jason M. Nelson Eastern Illinois University We compared an electronic audience response

More information

Mixed-Mode Methods for Conducting Survey Research

Mixed-Mode Methods for Conducting Survey Research Mixed-Mode Methods for Conducting Survey Research Herbert M. Baum, Ph.D.; Anna Chandonnet M.A.; Jack Fentress M.S., M.B.A.; and Colleen Rasinowich, B.A. www.datarecognitioncorp.com In November 2010 the

More information

Surveys. Methods for Evaluating Advising Program/U.C. Berkeley 1

Surveys. Methods for Evaluating Advising Program/U.C. Berkeley 1 Surveys Methods for Evaluating Advising Program/U.C. Berkeley 1 The purpose of surveys in an Advising Program evaluation is to gain insight into respondents attitudes and perceptions about advising services

More information

CHAPTER. Market Research. Research? Section What Is Your Competitive Advantage?

CHAPTER. Market Research. Research? Section What Is Your Competitive Advantage? CHAPTER Market Research Section Research? What Is Market Section What Is Your Competitive Advantage? SECTION What Is Market Research? OBJECTIVES Explain why market research is important Consider important

More information

Plotting Likert and Other Rating Scales

Plotting Likert and Other Rating Scales Plotting Likert and Other Rating Scales Naomi B. Robbins 1, Richard M. Heiberger 2 1 NBR, 11 Christine Court, Wayne, NJ 07470-6523 2 Temple University, 332 Speakman Hall (006-12), Philadelphia, PA 19122-6083

More information

The Science and Art of Market Segmentation Using PROC FASTCLUS Mark E. Thompson, Forefront Economics Inc, Beaverton, Oregon

The Science and Art of Market Segmentation Using PROC FASTCLUS Mark E. Thompson, Forefront Economics Inc, Beaverton, Oregon The Science and Art of Market Segmentation Using PROC FASTCLUS Mark E. Thompson, Forefront Economics Inc, Beaverton, Oregon ABSTRACT Effective business development strategies often begin with market segmentation,

More information

Quality Assurance for doctoral education

Quality Assurance for doctoral education Quality Assurance for doctoral education modified from A.Mleczko, modified Jadwiga Mirecka Bologna Expert Warsaw 17-18 June 2010 1 A demand for Quality Assurance in Doctoral (PhD) studies derives from

More information

Math and Science Bridge Program. Session 1 WHAT IS STATISTICS? 2/22/13. Research Paperwork. Agenda. Professional Development Website

Math and Science Bridge Program. Session 1 WHAT IS STATISTICS? 2/22/13. Research Paperwork. Agenda. Professional Development Website Math and Science Bridge Program Year 1: Statistics and Probability Dr. Tamara Pearson Assistant Professor of Mathematics Research Paperwork Informed Consent Pre-Survey After you complete the survey please

More information

An International Comparison of the Career of Social Work by Students in Social Work

An International Comparison of the Career of Social Work by Students in Social Work Acta Medicina et Sociologica Vol 5., 2014 5 An International Comparison of the Career of Social Work by Students in Social Work Gergely Fábián*, Thomas R. Lawson**, Mihály Fónai***, János Kiss*, Eric R.

More information

Unit 3. Answering the Research Question: Quantitative Designs

Unit 3. Answering the Research Question: Quantitative Designs brockopp06 7/30/02 11:39 AM Page 191 Unit 3 Answering the Research Question: Quantitative Designs brockopp06 7/30/02 11:39 AM Page 192 192 Intentional Blank brockopp06 7/30/02 11:39 AM Page 193 6 Measurement

More information

Autonomy etalk White Paper. Optimizing Customer Feedback: Executing a Valuable Post-Call Survey

Autonomy etalk White Paper. Optimizing Customer Feedback: Executing a Valuable Post-Call Survey Autonomy etalk White Paper Optimizing Customer Feedback: Executing a Valuable Post-Call Survey Proprietary and Confidential to Autonomy etalk. Optimizing Customer Feedback: Executing a Valuable Post-Call

More information

Question Bank FACTSHEET 5

Question Bank FACTSHEET 5 Question Bank FACTSHEET 5 Survey Interviewing Introduction: The Survey Interview This factsheet will describe the processes involved in the Survey interview. It will focus only on the face to face interview

More information

Probability of Selecting Response Mode Between Paper- and Web-Based Options: Logistic Model Based on Institutional Demographics

Probability of Selecting Response Mode Between Paper- and Web-Based Options: Logistic Model Based on Institutional Demographics Probability of Selecting Response Mode Between Paper- and Web-Based Options: Logistic Model Based on Institutional Demographics Marina Murray Graduate Management Admission Council, 11921 Freedom Drive,

More information

ColoradoFIRST Impact on Participant Future Education and Career Choices

ColoradoFIRST Impact on Participant Future Education and Career Choices 1475 LAWRENCE STREET SUITE LC-2401 DENVER, CO 80202 Executive Summary ColoradoFIRST Impact on Participant Future Education and Career Choices ColoradoFIRST and FIRST Robotics Competition (FRC ) have certainly

More information

Response Rate and Measurement Differences in Mixed Mode Surveys. Using Mail, Telephone, Interactive Voice Response and the Internet 1

Response Rate and Measurement Differences in Mixed Mode Surveys. Using Mail, Telephone, Interactive Voice Response and the Internet 1 March 21, 2008 Response Rate and Measurement Differences in Mixed Mode Surveys Using Mail, Telephone, Interactive Voice Response and the Internet 1 by Don A. Dillman, Glenn Phelps, Robert Tortora, Karen

More information

Visual Design, Order Effects, and Respondent Characteristics in a Self-Administered Survey

Visual Design, Order Effects, and Respondent Characteristics in a Self-Administered Survey Survey Research Methods (2007) Vol.1, No.3, pp. 121-138 ISSN 1864-3361 http://www.surveymethods.org c European Survey Research Association Visual Design, Order Effects, and Respondent Characteristics in

More information

EDUCATION AND LAW ENFORCEMENT 1. Education and Law Enforcement: Final Version of Research Proposal. Ralphie Manzano

EDUCATION AND LAW ENFORCEMENT 1. Education and Law Enforcement: Final Version of Research Proposal. Ralphie Manzano EDUCATION AND LAW ENFORCEMENT 1 Education and Law Enforcement: Final Version of Research Proposal Ralphie Manzano University of South Florida, Sarasota-Manatee EDUCATION AND LAW ENFORCEMENT 2 Abstract

More information

Chapter 4: Methodology of the Conducted Research. and state the research purpose, hypothesis and main research objectives. 4.1.

Chapter 4: Methodology of the Conducted Research. and state the research purpose, hypothesis and main research objectives. 4.1. Chapter 4: Methodology of the Conducted Research This chapter has the objective of describing the methodology of the conducted research in detail so that each step of the process can be understood and

More information

Michigan Department of Treasury Tax Compliance Bureau Audit Division. Audit Sampling Manual

Michigan Department of Treasury Tax Compliance Bureau Audit Division. Audit Sampling Manual Audit Division Audit Sampling Manual Disclosure This manual is not intended as a statement of law, Department policy, or of the Treasurer s official position. The information contained in this manual has

More information

Published entries to the three competitions on Tricky Stats in The Psychologist

Published entries to the three competitions on Tricky Stats in The Psychologist Published entries to the three competitions on Tricky Stats in The Psychologist Author s manuscript Published entry (within announced maximum of 250 words) to competition on Tricky Stats (no. 1) on confounds,

More information

Research Overview: Telephone versus Online Research Advantages and Pitfalls

Research Overview: Telephone versus Online Research Advantages and Pitfalls . 26030 Highway 74, Suite A, P.O. Box 1094 Kittredge, CO 80457 P: 303.679.6300 F: 303.679.6680 info@praxigroup.net www.praxigroup.net Research Overview: Telephone versus Online Research Advantages and

More information

DATA ANALYSIS, INTERPRETATION AND PRESENTATION

DATA ANALYSIS, INTERPRETATION AND PRESENTATION DATA ANALYSIS, INTERPRETATION AND PRESENTATION OVERVIEW Qualitative and quantitative Simple quantitative analysis Simple qualitative analysis Tools to support data analysis Theoretical frameworks: grounded

More information

IVR: Interactive Voice Response

IVR: Interactive Voice Response Web material accompanying The International Handbook of Survey Methodology Chapter 15 IVR: Interactive Voice Response Darby Miller Steiger Beverly Conroy The Gallup Organization ADDITIONAL WEB MATERIAL

More information

survey Design & Delivery S o c i a l S c i e n c e T o o l s f o r C o a s t a l P r o g r a m s I n t r o d u c t i o n t o

survey Design & Delivery S o c i a l S c i e n c e T o o l s f o r C o a s t a l P r o g r a m s I n t r o d u c t i o n t o S o c i a l S c i e n c e T o o l s f o r C o a s t a l P r o g r a m s survey I n t r o d u c t i o n t o Design & Delivery Social Science Tools for Coastal Programs Introduction to Survey Design and

More information

Test procedures and protocols standardisation and international harmonisation

Test procedures and protocols standardisation and international harmonisation Session 7: Measuring networked standby power consumption* Test procedures and protocols standardisation and international harmonisation Bruce Nordman, LBNL, US Lloyd Harrington, EES, AU *Actually, network

More information

The Five Rules for Reliable Marketing Research

The Five Rules for Reliable Marketing Research URBAN WALLACE ASSOCIATES 35 Bedford St., Suite 8, Lexington, MA 02420 ph 781 862 0033 fax 781 862 1292 web www.uwa.com The Five Rules for Reliable Marketing Research Marketing Research Produces New Knowledge

More information

Best Practices for REDCap Database Creation

Best Practices for REDCap Database Creation A joint project from the UC Davis CTSC and Information Technology Best Practices for REDCap Database Creation This document borrowed and revised from the University of Colorado, Denver provides general

More information

9. Sampling Distributions

9. Sampling Distributions 9. Sampling Distributions Prerequisites none A. Introduction B. Sampling Distribution of the Mean C. Sampling Distribution of Difference Between Means D. Sampling Distribution of Pearson's r E. Sampling

More information

IBM SPSS Statistics for Beginners for Windows

IBM SPSS Statistics for Beginners for Windows ISS, NEWCASTLE UNIVERSITY IBM SPSS Statistics for Beginners for Windows A Training Manual for Beginners Dr. S. T. Kometa A Training Manual for Beginners Contents 1 Aims and Objectives... 3 1.1 Learning

More information

Use of Electronic Resources at Krishnasamy College of Engineering & Technology Library, Cuddalore

Use of Electronic Resources at Krishnasamy College of Engineering & Technology Library, Cuddalore http://unllib.unl.edu/lpp/ Library Philosophy and Practice 2012 ISSN 1522-0222 Use of Electronic Resources at Krishnasamy College of Engineering & Technology Library, Cuddalore Dr. S. Dhanavandan Librarian

More information

Alumni Perspectives: Evaluation of the Nonprofit Leadership and Management Masters Program

Alumni Perspectives: Evaluation of the Nonprofit Leadership and Management Masters Program Alumni Perspectives: Evaluation of the Nonprofit Leadership and Management Masters Program June 2008 Linda Siefert, Ed.D. Director of Assessment Heather Carpenter, MMNA Viterbi Family Doctoral Fellow and

More information

Survey Design. Assessment Spotlight Hosted by the Student Affairs Assessment Council University of North Carolina at Chapel Hill

Survey Design. Assessment Spotlight Hosted by the Student Affairs Assessment Council University of North Carolina at Chapel Hill Survey Design Assessment Spotlight Hosted by the Student Affairs Assessment Council University of North Carolina at Chapel Hill Surveys are Self-report data collection instruments (Gideon, 2012) Tools

More information

1.1 What is Statistics?

1.1 What is Statistics? 1.1 What is Statistics? (Page 1 of 15) 1.1 What is Statistics? Definition of Statistics Statistics is the study of how to collect, organize, analyze, and interpret numerical information from data. Descriptive

More information

Using Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP)

Using Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP) Using Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP) By Katie Wright-Revolledo, INTRAC Introduction Qualitative methods can provide insight

More information

OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH

OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH OPEN UP YOUR RESEARCH DESIGN CHOOSE THE DATA COLLECTION APPROACH THAT BEST FITS THE RESEARCH WWW.CFMC.COM Copyright 2015 CONTENTS IDEA IN ACTION 2 DATA COLLECTION MODES 2 DEMOGRAPHICS MATTER 3 COSTS MATTER

More information

Providence College Local Faculty Survey: Analytical Memo of Findings

Providence College Local Faculty Survey: Analytical Memo of Findings Providence College Local Faculty Survey: Analytical Memo of Findings October 1, 2013 Overview The following memorandum provides an analytical narrative of the results of the 2013 Ithaka S+R Local Faculty

More information

Types of Research. Strength of Research. Data Collection 9/13/2011

Types of Research. Strength of Research. Data Collection 9/13/2011 Market Research and Information Technology Principles of Marketing Chapter 5 1 Importance of Marketing Research Determine current consumer needs Shifts in attitudes and purchasing patterns Changes in the

More information

Measurement in ediscovery

Measurement in ediscovery Measurement in ediscovery A Technical White Paper Herbert Roitblat, Ph.D. CTO, Chief Scientist Measurement in ediscovery From an information-science perspective, ediscovery is about separating the responsive

More information

EcoPinion Consumer Survey No. 19 The Conflicted Consumer Landscape in the Utility Sector. May 2014

EcoPinion Consumer Survey No. 19 The Conflicted Consumer Landscape in the Utility Sector. May 2014 EcoPinion Consumer Survey No. 19 The Conflicted Consumer Landscape in the Utility Sector May 2014 Overview 2 Utilities are increasingly seeing customers as core to their business model on a going forward

More information

Topic #1: Introduction to measurement and statistics

Topic #1: Introduction to measurement and statistics Topic #1: Introduction to measurement and statistics "Statistics can be fun or at least they don't need to be feared." Many folks have trouble believing this premise. Often, individuals walk into their

More information

Council of Ambulance Authorities

Council of Ambulance Authorities Council of Ambulance Authorities Patient Satisfaction Survey 2013 Prepared for: Mojca Bizjak-Mikic Manager, Data & Research The Council of Ambulance Authorities Prepared by: Natasha Kapulski Research Associate

More information

Usability Evaluation with Users CMPT 281

Usability Evaluation with Users CMPT 281 Usability Evaluation with Users CMPT 281 Outline Usability review Observational methods Interview methods Questionnaire methods Usability ISO 9241-11: The extent to which a product can be used by specified

More information

Business Statistics: Intorduction

Business Statistics: Intorduction Business Statistics: Intorduction Donglei Du (ddu@unb.edu) Faculty of Business Administration, University of New Brunswick, NB Canada Fredericton E3B 9Y2 September 23, 2015 Donglei Du (UNB) AlgoTrading

More information

How to Write a Marketing Plan: Identifying Your Market

How to Write a Marketing Plan: Identifying Your Market How to Write a Marketing Plan: Identifying Your Market (Part 1 of 5) Any good marketing student will tell you that marketing consists of the four functions used to create a sale: The right product to the

More information

Unraveling the Career Conundrum: Faculty Agency in Collaborative Undergraduate Nursing Programs

Unraveling the Career Conundrum: Faculty Agency in Collaborative Undergraduate Nursing Programs Unraveling the Career Conundrum: Faculty Agency in Collaborative Undergraduate Nursing Programs Michele Drummond-Young Funded by NERU Pilot Project Fund 2008 SON, McMaster University Introduction Context

More information

Calculating inter-coder reliability in media content analysis using Krippendorff s Alpha

Calculating inter-coder reliability in media content analysis using Krippendorff s Alpha Calculating inter-coder reliability in media content analysis using Krippendorff s Alpha First version (01/02/2012). Knut De Swert (University of Amsterdam) Please send your comments, suggestions, additions

More information

Data Analysis and Interpretation. Eleanor Howell, MS Manager, Data Dissemination Unit State Center for Health Statistics

Data Analysis and Interpretation. Eleanor Howell, MS Manager, Data Dissemination Unit State Center for Health Statistics Data Analysis and Interpretation Eleanor Howell, MS Manager, Data Dissemination Unit State Center for Health Statistics Why do we need data? To show evidence or support for an idea To track progress over

More information