Usability Evaluation with Users CMPT 281
|
|
- Luke Greene
- 8 years ago
- Views:
Transcription
1 Usability Evaluation with Users CMPT 281
2 Outline Usability review Observational methods Interview methods Questionnaire methods
3 Usability ISO : The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction, in a specified context of use Efficiency Effectiveness Satisfaction Context: user, task, environment
4 Nielsen s model of usability
5 Operationalizing usability How to assess usability criteria? What measures? What thresholds? What is usable enough?
6 Observational Evaluation Methods
7 Observational Methods Simple observation Think-aloud protocols Retrospective think-aloud protocols Co-discovery learning Recording observations
8 Simple Observation User is given a task, and evaluator just watches the user Problem: no insight into the user s decision process or attitude
9 Think-aloud Protocol Subjects are asked to say what they are thinking/doing: What they believe is happening What they are trying to do Why they took an action Gives insight into what the user is thinking
10 Think-aloud Protocol Problems: Awkward/uncomfortable for subject (thinking aloud is not normal!) Thinking about it may alter the way people perform their task Hard to talk when they are concentrating on problem Still the most widely used method in industry
11 Other Problems with Think-aloud
12 Retrospective Think-aloud Problems with Think-aloud: Awkward for subject (thinking aloud not normal!) Thinking about it may alter the way people perform their task Hard to talk when they are concentrating on problem Solution: videotape the experience, perform a retrospective think-aloud Has its own problems Awkwardness of watching themselves on video Awkwardness of reliving mistakes Reflection of the experience rather than in context 12
13 Co-discovery Learning Two people work together on a task Normal conversation between the two users is monitored Removes awkwardness of think-aloud, more natural Provides insights into thinking process of both users 13
14 Field Studies Observe in the field = natural environment Sit and observe Video records Join the culture (ethnography)
15 Field Studies Observe in the field = natural environment Sit and observe Video records Join the culture (ethnography) Requires that the system be fully deployed Highest degree of realism Can be highly specific to the particular setting Can take a long time
16 Recording Observations Paper and pencil primitive but cheap evaluators record events, interpretations, and extraneous observations evaluator seems disengaged problem: writing is slow prepared coding schemes can help; just tick off events Audio recording capture discussion (think aloud, co-discovery) hard to synchronize streams (e.g., interface actions) (expensive) tools exist to help transcription is slow and difficult! tools exist to help
17 Recording Observations Video recording can see what a user is doing (good to use one camera/scan converter for screen + one for subject) can be intrusive (at least initially) analysis can be challenging annotation is time consuming and dull Companies often build usability labs with one-way mirrors, video cams, etc.
18 Analyzing Observation Data Qualitative data: interpreted to tell a story Qualitative data: categorized Quantitative data: presented as values, tables, charts, and graphs often treated with statistical tests How do you know which analysis is appropriate? Depends on what you are using it for
19 Interviews
20 Querying Users with Interviews conversations with a purpose excellent for pursuing specific issues more interactive than observation: address specific issues of interest more flexible than questionnaires: probe more deeply on interesting issues as they arise problems accounts are subjective time consuming (to conduct and to analyze) evaluator can bias the interview prone to rationalization of events/thoughts by user user s reconstruction may be wrong
21 Planning the Interview general what is the purpose of the interview? how many people? (breadth vs. depth) length of interview & number of sessions scheduling interviews (location, times, people) will the interview be recorded? (audio, video; transcription) avoid: asking long questions using compound sentences using jargon asking leading questions and generally be alert to unconscious biases.
22 Interviews three main types: 1. open-ended / unstructured 2. semi-structured 3. structured control & pre-determined questions other categories (can include types above): 4. group 5. retrospective
23 Unstructured Interviews most like a conversation, often go into depth open questions exploratory key is to listen rather than talk: practice silence! pros/cons: + rich data, things interviewer may not have considered - easy to go off the rails - time-consuming & difficult to analyze - impossible to replicate
24 Structured Interviews predetermined questions (like questionnaire, often with a flowchart) closed questions short, clearly worded questions confirmatory pros/cons: + replicable - potentially important detail can be lost better (cheaper) with a questionnaire?
25 Semi-structured Interviews Between structured & unstructured Uses elements of both In usability studies, unstructured and semistructured are the most common
26 Group Interviews (Focus Group) 3 10 people interviewed at one time usually has agenda, but may be structured/unstructured skilled moderator critical! usually recorded pros/cons: + can accommodate diverse and sensitive issues + opinions developed within a social context + good way to locate proto-users : most articulate, imaginative participants can help later w/participatory design - some interviewees may dominate - expensive: usually pay participants + professional moderator - people may not know what they think (or be afraid to express it)!
27 Retrospective Interview post-test interview to clarify events that occurred during system use: record what happened, replay it, and ask about it pros/cons: + excellent for grounding a post-test interview + avoids erroneous reconstruction + users often offer concrete suggestions - requires a second session I didn t see it. Why don t you make it look like a button? Do you know why you never tried that option?
28 Overview of an Exploratory Interview 1. explain purpose of the interview allow time to get acquainted with the interviewee provide understanding and background 2. enumerate activities find out what the user does 3. explain work methods find out how the user does things (skills and knowledge) 4. trace interconnections determine other people and activities that are related 5. identify performance issues explore current problems and impediments to success
29 Things You Uncover during Interviews exceptions lots of things people do are not in the manual many jobs evolve to fit changing circumstances much of this is not documented many times management does not know about this domain knowledge most people know a lot about their jobs, and those they work with terminology, common phrases, specific details audio recording helps capture this video recording helps provide body language written notes can provide context, but not always details
30 Questionnaires (Surveys)
31 Querying Users with Questionnaires closed or open questions get evidence of wide general opinion or experiences after an experiment pros/cons: + preparation expensive, but administration cheap can reach a wide subject group (e.g. mail or ) + does not require presence of evaluator + results can be quantified - risk: low response rate and/or low quality responses
32 Questionnaires: Designing Questions establish the purpose of the questionnaire: what information is sought? how would you analyze the results? what would you do with your analysis? determine the audience you want to reach typical: random sample of between 50 and 1000 users of the product -- why a random sample? test everything before sending it out: test the wording test the timing test the validity test the analysis
33 Administering Questionnaires in-person administration take home (conventional) requires time to administer, but highest completion rate often subjects don t complete / return the questionnaire permits subjects to answer on their own time responses may tend to be more free-form attachments may be a problem response rates depend on trust in source web-based forms general issues standardize formats and responses scripts to ensure correct / complete payment or incentives anonymity self-selection
34 Styles of Questions: Open-ended asks for opinions good for general subjective information but difficult to analyze rigorously E.g., can you suggest any improvements to the interface?
35 Styles of Questions: Closed restricts responses by supplying the choices for answers can be easily analyzed but can still be hard to interpret, if questions / responses not well designed! alternative answers should be very specific Do you use computers at work: O often O sometimes O rarely In your typical work day, do you use computers: O over 4 hrs a day O between 2 and 4 hrs daily O between 1 and 2 hrs daily O less than 1 hr a day
36 Styles of Questions: Scalar - Likert Scale measure opinions, attitudes, and beliefs ask user to judge a specific statement on a numeric scale scale usually corresponds to agreement or disagreement with a statement Characters on the computer screen are hard to read: strongly agree strongly disagree
37 Styles of Questions: Scalar - Semantic Differential Scale explore a range of bipolar attitudes about a particular item each pair of attitudes is represented as a pair of adjectives WebCT is: clear confusing attractive ugly
38 Styles of Questions: Multi-Choice respondent offered a choice of explicit responses How do you most often get help with the system? (tick one) O on-line manual O paper manual O ask a colleague Which types of software have you used? (tick all that apply) O word processor O data base O spreadsheet O compiler
39 Styles of Questions: Ranked respondent places an ordering on items in a list useful to indicate a user s preferences forced choice Rank the usefulness of these methods of issuing a command (1 most useful, 2 next most useful..., 0 if not used) 2 command line 1 menu selection 3 control key accelerator
40 Styles of Questions: Combining Open-ended & Closed Questions gets specific response, but allows room for user s opinion It is easy to recover from mistakes: disagree agree comment: the undo facility is great!
41 Herman Miller Aeron Chair Comfort Likert 1-10 (want 7.5) Got 4.5 Eventually inched up to 8 before release Aesthetics Likert 1-10 Got 2-3 (never above 6!) Usually a relationship between these two but it didn t happen! Focus Group Check on pricing Architects and designers liked Facility managers and ergonomicists hated! Entire design was *actually* user friendly Where is this chair at today?
42 Considerations Style of Questions Open Ended Closed Choose one Choose all that apply Ratings (scale) Rankings Ease of Analysis Poor Depends Easy Somewhat easy* Easy Somewhat easy *Note: Can t really make a pie chart if the responses don t add up to 100%
43 Be Considerate of Your Respondents not just because it s nice, but it works better. questionnaire length (short is good): think in terms of reasonable completion times do not ask questions whose answers you will not use! privacy invasions: be careful how / what you ask motivation why should the respondent bother? usually need to offer something in return but be careful about introducing bias.
44 Deployment Issues Online/ html tools U of S survey tool Survey Monkey Choice impacts ease of analysis responses? Go directly to database
45 Presenting questionnaire results Choose one
46 Presenting questionnaire results Choose one Choose all that apply
47 Summary: Questionnaires 1. Establish purpose 2. Determine audience 3. Variety of administration methods (for different audiences) 4. Design questions: many kinds, depend on what you want to learn 5. Be considerate of your respondents 6. Motivate your respondents (without biasing them).
User Interface Design
User Interface Design Winter term 2005/2006 Thursdays, 14-16 c.t., Raum 228 Prof. Dr. Antonio Krüger Institut für Geoinformatik Universität Münster 20. Februar 06 IfGi Universität Münster User Interface
More informationQuestionnaires. Unit 7
Questionnaires Unit 7 Learning outcomes Understand when/how to use questionnaires in interaction design Surveys Satisfaction questionnaires (post use) Learn how to prepare them Different types of scales
More informationDATA ANALYSIS, INTERPRETATION AND PRESENTATION
DATA ANALYSIS, INTERPRETATION AND PRESENTATION OVERVIEW Qualitative and quantitative Simple quantitative analysis Simple qualitative analysis Tools to support data analysis Theoretical frameworks: grounded
More informationIntroduction to Usability Testing
Introduction to Usability Testing Abbas Moallem COEN 296-Human Computer Interaction And Usability Eng Overview What is usability testing? Determining the appropriate testing Selecting usability test type
More informationUsability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries
Usability Test Plan Docutek ERes v. 5.0.05 University of Texas Libraries Prepared for: INF 385p Usability (Dr. Randolph Bias) Prepared by: Laura Bright, Rachael Gilg and Heather Nodler Date: March 24,
More informationData Collection Strategies 1. Bonus Table 1 Strengths and Weaknesses of Tests
Data Collection Strategies 1 Bonus Table 1 Strengths and Weaknesses of Tests Strengths of tests (especially standardized tests) Can provide measures of many characteristics of people. Often standardized
More informationQualitative data acquisition methods (e.g. Interviews and observations) -.
Qualitative data acquisition methods (e.g. Interviews and observations) -. Qualitative data acquisition methods (e.g. Interviews and observations) ( version 0.9, 1/4/05 ) Code: data-quali Daniel K. Schneider,
More informationCapstone Suggestions for Survey Development for Research Purposes
Capstone Suggestions for Survey Development for Research Purposes 1. Begin by listing the questions you would like to answer with the survey. These questions will be relatively broad and should be based
More informationTypes of Research. Strength of Research. Data Collection 9/13/2011
Market Research and Information Technology Principles of Marketing Chapter 5 1 Importance of Marketing Research Determine current consumer needs Shifts in attitudes and purchasing patterns Changes in the
More information2. Choose the location and format for focus groups or interviews
Steps for Conducting Focus Groups or Individual In-depth Interviews 1. Plan the study Plan your study with the input of your strategy team, stakeholders, and research experts so that collaboratively you
More informationUsability Testing. Credit: Slides adapted from H. Kiewe
Usability Testing Credit: Slides adapted from H. Kiewe Materials for this class Readings: Nielsen, Why You Only Need to Test with 5 Users Dickelman, Usability Testing -- Fear and Loathing on the Keyboard
More informationDesigning & Conducting Survey Research
Designing & Conducting Survey Research Santa Monica College Fall 2011 Presented by: Hannah Alford, Director Office of Institutional Research 1 Workshop Overview Part I: Overview of Survey Method Paper/Pencil
More informationUser Testing Report An Analysis of METRO.org
User Testing Report An Analysis of METRO.org Hannah Goldstein, Michelle Magnotta, & Julie Robbins 1 Table of Contents Executive Summary 3 Introduction 4 Methodology 4 Findings & Recommendations 6 Conclusion
More informationEVALUATION METHODS TIP SHEET
EVALUATION METHODS TIP SHEET QUANTITATIVE METHODS: Quantitative data collection methods consist of counts or frequencies, rates or percentages, or other statistics that document the actual existence or
More informationUsability Test Results CyberLynx Annotator for Lotus 1-2-3. Executive Summary
Usability Test Results CyberLynx Annotator for Lotus 1-2-3 Executive Summary John Milanski ENG525 Usability Testing 25 Mar 1997 1 2 O VERVIEW Formal usability tests how well your software performs with
More informationModerating Usability Tests Principles & Practices for Interacting
Moderating Usability Tests Principles & Practices for Interacting Joe Dumas and Beth Loring Morgan Kaufmann, 2008 ISBN 978-0-12-373933-9 Introduction Getting Started 10 Golden Rules Initial Contacts Interacting
More informationGoogle Cloud Storage Experience Usability Study Plan
Google Cloud Storage Experience Usability Study Plan HCDE 517 Usability Testing Team 5G s (4 Gals + Google): Victoria Briscoe, Vina Chan, Sooyoung Hwang, Shu-Fan Wen ! Goals Google Cloud Storage is part
More informationUsability Testing. with Howard Kiewe. Eight Frequently Asked Questions
Usability Testing with Howard Kiewe Eight Frequently Asked Questions 2008, Howard Kiewe The Eight FAQs We will cover the following questions: 1. What is Usability? 2. What is a Usability Test? 3. Who are
More informationLECTURE 3 REQUIREMENTS GATHERING
LECTURE 3 REQUIREMENTS GATHERING Key Definitions The As-Is system is the current system and may or may not be computerized The To-Be system is the new system that is based on updated requirements The System
More informationHow do we know what we know?
Research Methods Family in the News Can you identify some main debates (controversies) for your topic? Do you think the authors positions in these debates (i.e., their values) affect their presentation
More informationScreen Design : Navigation, Windows, Controls, Text,
Overview Introduction Fundamentals of GUIs Screen Design : Navigation, Windows, Controls, Text, Evaluating GUI Performance - Methods - Comparison 1 Example: Automotive HMI (CAR IT 03/2013) 64, 68, 69 2
More informationClass 2: Qualitative Research Methods Urban and Regional Planning 298A San Jose State University
Class 2: Qualitative Research Methods Urban and Regional Planning 298A San Jose State University Why qualitative methods? Not everything that can be counted counts, and not everything that counts can be
More informationUsing qualitative research to explore women s responses
Using qualitative research to explore women s responses Towards meaningful assistance - how evidence from qualitative studies can help to meet survivors needs Possible questions Why do survivors of SV
More informationRunning surveys and consultations
Running surveys and consultations Susannah Wintersgill, Public Affairs Annette Cunningham & Rob Markham, Estates Tuesday 9 June 2015 Contents Five key stages of planning a survey Questionnaire design:
More informationMarketing Research Core Body Knowledge (MRCBOK ) Learning Objectives
Fulfilling the core market research educational needs of individuals and companies worldwide Presented through a unique partnership between How to Contact Us: Phone: +1-706-542-3537 or 1-800-811-6640 (USA
More informationTotal Recall Survey Report
Total Recall Survey Report Enrico Bertini, Denis Lalanne University of Fribourg Abstract The overall objective of the TotalRecall project is to support humans memory in the professional life, and more
More informationInteraction Design. Chapter 4 (June 1st, 2011, 9am-12pm): Applying Interaction Design I
Interaction Design Chapter 4 (June 1st, 2011, 9am-12pm): Applying Interaction Design I 1 II Applying Interaction Design May June July Applying Interaction Design I What is Design Research? Conducting Design
More informationTips & Tools #17: Analyzing Qualitative Data
Tips & Tools #17: Analyzing Qualitative Data Introduction Qualitative evaluation methods yield narrative data often describing experiences, perceptions, or opinions from key informant interviews, open
More informationFacilitator Script Slacker Usability Test Portable OOBE
Facilitator Script Slacker Usability Test Portable OOBE June 2-3, 2008 This is a rough script for the usability test facilitator. The script shows flow and intent. Deviation from the script will be necessary
More informationCHAPTER III METHODOLOGY. The purpose of this study was to describe which aspects of course design
CHAPTER III METHODOLOGY The purpose of this study was to describe which aspects of course design and/or instruction are more effective and successful in the online environment than in the face-to-face
More informationOutline. Lecture 13: Web Usability. Top Ten Web Design Mistakes. Web Usability Principles Usability Evaluations
Lecture 13: Web Usability Outline Web Usability Principles Usability Evaluations Wendy Liu CSC309F Fall 2007 1 2 What Makes Web Application Development Hard? Target audience can be difficult to define
More informationSurvey research. Contents. Chaiwoo Lee. Key characteristics of survey research Designing your questionnaire
Survey research Chaiwoo Lee Postdoctoral Associate MIT AgeLab chaiwoo@mit.edu agelab.mit.edu Contents Key characteristics of survey research Designing your questionnaire Writing the questions Putting them
More informationVII. Information Acquisition
VII. Information Acquisition What is Information Acquisition? Sampling and Investigating Hard Data Stakeholders and Interviewing Types of Information Acquisition Sampling Hard Data and Using Questionnaires
More informationWhat s Better? Moderated Lab Testing or Unmoderated Remote Testing?
Room: West B 11:30-1:00 What s Better? Moderated Lab Testing or Unmoderated Remote Testing? Susan Fowler You re probably very familiar with moderated in-lab usability testing, and you may have tried a
More informationImproving Government Websites and Surveys With Usability Testing and User Experience Research
Introduction Improving Government Websites and Surveys With Usability Testing and User Experience Research Jennifer Romano Bergstrom, Jonathan Strohl Fors Marsh Group 1010 N Glebe Rd., Suite 510, Arlington,
More informationWebb s Depth of Knowledge Guide
Webb Webb s Depth of Knowledge Guide Career and Technical Education Definitions 2009 1 H T T P : / / WWW. MDE. K 12.MS. US H T T P : / / R E D E S I G N. R C U. M S S T A T E. EDU 2 TABLE OF CONTENTS Overview...
More informationUsing Focus Groups in Program Development and Evaluation
Using Focus Groups in Program Development and Evaluation by Roger A. Rennekamp, Ph.D. and Martha A. Nall, Ed.D. Extension Specialists in Program and Staff Development 203 Scovell Hall University of Kentucky
More informationAlexander Nikov. 4.2. Usability Testing. 1. What is a Usability Test? Usability Testing Questions. 2. Who are the Test Participants?
COMP 3220 Human-Computer Interaction Usability Testing Questions 4.2. Usability Testing Alexander Nikov 1. What is a Usability Test? 2. Who are the Test Participants? 3. Why Usability Test? 4. Why Doesn
More informationWORKBOOK E: CONDUCTING IN-DEPTH INTERVIEWS
WORKBOOK E: CONDUCTING IN-DEPTH INTERVIEWS TABLE OF CONTENTS OVERVIEW OF IN-DEPTH INTERVIEWS... 3 Steps Involved in Conducting In-depth Interviews... 3 Advantages and Disadvantages of In-depth Interviews...
More informationResearch and Research Methods What we will cover: Formal vs. Informal Qualitative vs. Quantitative Primary vs. Secondary Focus Groups In-Depth Interviews Intercept Interviews 1 Research and Research Methods
More informationUsability Testing for Beginners on a Tight Budget
Usability Testing for Beginners on a Tight Budget Introduction Web Development Librarian at LSU Formerly Web Services Librarian at Tulane Held several web design positions at major corporations Usability
More informationA Comparative Study of Database Design Tools
A Comparative Study of Database Design Tools Embarcadero Technologies ER/Studio and Sybase PowerDesigner Usability Sciences Corporation 909 Hidden Ridge, Suite 575, Irving, Texas 75038 tel: 972-550-1599
More informationWhat Are Standards Of Rigor For Qualitative Research? Gery W. Ryan RAND Corporation
What Are Standards Of Rigor For Qualitative Research? Gery W. Ryan RAND Corporation In anticipation of spending two days discussing the question above, I think it would be helpful to make explicit some
More informationChapter 6 Multiple Choice Questions (The answers are provided after the last question.)
Chapter 6 Multiple Choice Questions (The answers are provided after the last question.) 1. According to your text, how many points should a rating scale have? a. Five b. Four c. Ten d. Somewhere from 4
More informationSurveys. Methods for Evaluating Advising Program/U.C. Berkeley 1
Surveys Methods for Evaluating Advising Program/U.C. Berkeley 1 The purpose of surveys in an Advising Program evaluation is to gain insight into respondents attitudes and perceptions about advising services
More informationEvaluating Survey Questions
Evaluating Survey What Respondents Do to Answer a Question Chase H. Harrison Ph.D. Program on Survey Research Harvard University Comprehend Question Retrieve Information from Memory Summarize Information
More informationCustomer Market Research Primer
Customer Market Research Primer This factsheet acts as a primer to the basic concepts of Market Research and how a business or individual could do their own research. Market research is an effective way
More informationINTERCALL STREAMING SURVEY REPORT
INTERCALL STREAMING SURVEY REPORT Methodology CONTACT US 800.820.5855 www.intercall.com An online survey was conducted using the field services of Russell Research. The study was fielded between August
More informationDesign and Development of a Mobile Game - Based Learning Application in Synonyms, Antonyms, and Homonyms
Design and Development of a Mobile Game - Based Learning Application in Synonyms, Antonyms, and Homonyms Ellenita R. Red erred@mcl.edu.ph Kenneth Edward D. Domingo kenneth.domingo07@gmail.com Kristian
More informationAdvertising Research
Advertising Research Five Basic Methods of Market Research Source: Dun and Bradstreet Media, AllBusiness.com While there are many ways to perform market research, most businesses use one or more of five
More informationWhat do I do with all this text? Demystifying qualitative analysis
What do I do with all this text? Demystifying qualitative analysis ROSE HENNESSY, MPH PREVENTION & EVALUATION COORDINATOR WISCONSIN COALITION AGAINST SEXUAL ASSAULT Objectives Learn and practice three
More informationA study to investigate Students and Teachers attitudes towards the use of computer games in St Johns School, to assist learning.
A study to investigate Students and Teachers attitudes towards the use of computer games in St Johns School, to assist learning. 1 Contents Page Abstract Page 3 Introduction Page 4 Methodology Page 5 and
More informationSoftware Development and Usability Testing
Software Development and Usability Testing Shneiderman, Chapter 4 Preece et al, Ch 9, 11-15 Krug, Rocket Surgery Made Easy Rubin, Handbook of Usability Testing Norman Neilsen Group www HCI in Software
More informationEvaluating Training. Debra Wilcox Johnson Johnson & Johnson Consulting
Debra Wilcox & Consulting Learning new behavior new or enhanced skills is the most powerful outcome of training. Behavioral change is the most difficult outcome to achieve, but may be the most important.
More informationWR305, Writing for the Web Usability Test Documents
WR305, Writing for the Web Usability Test Documents Introduction This documents contains the portfolio usability test informed consent form, participant demographic questionnaire, protocols, and surveys
More informationToolkit for the OSEP TA & D Network on How to Evaluate Dissemination A Component of the Dissemination Initiative
Toolkit for the OSEP TA & D Network on How to Evaluate Dissemination A Component of the Dissemination Initiative Richard Sawyer, PhD National Dissemination Center for Children with Disabilities Contents
More informationInteraction Design. Chapter 5 (May 22nd, 2013, 9am-12pm): Applying Interaction Design
Interaction Design Chapter 5 (May 22nd, 2013, 9am-12pm): Applying Interaction Design 1 II Applying Interaction Design May June July 2 Applying Interaction Design What is Design Research? Conducting Design
More informationCHAPTER THREE: METHODOLOGY. 3.1. Introduction. emerging markets can successfully organize activities related to event marketing.
Event Marketing in IMC 44 CHAPTER THREE: METHODOLOGY 3.1. Introduction The overall purpose of this project was to demonstrate how companies operating in emerging markets can successfully organize activities
More informationConsulting Performance, Rewards & Talent. Making Employee Engagement Happen: Best Practices from Best Employers
Consulting Performance, Rewards & Talent Making Employee Engagement Happen: Best Practices from Best Employers The Challenge Companies across the globe are taking the initiative to administer and manage
More informationDetermines if the data you collect is practical for analysis. Reviews the appropriateness of your data collection methods.
Performing a Community Assessment 37 STEP 5: DETERMINE HOW TO UNDERSTAND THE INFORMATION (ANALYZE DATA) Now that you have collected data, what does it mean? Making sense of this information is arguably
More information5. project evaluation
5. project evaluation CONTENTS 5.1 EVALUATION...5.2 5.2 TIMING...5.2 5.3 UNDERTAKING EVALUATION...5.3 5.4 EVALUATION METHODS...5.3 5.4.1 Asking people...5.3 5.4.2 Physical methods...5.5 COASTAL PLANNING
More informationCourse Catalog. www.airweb.org/academy
www.airweb.org/academy Course Catalog 2015 Hosted by the Association for Institutional Research, Data and Decisions Academy courses provide self-paced, online professional development for institutional
More informationSo today we shall continue our discussion on the search engines and web crawlers. (Refer Slide Time: 01:02)
Internet Technology Prof. Indranil Sengupta Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture No #39 Search Engines and Web Crawler :: Part 2 So today we
More informationCoaching: bringing out the best. Opinion piece Philip Brew
Coaching: bringing out the best Opinion piece Philip Brew Coaching: bringing out the best Philip Brew 1/6 Organisations need fully functioning human beings In today s demanding and complex environments,
More informationGE Capital The Net Promoter Score: A low-cost, high-impact way to analyze customer voices
GE Capital The Net Promoter Score: A low-cost, high-impact way to analyze customer voices The Net Promoter Score: A low cost, high impact way to analyze customer voices GE Capital s Net Promoter survey
More informationITS Training and Documentation Needs Assessment Project Report
ITS Training and Documentation Needs Assessment Project Report Sharon Beltaine User Support Services Information Technology Services Ithaca College Report ITS Training and Documentation Needs Assessment
More informationStages of Instructional Design V. Professional Development
Stages of Instructional Design V. Professional Development Derived from Gagné, R. M., Briggs, L. J., & Wager, W. W. (1992). Principles of Instructional Design (4th ed.). Fort Worth, TX: Harcourt Brace
More informationUsability Testing Basics. An Overview
Usability Testing Basics An Overview Contents Usability Testing Defined... 3 Decide What to Test... 3 Determine When to Test What... 4 Decide How Many to Test... 4 Design the Test... 5 Consider the Where,
More informationUser interface design. Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 16 Slide 1
User interface design Ian Sommerville 2004 Software Engineering, 7th edition. Chapter 16 Slide 1 Objectives To suggest some general design principles for user interface design To explain different interaction
More informationHow To Collect Data From A Large Group
Section 2: Ten Tools for Applying Sociology CHAPTER 2.6: DATA COLLECTION METHODS QUICK START: In this chapter, you will learn The basics of data collection methods. To know when to use quantitative and/or
More informationFan Fu. Usability Testing of Cloud File Storage Systems. A Master s Paper for the M.S. in I.S. degree. April, 2013. 70 pages. Advisor: Robert Capra
Fan Fu. Usability Testing of Cloud File Storage Systems. A Master s Paper for the M.S. in I.S. degree. April, 2013. 70 pages. Advisor: Robert Capra This paper presents the results of a usability test involving
More informationInformation collection methods Choosing tools for assessing impact
Information collection methods Choosing tools for assessing impact Acknowledgements This guide has been developed by Sally Cupitt and Jean Ellis for Charities Evaluation Services. This second edition was
More informationUsability Testing (in HCI 304-424B) McGill University, January 26 th, 2006. Usability Testing. with Howard Kiewe. Seven Infrequently Asked Questions
Usability Testing with Howard Kiewe Seven Infrequently Asked Questions 2006, Howard Kiewe The Seven IAQs We will cover the following questions: 1. What is a Usability Test? 2. Who are the Test Participants?
More informationAnalysing Qualitative Data
Analysing Qualitative Data Workshop Professor Debra Myhill Philosophical Assumptions It is important to think about the philosophical assumptions that underpin the interpretation of all data. Your ontological
More informationTrier 4. Marketing Research MANAGING MARKETING INFORMATION TO GAIN CUSTOMER INSIGHTS
Trier 4 Marketing Research MANAGING MARKETING INFORMATION TO GAIN CUSTOMER INSIGHTS Marketing Information and Customer Insights Fresh and deep insights into customers needs and wants Difficult to obtain
More informationMarket Research: Friend or Foe to Christian Charities?
Market Research: Friend or Foe to Christian Charities? by Redina Kolaneci Senior Fundraising & Stewardship Consultant McConkey Johnston international UK If you are ignorant of yourself, in addition to
More informationEvaluation: Designs and Approaches
Evaluation: Designs and Approaches Publication Year: 2004 The choice of a design for an outcome evaluation is often influenced by the need to compromise between cost and certainty. Generally, the more
More information'& ##! %1# ##!!* #!!! 23!!!
!" 1 !"# $ Companies invest a lot of time and money into tools for functional test automation and performance testing. I think this tends to lead to the belief that the best way to utilize these tools
More informationAppendix N INFORMATION TECHNOLOGY (IT) YOUTH APPRENTICESHIP WEB & DIGITAL COMMUNICATIONS PATHWAY WEB & DIGITAL MEDIA UNIT UNIT 6
Appendix N INFORMATION TECHNOLOGY (IT) YOUTH APPRENTICESHIP WEB & DIGITAL COMMUNICATIONS PATHWAY WEB & DIGITAL MEDIA UNIT UNIT 6 Web & Digital Communications Pathway: (Unit 6) PAGE 1 OF 12 Unit 6: Pathway
More informationMARKETING RESEARCH AND MARKET INTELLIGENCE (MRM711S) FEEDBACK TUTORIAL LETTER SEMESTER `1 OF 2016. Dear Student
MARKETING RESEARCH AND MARKET INTELLIGENCE (MRM711S) FEEDBACK TUTORIAL LETTER SEMESTER `1 OF 2016 Dear Student Assignment 1 has been marked and this serves as feedback on the assignment. I have included
More informationSoftware Engineering. Requirements elicitation - Facts finding. Software Engineering Requirements Elicitation Slide 1
Software Engineering Requirements elicitation - Facts finding Software Engineering Requirements Elicitation Slide 1 Chapter Objectives To introduce software the Requirements Engineering Process To describe
More informationResearch and Digital Game- based Play: A Review of Martha Madison
Research and Digital Game- based Play: A Review of Martha Madison White Paper Compiled by Anne Snyder, Ph.D. Second Avenue Learning is a certified women- owned business, specializing in the creation of
More informationwww.doncaster.gov.uk
Market Research It is essential that market research is undertaken to establish there is a need for childcare within a specified area and have knowledge of the local community that a business wishes to
More informationWriting Better Survey Questions
Writing Better Survey Questions Mike Pritchard 5 Circles Research 425-968-3883, mikep@5circles.com, www.5circles.com, @thatresearchguy Why are we here? Increase competence and consciousness Good questions
More informationTIPSHEET IMPROVING RESPONSE SCALES
TIPSHEET IMPROVING RESPONSE SCALES As important as it is to have good question wording in a survey, it is equally important to have well constructed and reliable question response options. Researchers
More informationClient Satisfaction Quality Improvement Tool and Sample Surveys
Client Satisfaction Quality Improvement Tool and Sample Surveys August 2011 Prepared For California Department of Public Health, Office of Family Planning Family PACT Technical Assistance and Training
More informationResearch design and methods Part II. Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT
Research design and methods Part II Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT From last week Research methodology Quantitative vs. Qualitative vs. Participatory/action research Research methods
More informationUsing Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP)
Using Qualitative Information For Impact Assessment: Introducing The Qualitative Impact Assessment Protocol (QUIP) By Katie Wright-Revolledo, INTRAC Introduction Qualitative methods can provide insight
More informationGuidelines for Using the Retrospective Think Aloud Protocol with Eye Tracking
Guidelines for Using the Retrospective Think Aloud Protocol with Eye Tracking September, 2009 Short paper by Tobii Technology Not sure of how to design your eye tracking study? This document aims to provide
More informationDoes self-reflection improve learning autonomy and student satisfaction with feedback?
Self-reflection as a tool in improving student satisfaction with feedback? Jarka Glassey School of Chemical Engineering and Advanced Materials, Newcastle University SUMMARY Project Aims Dimensions of this
More informationBy Judith Sharken Simon
How to Conduct a Focus Group By Judith Sharken Simon While focus groups are often seen as a technique reserved for corporations involved in expensive, competitive advertising, they can also be extremely
More informationSemi-structured interviews
Semi-structured interviews 3 rd December 2014 Prof. Edwin van Teijlingen BU Graduate School Aim of this session: introduce various ways of conducting interviews with qualitative methods; outline strength
More informationE-learning Course Design Don McIntosh, Ph.D. - February 23, 2006
E-learning Course Design Don McIntosh, Ph.D. - February 23, 2006 Introduction Imagine your company is embarking on a program to implement e-learning to complement its classroom training program, and you
More informationOnboarding and Engaging New Employees
Onboarding and Engaging New Employees Onboarding is the process of helping new employees become full contributors to the institution. During onboarding, new employees evolve from institutional outsiders
More informationExploratory Research Design. Primary vs. Secondary data. Advantages and uses of SD
Exploratory Research Design Secondary Data Qualitative Research Survey & Observation Experiments Företagsakademin, Henriksgatan 7 FIN-20500 Åbo Primary vs. Secondary data Primary data: originated by the
More informationManual on Training Needs Assessment
Project on Improvement of Local Administration in Cambodia What is Training Needs Assessment? Five Steps of Training Needs Assessment Step 1: Identify Problem and Needs Step 2: Determine Design of Needs
More informationAnalyzing Qualitative Data For Assessment October 15, 2014
Analyzing Qualitative Data For Assessment October 15, 2014 Jeremy Penn, Ph.D. Director Resources Data Analysis, Chapter 6 in Schuh book (pages 158-168) Creswell (2002) or (1998) you can borrow from me
More informationAssessing Online Asynchronous Discussion in Online Courses: An Empirical Study
Assessing Online Asynchronous Discussion in Online Courses: An Empirical Study Shijuan Liu Department of Instructional Systems Technology Indiana University Bloomington, Indiana, USA shijliu@indiana.edu
More informationKMYMONEY 1.0.2 USABILITY TESTING REPORT
KMYMONEY 1.0.2 USABILITY TESTING REPORT Grace Chang & Pallavi Damera Instructor: Judy Ramey University of Washington Page 1 of 25 TABLE OF CONTENTS EXECUTIVE SUMMARY Product Description... 3 Audience...
More informationQualitative Methods in Empirical Studies of Software Engineering. by Carolyn B. Seaman
Qualitative Methods in Empirical Studies of Software Engineering by Carolyn B. Seaman Overview topics of the paper qualitative methods data collection methods participant observation interviewing coding
More information