Usability Testing. with Howard Kiewe. Eight Frequently Asked Questions

Transcription

1 Usability Testing with Howard Kiewe Eight Frequently Asked Questions 2008, Howard Kiewe The Eight FAQs We will cover the following questions: 1. What is Usability? 2. What is a Usability Test? 3. Who are the Test Participants? 4. Why Usability Test? 5. Why Doesn t Everyone Test? 6. When to Usability Test? 7. Where to Usability Test? 8. How to Usability Test? Usability Testing, 1/9/2008 McGill University HCI B 2 Howard Kiewe 1

2 1. What is a Usability? The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction. (ISO 9241) Usability can be objectively measured Usability Testing, 1/9/2008 McGill University HCI B 3 2. What is a Usability Test? The evaluation of a product s usability through direct observation of user behavior during a structured task. Also called a usability evaluation or simply an evaluation. Key Concepts: User not Usability Expert (unlike a heuristic evaluation) User Behavior not User Opinion (unlike a survey) Direct Observation not Indirect Obser. (unlike a server log) Structured Task not Any Task (unlike an ethnography) Usability Testing, 1/9/2008 McGill University HCI B 4 Howard Kiewe 2

3 3. Who are the Test Participants? A representative sample of the user population Usability Testing, 1/9/2008 McGill University HCI B 5 4. Why Usability Test? Reasons Your users are not like you Your users cannot explain their usability issues Goals Find usability issues Verify that a user interface (UI) meets its usability goals Choose between competing designs Bring all members of the development team on board Usability Testing, 1/9/2008 McGill University HCI B 6 Howard Kiewe 3

4 5. When to Usability Test? Business Objectives User Research UX Requirements Prerequisites User experience (UX) requirements & goals Prototype Phases Low fidelity High fidelity UX Evaluation UX Design Usability Testing, 1/9/2008 McGill University HCI B 7 6. Why Doesn t Everyone Test? It takes time and costs money: Step Hours Design test plan & materials 32 Design test environment 8 Run pilot test 8 Revise test tasks/materials 8 Run test/collect data 32 Summarize data 16 Document/present results 40 Total 144 Why invest the resources when: My users are like me or My users can explain their usability issues *Adapted from Mayhew (1999) The Usability Engineering Lifecycle Usability Testing, 1/9/2008 McGill University HCI B 8 Howard Kiewe 4

5 7. Where to Usability Test? User s Environment High ecological validity Poorly controlled Inexpensive Convenient for the user Potential interruptions Videotaping inconvenient Third-party observations intrusive Usability Lab Low ecological validity Well controlled Expensive Inconvenient for the user No interruptions Videotaping convenient Third-party observations unobtrusive Usability Testing, 1/9/2008 McGill University HCI B 9 8. How to Usability Test? Seven Steps: 1. Complete Prerequisites 2. Plan 3. Develop Materials 4. Pilot Test 5. Recruit Participants 6. Test 7. Summarize, Analyze, & Report Usability Testing, 1/9/2008 McGill University HCI B 10 Howard Kiewe 5

6 Prerequisites & Plan Prerequisites 1. UX requirements & goals 2. Prototype Plan 1. Choose test focus Ease or learning vs. Ease of use 2. Choose a user focus Cannot test everyone 3. Design a test task Cannot test all tasks Usability Testing, 1/9/2008 McGill University HCI B 11 Develop Test Script 1. Observer briefing 2. User introduction 3. Informed consent 4. Non Disclosure Agreement 5. Pretest questionnaire 6. User or training documentation (optional) 7. Test tasks 8. Data collection sheet 9. Posttest questionnaire Usability Testing, 1/9/2008 McGill University HCI B 12 Howard Kiewe 6

7 Test Script (2) User introduction Thank you & about the product We are testing the interface, not you It s weird. I m just going to observe. Data collection sheet Action, User comments, Observations Posttest questionnaire (mostly affect, usually a Likert Scale) The instructions and prompts are helpful: Strongly Agree Strongly Disagree Standard instruments available, ie: Software Usability Measurement Inventory (SUMI) Usability Testing, 1/9/2008 McGill University HCI B 13 Pilot Test 1. Recruit pilot user 2. Set up test environment 3. Run pilot test 4. Revise materials & environment 5. Adjust timing Don t skip the pilot! Usability Testing, 1/9/2008 McGill University HCI B 14 Howard Kiewe 7

8 Recruit Participants users per test run hours per test, 30 min between them. 3. Consider participant motivation 4. Consider an incentive 5. Recruitment is work. Be persistent. Usability Testing, 1/9/2008 McGill University HCI B 15 Test Facilitator Role Consider think aloud, pair testing, or posttest review Don t lead or help Ask questions to gain insight Avoid distracting users Ask posttest questions Observer Role Take notes Usability Testing, 1/9/2008 McGill University HCI B 16 Howard Kiewe 8

9 Questions to Gain Insight Ask What are you thinking? Is that what you expected? What would you like to accomplish? Avoid Why (encourages justification) Usability Testing, 1/9/2008 McGill University HCI B 17 Summarize, Analyze, & Report Summarize & Analyze Count # of errors, time per task Use only descriptive statistics like averages Report List elements that work List issues, including frequency & severity Make suggestions for improvement Usability Testing, 1/9/2008 McGill University HCI B 18 Howard Kiewe 9

10 Exercise: 1) Scripting Develop Short Script 1. User introduction (1 paragraph) 2. Test tasks (1 sentence) 3. Data collection sheet (action, user comment, observations) 4. Posttest questionnaire (a few Likert questions) For These Goals 1. Quickly and easily set the time. 2. Rate the phone or watch as easy to use and enjoyable to use. Usability Testing, 1/9/2008 McGill University HCI B 19 Exercise: 2) Testing Facilitator Role Read introduction & test tasks Use paired testing Ask questions to gain insight Ask posttest questions Observer Role Take notes on data collection sheet Usability Testing, 1/9/2008 McGill University HCI B 20 Howard Kiewe 10

11 Exercise: 3) Reporting To Class: List elements that work List issues, including frequency & severity Make suggestions for improvement Usability Testing, 1/9/2008 McGill University HCI B 21 Usability Reading Kiewe, H. (2006). The UXE White Paper: User Experience Engineering Essentials [Electronic Version]. user-experience.info, Retrieved January 9, 2008 from User experience engineering (UXE) is a structured research, design, and evaluation process whose goal is to make user interactions with a product or service easy, efficient, and enjoyable. This white paper summarizes the UXE process and evaluates its business case. Mayhew, D. J. (1999). The Usability Engineering Lifecycle: A Practitioner's Handbook for User Interface Design San Diego: Morgan Kaufmann. Detailed presentation of the usercentred design process. NCI. (2006). Research-Based Web Design & Usability Guidelines. Retrieved January 9, 2008, from Lists web design and usability principles and rates them according to the strength of supporting research. A quick way to learn about and evaluate design principles. Updated regularly. Stone, D. L., Jarrett, C., Woodroffe, M., & Minocha, S. (2005). User Interface Design and Evaluation. San Francisco, CA: Elsevier/Morgan Kaufmann. A recently published textbook on user interface design from a user-centred and human factors perspective. Broad and current coverage in an accessible style. Usability Testing, 1/9/2008 McGill University HCI B 22 Howard Kiewe 11

12 Thank you! Howard Kiewe howard [at] kieweconsulting [dot] com hkiewe [at] cim [dot] mcgill [dot] ca +1 (514) User experience, usability, & e-learning research & design Usability & user experience (UX) research, design, & consulting Task analysis, ethnography, & focus groups Information architecture (IA), GUI & interaction Design Usability testing (remote & local), heuristic evaluation Usability Testing, 1/9/2008 McGill University HCI B 23 Howard Kiewe 12