Software Testing CS 408

Transcription

1 Software Testing CS 408 Lecture 7: Usability Testing 2/7/17 Myers, Chapter 7

2 Basics An important category of system test cases is one that attempts to find human-factor, or usability, problems Some questions you might ask to derive testing considerations: l. Has each user interface been tailored to the intelligence, educational background, and environment of the end user? 2. Are the outputs of the program meaningful, appropriate for the typical user, and devoid of computer gibberish? 3. Are the error messages straightforward or does the user need a degree in Computer Science to comprehend them? 4. Does the total set of user interfaces exhibit an underlying consistency and uniformity of syntax, conventions, semantics, format, style, and abbreviations? 5. Does the system contain an excessive number of options, or options that are unlikely to be used? 2

3 Basics 7. Does the system return some type of immediate acknowledgment for all input? Item can change color or a button object can depress Processing time -- message should be displayed informing the user of what is going on 8. Is the program easy to use? Is the input case-sensitive without making this fact clear to the user? If a program requires navigation through a series of menus or options, is it clear how to return to the main menu? Can the user easily move up or down one level? Can an action be easily undone (when possible)? 9. Is the design conducive to user accuracy? More important -- Does the design make it too easy for users to do in-accurate things? 10. Are user actions easily repeated in later sessions? Is the software design conducive to the user learning how to be efficient in using the system? 3

4 Process Usability or user-based testing basically is a black-box testing technique If users perceive, because of improper design, a cumbersome user interface, or specifications missed or ignored, that a given application does not perform according to its specifications, the development process is viewed as a failure! Usability testing is more than simply seeking user opinions or high-level reactions to a software application Any usability test should begin with a plan Establish practical, real-world, repeatable exercises for each user to conduct Design these testing scenarios to present the user with all aspects of the software During each phase of the test, have observers document the user experience as they perform each task When the test is complete, conduct an interview with the user Write down detailed instructions for user tests, to ensure that each user starts with the same information, presented in the same way 4

5 Test User Selection A complete usability testing protocol typically involves multiple tests from the same users, as well as tests from multiple users Reason for multiple tests from same users -- - One area we want to test is user recall, how much of what a user lea about software operation is retained from session to session Software targeted for a specific end-user type or industry should be test by expert users, people already familiar with this class of application in real-world environment Software with a more general target market -- - Hallway testing, users chosen for software testing are selected at ra from people passing by in the hallway 5

6 How many users? You need to find the right number of testers who can identify the most errors for the least cost Approximately 83 percent of the errors are detected by only 5 testers Approximately 98 percent of the errors are detected by 10 testers Controversial? But, it is clear that you do not need thousands... or even hundreds... of user testers Among the considerations important to developers who are designing a usability testing program are whether the number of users and their individual orientations represent sufficiently the total population of potential users 6

7 Data Gathering Methods Video recording a user test and using a think-aloud protocol can provide excellent data on software usability and user perceptions about the application Test participants describe out loud their task, what they are thinking about the task, and/or whatever else comes to their mind as they move through the testing scenario Follow up with participants after the test to get posttest comments, feelings, and observations Remote user testing -- - Application is installed at the user's location where the software may ultimately be applied - Remote testing has the advantage of placing the user in a familiar environment, one in which the final application likely will be used, thus removing the potential for external influences modifying test results 7

8 Usability Questionnaire Develop questionnaires that generate responses that can be counted and analyzed - Yes/no answers - True/false answers - Agree/disagree on a scale Instead of asking "What is your opinion of the main menu system," ask a series of questions that requ an answer from l to 5, where 5 is totally agree and l is totally disagree: - l. The main menu was easy to navigate The screen design led me quickly to the correct software operational choices The menu operations did not provide enough feedback to verify my choices Good practice to ask the same question more than once, but present it from the opposite perspective s that one elicits a negative response and the other a positive one. 8