Usability Testing Guiding Principles Finding Usability Problems This is a step by step guideline to usability testing. Usability testing is Usability testing is testing anything with which users interact, for the purpose of assessing and modifying functionality. Usability testing stems from the purpose of a project. It is encouraged to review the goals of a project, take a step back and evaluate the parameters. In regards to usability testing, qualitative testing is the focus, while quantitative testing, if possible and appropriate, can be used, as well. 1. Are you ready for testing? Each project typically has several different phases. Usability testing is valuable in each step and can vary from each stage. In the MyNWCCD portal usability testing, the first phase was approximately 15 minutes, made up of basic functionality questions. The testing varied through each phase. Do not engage in the usability testing process if you do not intend to use the feedback. Keep in mind that when testing users, you create an assumption for the user that change to the project is the end result. It is vital to understand the entire process of making a change to the tested project. Ask yourself: Will performing usability testing and receiving feedback alter this project in any way? 2. How to pick what to actually test. (Prioritize) Usability testing can be done in nearly any time frame of a project. If your project is in a very beginning phase, testing can be done using sketches of the interface or user experience. If your project is already in a launched phase, you can test the actual user interface or experience. Usability testing can literally be done at any phase in a project s lifetime. You should identify the basic functional elements of the project, if they work and are they user-friendly. You should also identify the most important aspect of the project, and if the suitable questions are binary (involving two things) or process-related. Usability testing can be tailored to garnering feedback from known problem areas. Testing can be done via survey, websites, Web applications, online forms and a variety of additional methods. You should brainstorm and prioritize a list to decide what to test first. It should be decided if the project is long term or short term, and if it is intended to create a culture of testing. If it is a single project, a simple testing schedule may suffice. Learning from prior phases is an important part of this process. Use the information gathered, if you have had prior phases, to build on what you are testing. Retesting past problems or areas you have addressed will let you know if you have successfully made the process easier for users. 3. How to actually build the test. User testing can be done electronically, through a focus group, one on one, or through a list of questions that help mold the project in early phase. Create the actual questions or script, which should be objective, neutral and without bias. To warrant useful feedback, questions should not be leading. You
should determine how you will present the test. How long do you want to keep the user testing the project? This will help determine the length of the script, such as how many questions will be presented. Best practices for how long to keep your users in a test are determined by many different factors. Are you giving the users an incentive for giving you their time? Are the users people who live busy, hectic lifestyles? Making it clear up front how long the test takes is important for keeping your users engaged and happy. For rough estimates: 5-10 minutes would be considered a very short length test, 10 20 minutes would be a short length test, 20 40 minutes would be a medium length test, and over 45 minutes would be considered a long length test. 4. Determine how to choose your testers. Identify your target audience or constituency. Your pool of testers should come from this group. Testers should identity how many users should be tested to reflect a sufficient microcosm. The scientific process, however, should not trump common sense. Incentives can also be provided to users for taking time to test. Doing usability testing in any capacity, no matter how small the number of users, is more important than not doing it at all. Pick a manageable amount of users that you and your team can handle and focus on delivering a quality test. Whether that number is five users or 40 users, you will most likely see problem areas come about in your project. 5. How to conduct a successful test. Usability testing can be done electronically, one on one, or in a group setting. Identity which format is appropriate for your project to warrant the most useful feedback. Who administers the test is an important aspect to not overlook. The tester should be confident enough to deliver the test in an unbiased, objective manner. Be cautious of allowing team members who have close emotional ties to the project deliver the test. Always remember that you are testing the project, not the user. Make this clear upfront. Providing a comfortable environment for the user is just as important as designing the questions for the test. 6. Have others come view the sessions. If possible, having third parties view the sessions can be useful in assessing the testing. This can be important for the outcome as it can provide neutral feedback to enhance the testing process. Fixing Usability Problems 1. Discussing the test results When studying the results of usability testing, identifying trends and key issues is imperative. Some aspects are easy fixes and should be addressed. If there are major roadblocks or deal breakers, address them immediately. Results should also present which aspects need to be retested. Results can also provide guidelines to adjustments that need attention. You should learn what you did correctly and what needs to be modified in the actual test. You should alter the test accordingly and document changes.
2. Fixing the problems and retesting Document all changes to the usability tests. Do not sit on problems. Schedule time to modify the test and execute. 3. How often you should test. The tenure of the test should correlate with the length of the project. Is it a long-term or short-term project? Document the test settings and how often each testing phase occurred. Review your testing plan and add more phases if warranted.
Appendix Sample scripts MyNWCCD Phase 2 Usability test script Hi,. My name is, and I m going to be walking you through this session today. Before we begin, I have some information for you, and I m going to read it to make sure that I cover everything. You probably already have a good idea of why we asked you here, but let me go over it again briefly. We re asking people to try using an early version of our portal that we re working on so we can see whether it works as intended. The session should take about 20 minutes. The first thing I want to make clear right away is that we re testing the site, not you. You can t do anything wrong here. In fact, this is probably the one place today where you don t have to worry about making mistakes. As you use our portal, or MyNWCCD, I m going to ask you as much as possible to try to think out loud: to say what you re looking at, what you re trying to do, and what you re thinking. This will be a big help to us. Also, please don t worry that you re going to hurt our feelings. We re doing this to improve the site, so we need to hear your honest reactions.
If you have any questions as we go along, just ask them. I may not be able to answer them right away, since we re interested in how people do when they don t have someone sitting next to them to help. But if you still have any questions when we re done I ll try to answer them then. Also, is here. Do you have any questions so far? OK. Before we look at the site, I d like to ask you just a couple of quick questions. Now, roughly how many hours a week altogether just a ballpark estimate would you say you spend using the Internet, including Web browsing and email, at work and at home? (0 5 hours) (6 15 hours) (16 40 hours) (41+ hours) And out of the number what is the percentage of time used to check your email? (0 25%) (26 50%) (51 75%) (76 100%) OK, great. We re done with the questions, and we can start looking at things. Web browser should now be open to MyNWCCD home page First, I m going to ask you to look at this page and tell me what you make of it: what strikes you about it, whose site you think
it is, what you can do here, and what it s for. Just look around and do a little narrative. You can scroll if you want to, but don t click on anything yet. Thanks. Now I m going to ask you to try doing some specific tasks. I m also going to ask you to do these tasks without using Search. We ll learn a lot more about how well the site works that way. And again, as much as possible, it will help us if you can try to think out loud as you go along. Allow the user to proceed until you don t feel like it s producing any value or the user becomes very frustrated. Repeat for each task or until time runs out. Start of Questions: Student 1. Where would you go if you need to find information about our Campus Police Safety & Health/Campus Police Have the user actually click on the link this time. 2. From here, how would you find your way back to your MyNWCCD home page? 3. Where would you find student employment opportunities? Financial Life/Student Employment or Student Services/Student Employment
4. Where would you find information about the hours for lunch? Campus Life/Food Services 5. Where would find degree requirements? Academic Life/Degree Requirements 6. On your MyNWCCD home page, where would you find a way to register for classes? WebAdvisor webpart 7. Where would you go to check the balance of your account? Financial Life/Manage My Account 8. On your MyNWCCD home page, find out how many English classes are available next semester. WebAdvisor webpart 9. Where would you look to find out what events are happening this week on campus? Announcements, Campus News, or Campus Events webpart Employee 1. Where would you go if you need to find information about our Campus Police Safety & Health/Campus Police
2. From here, how would you find your way back to your MyNWCCD home page? 3. Where would you find student employment opportunities? Financial Life/Student Employment or Student Services/Student Employment 4. Where would you find information about the hours for lunch? Employee Information/Food Services 5. Where would find information on Community Education? Academic Life/Community Education 6. From looking at your MyNWCCD home page, where would you find a way to register for classes? WebAdvisor webpart 7. Where would you go to check the balance of your account? Financial Life/Manage My Account 8. On your MyNWCCD home page, find out how many English classes are available next semester. WebAdvisor webpart 9. From looking at your MyNWCCD home page, where would you look to find out what s happening on campus? Announcements, Campus News, or Campus Events webpart
10. Please post an announcement to the MyNWCCD home page that you are excited to start using MyNWCCD. Thanks, that was very helpful. Do you have any questions for me, now that we re done? Thank them and escort them out.