THINK ALOUD PROTOCOL ANALYSIS

Transcription

1 THINK ALOUD PROTOCOL ANALYSIS User Testing I tested five people using the think aloud protocol analysis procedure. The design I used for testing was the revised version of my user interface after the HCI Cognitive Walkthrough and HCI Heuristic Analysis was completed. This was the third revision of my project. 3 rd Version my User Interface. During the think aloud protocol analysis, I received feedback that the reading text is really good for some learners. Each user in the think aloud analysis stated that they preferred the text over the video. After discussing this with my clients, they wanted both the text and the video so the learners could choose which one they want to review. Taking the redundancy principle into consideration and my client s feedback, I decided to put both the video on the page and the text explaining the program. Three of the users in the think aloud test stated that they really liked the colors or states of the buttons.

2 The procedure I used during testing was instructions and practice tasks. The instructions were the following two practice tasks. 1. You are a new undergraduate UVU college student in the Digital Media Department. What is the Digital Media Department Portfolio Review Acceptance program? 2. Who is your Digital Media Academic Advisor and what is their contact information? The users completed the steps to find out about the Portfolio Review Acceptance program by clicking on the Portfolio Review button in the user system. This took them through the stepby-step process of the program by reading text or reviewing a short video with pictures and music. They also were to find who their academic advisor is and the contact information. One user also went into the Tutorial module and completed the captivate tutorial. The general characteristics of the participants are adults over 18, somewhat computer savvy; most of them are teachers so they understand cognitive processes. There was one nurse who tested my interface. She gave me some good feedback about the words on the Department Advisors page. She stated that the color of the instructions was hard to read and one of the advisor s pictures was too small. I also received feedback from another user about these same items. I changed the wording to a darker color and put a larger size picture on the page. This was changed because some users may have eyesight problems. Even if they don t have eyesight problems, it is frustrating when information is hard to read.

3 I changed the color of this wording because several of the participants in the think aloud analysis stated that it was hard to read with this light green color. I also put a different picture of Joyce on this page because several participants thought it was too hard to see her.

4 The following is the updated version UPDATED PAGE WITH TEXT COLOR CHANGE AND NEW PICTURE Comments by Users in the Think Aloud Analysis: Test User #1: Weird title on the page, What is overview? I m not sure of the steps I need to take for Portfolio Review after reviewing the video so I will go into the text version. Actual steps listed, I like the list. Video version did not give the steps in the same way. I understand it now after reading the text. I like the text best on the Portfolio Review page. easy to get back on the tab

5 I need to find out about my advisor so I m viewing tabs and I click on Department Advisors. Tells me areas of specialties. The color of the font is too light. Light green is hard to see. Test User #2: Click on Portfolio Review and I m assuming I can find out information about it. Thought big writing was a link but then I realize the link below is what I should click on. Choose video or text. I m going to choose text. Steps for getting into the program. User reads steps. Now I need to find my advisor so I need to go back. Click on tab to close, now I m back to page. There is a link to Department Advisors. I see the advisors and what they specialize in and I can call for an appointment. I m going to choose Leann because she advises for what I am interest in. address and phone number to call for an appointment. Trying to click on the picture, nothing happens.

6 Moderate issues found in HCI Think Aloud Analysis: Picture is too small, writing is too light, and when you click on the picture nothing happens. Users thought the picture would be a link. Test User #3: Portfolio Review and Overview & Explanation title is weird wording. Where is explanation in the video? Portfolio examples is nice to see. Cool flowchart for Internet Technologies information on the Degree Emphasis page. Found two different advisors to call on the Department Advisors tab. My advisor is Joyce Porter because I m going to do Internet Technologies. I can her or call to schedule an appointment This tester actually completed the tutorial with the following comments: Maybe this tutorial will tell me more about what the Portfolio Review program is. Speech to text, what are the dates? Take a quiz and get 80%. In the survey section, I still have questions about like deadlines. Test User #4: Not sure which is a process or a program? Animation is cute, UVU mascot or something? On the Portfolio Review page, I m not sure if the video & text are going to give me the exact same info, one is just video and one is text? I m going to look at the text first because I hate wasting my time on videos. Steps to the program identified in text but I wasn t sure when I was on the home page if it was a program or a process but the text makes it clear it is a program. The text is more straightforward. I m supposed to find my advisor next. I want to look at Department Advisors. That seems pretty clear. It s kind of hard to see her (Joyce) because she s far away and it s (picture) so small. I love when they have pictures because it s sometimes really helpful that you know you re going to the right person. In the office you can recognize the person. You know the person you are looking for because you ve seen her. I think this is super helpful. So basically, these are the advisors. It seems like one of them would be my advisor but I m not sure which one because I m not sure if I m specializing in Internet Technologies or Project and Information Management. So, Gaming and Animation, Digital Motion Picture Production, and Audio. Huh, so I guess it s just one of these two people but I m not sure how you choose between them. Maybe this person, Heather Rubey, does it. I wonder if I could just her because I hate calling because I m always either on the bus or doing something. Test User #5: The text is easier to review and understand. I can see all the steps I need to do to accomplish the Portfolio Review. I need to find out who my advisor is. I m clicking on this picture and nothing is happening. Don t know if it is a link or if it is just not working. Light green is kind of hard to read. I don t know which emphasis I want to do yet. So, maybe I will call the administrative assistant to see if I can talk to one of them. It s kind of hard to read & see this page for old eyes. SUMMARY OF TESTING I found several critical showstopper issues in the HCI Cognitive Walkthrough. These were the problems with the buttons, they were not consistent, and there were no states for the buttons so the users didn t know where they were in the site. In the HCI Heuristic Analysis I found a critical showstopper which was that the outside links did not open in a new browser tab. The user would have to click on the back arrow many times or click on the x and it would take them

7 completely out of the site. In the HCI Think Aloud Analysis, I found some moderate issues because most user s thought the wording was too light and the picture was too small of Joyce on the Department Advisors page. This could be a problem for people with poor eyesight. It is also frustrating to anyone when the wording is hard to read. The following screen shots show the prior pages with problems: Buttons were not consistent and needed states. This was found in the HCI COGNITIVE WALKTHROUGH.

8 Page did not open in a browser window. This was found in the HCI HEURISTIC ANALYSIS.

9 Light colored writing and picture is too small. This was found the HCI THINK ALOUD PROTOCOL ANALYSIS. The revised version can be found at: