Usability Test Plan for LVC 150 Prototype LVC 150 Mobile Web App Prepared for: Cc: Dr. Jeff Ritchie, Project Advisor Completed by: Emily Johnson Haley Dobart Tim Harris Ryan Dotson Date: March 23, 2014 Purpose: The purpose of this usability test is to determine usability and functionality problems with our current prototype for the LVC 150 mobile web app. We will be collecting data to understand what parts of our app users like and what parts we need to improve to better serve our user groups. This test is part of a long term project to develop a mobile web app to celebrate LVC s 150th anniversary, share the history of LVC, and encourage positive engagement with the college and campus. The team is interested in seeing how people will use the app we have created, namely members of four main groups: current students, alumni, community members, and prospective students. Problem Statement and Test Objectives: The results of this test will provide the team with both qualitative and quantitative data that will help them better the app. Participants will be asked to perform a number of tasks relating to the navigation, functionality, and content of the mobile web app. Our objectives are as follows: Usability How usable is the app? Do users run into any problems? How do they attempt to solve them? How can our design better afford forgiveness and avoid problems in the first place? Functionality Is the app functional? Do users feel that the sequence of screens makes sense? How would they use the app in real life? Design Is the app simple enough for a variety of users to be successful in using it? Could the design be simplified or changed to better afford the desired behaviors? Language Is the language used in the app appropriate? Is it clear? Does it provide enough instruction and guidance without being cluttered or distracting?
Satisfaction Do users enjoy using the app? Do they see themselves using the app in real life? How often? Under what circumstances? Participants will complete a pre test questionnaire, complete the test, and then take a post test questionnaires. These surveys will give the team insight as to the participants past experiences and their feelings and impressions toward the LVC 150 app. The test will be held in the Digital Communications studio. The tests will take between 10 and 15 minutes, including pre and post test questionnaires. Participants will be videotaped and notes will be taken about their performance in the test. User Profile: In order to accurately assess our current prototype for the LVC 150 mobile web app, we identified four target groups prospective students, current students, alumni, and community members. However, the main focus will be on two main primary audiences: prospective students and alumni of Lebanon Valley College. We then created descriptions and personas for the user group. The characteristics of the user group, in addition to questions about prior experience and behavior, will be a part of the screening questionnaire for identifying suitable test participants. Methodology and Tasks/Scenarios: This section of the test plan explains the testing methods and scenarios that will be used with five participants in 15 minute sessions. Number of Participants: We plan to test a total of five participants. In order to account for external circumstances, we will recruit seven people, so that we have two backup participants. Length of Sessions: The total length of each session will be 15 minutes, broken down as follows: Welcome/pre test questionnaire: 3 minutes Preform Task scenarios: 10 minutes Post test questionnaire: 2 minutes Test Procedure: The test will start with an introduction and pre test questionnaire, followed by task scenarios, and a post test questionnaire.
Guided by a moderator, the test participants will be notified if any actions taken during the course of the usability test are not permitted. It is important to note that the moderator should remain unbiased during the course of the test and should not try to guide or mislead the test participant. Once the participant has completed all areas of the test, including the pre and post test assessments, the moderator will inform the test taker that the test has been completed. Intro/Pre test Questionnaire: The participant will be asked to sign the video consent form. Then the moderator will explain the process and ask the participant to think out loud. The participant will then complete the pre test questionnaire. (Appendix) Task 1: Using your mobile device, please navigate through the LVC 150 mobile site. You are at Lebanon Valley College and would like to find out information based on your location. How do you go about finding out this information based on your location? Click on Shroyer Health Center. What story is at this location? Select this story and navigate through it. Please return to the map to see what other stories are available. Task 2: You are not on Lebanon Valley College s campus but want to explore the LVC 150 mobile site. How do you go about finding all of the stories? Now navigate to your favorite story in order to be able to find out information about it.
Closing/Post test Questionnaire: The moderator will ask the participant to fill out the post test questionnaire. They can also write any suggestions they may have to make the site better overall. (Appendix)
Usability Test Script Testing Permission: I hereby give my permission to be videotaped and a test subject as part of my participation in the LVC150 usability test conducted on in the Digital Communications Studio. I also state that by signing this document that I am 18 years of age or older. Only my first name may be reported in association with the session results. I understand and consent to the use and release of the video recording to the Digital Communications Students and the client. I further understand that the video recording and any highlights extracted from it may also be used for review by the client, at professional meetings, and as part of research. I understand the following: 1. My participation is voluntary 2. I have the right to withdraw without penalty 3. I am being protected from known risks and risks are being minimized where possible. 4. My confidentiality is ensured by clearly stating the means by which the data is collected. I give up any rights to the video recording and understand that the recording may be used for the purposes described in this release form without further permission. I understand that if for any reason I do not want to continue I can leave at any time during this recording session. I can deny consent at any time. Printed Name Signature Date
Introduction to the Team: Our team is made up of four digicom students Emily Johnson, Ryan Dotson, Tim Harris, and Haley Dobart. This test is being performed as part of an assignment for our Storytelling for Digital Media. Introduction to the Test: Welcome to our usability test, thank you for volunteering to be a participant. Today we will be testing our design for the LVC 150 mobile web app. Pre test Questionnaire: Your feedback today will be invaluable in our study of the mobile site. Before we start, please fill out this brief questionnaire. This will help us to better understand your background in using the Internet and related technology. First Name (please print): 2. What is your age? 3. What is your gender? 4. What is your relationship to LVC? 4b. If student, what is your major? If alum, what year? 4c. Would you be interested in learning about the history of LVC through a mobile app? If so, why? If not, why not? 5. On a scale of 1 5, how comfortable are you using a mobile site? 1 Very comfortable 2 Somewhat comfortable 3 Neutral 4 Somewhat uncomfortable 5 Very uncomfortable 6. How many hours a week do you use the Internet? (circle one) 0 5 hours 6 10 hours 11 15 hours 16 or more hours 7. How many hours a week do you use the Internet on your phone? (circle one) 0 5 hours 6 10 hours 11 15 hours 16 or more hours During Testing:
We have two scenarios for you to work through today. Remember, we are not testing you. We are testing the mobile site in order to make it better for all users. Please think out loud as you complete these scenarios, explaining what you are doing and why. When you are done with a task, please say so, and we will give you a brief questionnaire. Then we will begin the next scenario. [If participant is not thinking aloud:] Please try your best to think aloud as you complete the task/questionnaire. Your thinking aloud helps our group understand what you are doing and why. Task 1: Using your mobile device, please navigate through the LVC 150 mobile site. You are at Lebanon Valley College and would like to find out information based on your location. How do you go about finding out this information based on your location? Click on Shroyer Health Center. What story is at this location? Select this story and navigate through it. Please return to the map to see what other stories are available. Post test Questionnaire: Please rate the following: Very Easy Some what Easy Neither Easy nor Difficult Somewhat Difficult Very Difficult Understanding Navigation on the on campus site Mobile site terminology Other Notes regarding the task:
Task 2 : You are not on Lebanon Valley College s campus but want to explore the LVC 150 mobile site. How do you go about finding all of the stories? Now navigate to your favorite story in order to be able to find out information about it. Post test Questionnaire: Please rate the following: Very Easy Easy Neither Easy nor Difficult Somewhat Difficult Very Difficult Understanding Navigation on the off campus site Mobile site terminology Other Notes regarding the task: After Testing: Thank you for your feedback and participation today! It has been extremely helpful in giving us a better idea of how people use the LVC 150 mobile site. If you have any suggestions to make this mobile site better please share them here.