Toyota Prius Hybrid Display System Usability Test Report Sample Test Script Author Shobana Subramanian
Usability Test Script/ Protocol for Novice User 2 Participant tasks Observation & Checklist Initial Setup Hello! Thanks for participating in our research. My name is and I will be guiding you through the process. You will be interacting with a car driving simulator that mimics the Prius Hybrid display system. Please note that we are not assessing your performance we are just interested in how you interact with the system. We will first start with completing the consent form. It essentially says you are voluntarily agreeing to participate in our research and you will not disclose information from this research to anyone. Consent form completed correctly? When forms are completed, participant taken to the lab room with simulator This is a driving simulator. There are three screens meant to give you a full view of the traffic and road to mimic you sitting in the driving seat in a regular car. Please take a seat on the driving seat. I want you to make yourself comfortable on this seat as if you were in a real car. Check to see if the traffic scenario s first screen is on.
3 Please give me a moment while I adjust the instruments. Adjust eye tracker so the participant s eyes are clearly within range, and press Start on software. Make sure cameras light are on. Once participant is situated, begin tasks FREE-FORM Please turn on the engine by pressing the start button near the gears [Researcher will physically point to the button]. Now, please spend some time doing whatever you want with this product. You may drive the simulator and explore any of the display s screens. Once the time is up, I will come back and get you started on the next task. Does the P (participant) look at the manual on the passenger seat: If so, how long do they read the manual: Under 2 minutes Longer What screens did they look at first? What screens did they look at most? Do they make any comments about the manual or the car as they begin to interact with it? 1. If so, List of interface components they make any comments about and nature of comment: What controls do they play around with? --Before starting the car? --While driving? Do they drive the car?
4 Do they appear to interact with the interface? 1. Do they lean forward? 2. Where does their gaze appear to be (interface, road, side view mirrors?) ---In the first few minutes? --Five minutes later? Do they make any comments while driving? If so, please note. GOAL-BASED The researcher enters the lab room again after 10 minutes The next task will involve you driving from point A to point B. Please use your own driving judgment to complete the task. Here is the route (handed to the participant, and read aloud twice by researcher): You will be parked outside Martin Luther King Library downtown on 4 th street. Please use the 4 th street to go to 280 N. Please head to Palo Alto using 280, 85 and 101 going north. Exit at Oregon Expressway on 101 and turn right at the exit. Stay on Oregon Expressway. The simulator will stop at some point there. Read the instructions out loud twice. Leave written instructions on small table near driving seat. As you drive, you will asked some questions
for which you have to provide a verbal answer 5 Confidence level using the car immediately? High Medium Low What verbal behavior is observed, if any, throughout trip. [SA Probes in regular intervals] What is your current MPG? Is the car currently running on gas or electricity? Any general patterns observed during course of trip? First MPG: Correct Gas or Electric: Corrent Incorrect Incorrect How fast are you currently going? Speed: Correct Incorrect How much energy is left on the battery? What is your current MPG? Energy Left: Correct Second MPG: Correct Incorrect Incorrect During the SA probes, Are the interface controls adjusted? If so, which ones? Posture in general? --Mostly Straight on seat --Often Leaning forward --Posture kept changing Any strong emotional reactions? --Anxiety? --Excitement? --Other?
6 TASK-BASED Researcher returns after 20 minutes when traffic stimulator stops Researcher sits on the small table near the driving car and poses the following questions: I will now be asking you questions about the interface itself. I want to know what you think about the car. I will be pointing out to some features and please let me briefly what you think it does, and how it operates: Researcher points to Fuel Consumption History Display: Do you know what information this display shows? If yes, follow up: Can you tell me, from looking at the display, the different interval levels for which information is available? Answer: Average fuel consumption (and regenerative energy-optional answer) Correct Ans.? Answer: 1 minute and 5 minute intervals Correct Ans.? Researcher Points to Energy Monitor, first to the hybrid battery icon and asks: Do you know what this is for? Answer: Usage and regeneration of electrical energy Correct Ans.? Researcher then points to the fuel/gasoline engine icon on the Energy Monitor: Answer: Gasoline engine operation status Correct Ans.? Do you know what this is for? Did you notice the arrows? Do you know what the arrows are for? If Yes, can you tell me what how they Answer: Energy flow, has different colors,
work when you drive? moves in different directions depending on acceleration or deceleration, about how car is powered whether by gasoline or electric motor or hybrid battery 7 How would you change settings for the display? Basic Answer: Press the DISP button on the steering wheel. Correct Ans.? Was a more complex answer provided? Notes: Researcher points to Hybrid System indicator: Do you know what it means when the indicator is only highlighted in the area marked CHG? Answer: CHG=Charge. Energy being charged by regenerative braking Correct Ans.? Again on the Hybrid System Indicator: Do you kno what it means when the indicator is highlighted all the way to the area marked PWR? Answer: Environmentally friendly driving range exceeded. Correct Ans.? Did you notice the ECO light on the indicator? If Yes, what does it mean? Can you give me these readings? Odometer Answer: Driving is highly fuel efficient. Correct Ans.? Readings Correct? Trip meter
8 Average mpg Average mph Distance to empty OPEN-ENDED Our tasks are now complete. I would now like to get your feedback about your experiences driving this car. The questions are on the screen, which I will now read aloud to you. Please provide verbal responses. Make sure the computer screen switches into survey question mode, and double check camera lights are on. Read aloud the information from the screen. What did you think of the hybrid display? Did you understand how it worked? Of the screen options you saw on the display, which one was the easiest to read? Why? Of the screen options you saw on the display, which one would you least use? Why? Was there anything in particular you strongly did not like?
On a scale of 1-5, please rate your concern for the car s fuel consumption while driving the car. 1 2 3 4 5 Very little High 1 2 3 4 5 9 On a scale of 1-5 please rate how distracting was the hybrid display. 1 2 3 4 5 Not Very Distracting Distracting 1 2 3 4 5 On a scale of 1-5, how confident are you in driving this car after this experience? 1 2 3 4 5 Low confidence High 1 2 3 4 5 On a scale of 1-5, can you rate how applealing the interface was for you? 1 2 3 4 5 Unappealing Very Appealing 1 2 3 4 5 On a scale of 1-5, can you rate whether you thought it was fun to drive the car 1 2 3 4 5 No Fun Great Fun 1 2 3 4 5 If you could change anything, what would it be? If the participants had strong emotional reactions, ask why?
What did you think of the Tracer display? 10 Would the hybrid display system affect your decision to purchase a Prius? In what ways? Any final thoughts?