Usability Report: Papa John s Pizza Web Site. Lisa Bradshaw June 24, 2013

Transcription

1 Usability Report: Papa John s Pizza Web Site Lisa Bradshaw June 24,

2 Table of Contents Executive Summary... 3 Methodology... 4 Questions from the Screener... 4 Participant Data and Links to Usability Session Videos... 4 Usability Test Script and Tasks: Ordering Pizza Online... 5 Final Task Questions... 7 Task Analysis... 7 Quantitative Data... 8 Qualitative Information Participant Participant Participant Participant Participant Recommendations

3 Executive Summary The expectations of adding the three requested pizzas to the cart were met; all five participants were eventually able to successfully complete the task. However, the participants experienced some problems while completing the task. The most common problems were finding the specialty pizzas after ordering a custom built pizza, and adding the mushroom topping to a pizza. Four of the five users had difficulty finding the specialty pizzas after ordering the first two (custom built) pizzas. Three of the users tried to go to the Create Your Own pizza screen to select a specialty pizza, and then failed to find a link back to Order Now, or to the specialty pizza page. Comments included: Well, I clicked Start Over three times, and nothing s happening, so at the bottom where it says Add to Order I m going to click Full Menu. I probably shouldn t have done that I don t see it. (laughing) There s no cancel or anything, not what I was expecting... Start Over didn t do it I guess I would probably prefer it being a little more noticeable. One user decided to build her own Hawaiian pizza when she failed to see the link to More Pizzas: I guess I can make a Hawaiian. When trying to locate the mushroom topping, users did not expect to look for Baby Portabello, although they all eventually determined that it was the only mushroom selection. Comments included: I don t see mushrooms Baby Portabellos. I guess I ll choose that. It s not what I expected would happen but I guess it s fine. Where are mushrooms? Oh, Baby Portabello. That s the only mushroom option I see, so I assume that s the kind I m supposed to order. I can t find mushrooms. It doesn t seem like there are any. Oh, there they are. One user had trouble getting started ordering anything; she studied the screen and scrolled up and down for 30 seconds before deciding to click a link to begin ordering. Another user, who did not have a delay or problems ordering, commented: It seemed a little overwhelming at first; it seemed kind of busy, there s a lot going on. 3

4 Methodology In this usability study, five users were observed interacting with the Papa John s web site, and adding three pizzas to their cart with varying topping options. A list of screener questions were developed to select the users, and a standard script with a list of tasks for all users was used in the usability sessions. The users actions on screen were recorded, and four of the five users facial reactions were recorded. Finally, the usability sessions were analyzed for quantitative data and qualitative information, the results were compiled, and recommendations made for improvement of the ordering system. Questions from the Screener 1. How often do you use the Internet? 2. How often do you order pizza? 3. How often would you say your orders were made successfully? 4. How do you usually place your pizza order? 5. How often do you order food online? 6. Where do you usually get your pizza delivered? 7. How many people do you usually order pizza for? 8. What are the ages and genders of the people you re usually ordering pizza for? 9. Do you or anyone you usually order for have any dietary restrictions or special ordering needs that might not be listed on the menu? 10. Do you use coupons or special promotions when ordering pizza? Participant Data and Links to Usability Session Videos User Age Gender Computer Skill Level Link to Video 1 45 F Intermediate F Intermediate M Intermediate to Advanced M Intermediate M Advanced 4

5 Usability Test Script and Tasks: Ordering Pizza Online Use this task sheet to take notes as you observe the study (no need to take detailed notes, you will be able to review the study after it is recorded) Make sure to review this tasks script before you start so you are familiar with the content. Hi,. My name is, and I m going to be walking you through this session today. Before we begin, I have some information for you, and I m going to read it to make sure that I cover everything. You probably already have a good idea of why we asked you here, but let me go over it again briefly. We re asking people to try using a Web site that we re working on so we can see whether it works as intended. The session should take about an hour. The first thing I want to make clear right away is that we re testing the site, not you. You can t do anything wrong here. In fact, this is probably the one place today where you don t have to worry about making mistakes. As you use the site, I m going to ask you as much as possible to try to think out loud: to say what you re looking at, what you re trying to do, and what you re thinking. This will be a big help to us. Also, please don t worry that you re going to hurt our feelings. We re doing this to improve the site, so we need to hear your honest reactions. If you have any questions as we go along, just ask them. I may not be able to answer them right away, since we re interested in how people do when they don t have someone sitting next to them to help. But if you still have any questions when we re done I ll try to answer them then. And if you need to take a break at any point, just let me know. You may have noticed the microphone. With your permission, we re going to record what happens on the screen and our conversation. The recording will only be used to help us figure out how to improve the site, and it won t be seen by anyone except the people working on this project. And it helps me, because I don t have to take as many notes. Pre Task Questions Before we get started I have a couple of questions: Have you ever ordered pizza online? If not: why haven t you ordered online? If so: was it for delivery or pick up? Why did you pick it up or why did you have it delivered? 5

6 What did you order? How would you describe your experience? Ice Breaker We are ready to get started so go ahead and launch the web browser and go to Google and search for pizza in your zip code. Tell me about the experience of using Google and browsing the results. Are you able to find what you need? Is there a pizza place close by? Is it the type of pizza you like? Task 1 We are going to be looking at papajohns.com web site. Even if you are not a Papa John s fan, imagine that the people you are with are fans of Papa John s and that is where you will be ordering the pizza. There are bunch of people at this party and you need to order 3 pizzas. Mushroom and Pepperoni Half onion and half sausage with light sauce A specialty pizza (you want some variety) Tell me a little more about your experience ordering pizza from this site. Is it what you expected? Task 2 - You want to sign up for deals and coupons but you don t want to register. You just want to give them your . Can you do this and what do you think you will receive by ? Task 3 Your pizza arrives and it is terrible! Your driver was rude and you are really upset about what just happened. You call the local store but get nowhere. You need to contact the corporate office, how would you do that? 6

7 Final Task Questions 1. Explain how you would go about placing an order for a pizza online. 2. Find the special pizza offers and place an order. 3. Order a large pizza with thick crust, half with peppers and onions and the other half with sausage and mushrooms. 4. Order a supreme pizza with extra sauce and without the olives. 5. You're having a party and need to order 10 pepperoni pizzas and 10 cheese pizzas with square slices. You also need to order 5 servings of bread sticks, extra sauce and 5 two liters of any kind of pop for your guests. Please try to place your order for these items. 6. You accidentally added an extra pizza to your order. Please remove it from your cart. 7. You have a coupon that you would like to apply to your order. Please try to redeem your coupon using the code "PIZZA." 8. You would like to pay by credit card, and you want to add a tip of $3.00 to the total amount. 9. You created an order for delivery, but you decided that you would like to pick it up instead. Please change the order to carry out. 10. Please submit your order using the credit card provided. Please indicate how you know your order was successfully completed. Task Analysis For this usability study I analyzed the task of selecting a pizza, choosing toppings, crust and sauce options, and placing the pizza into the cart. In the usability sessions, the participants were asked to order three different pizzas: Mushroom and pepperoni. Half onion and half sausage with light sauce. A specialty pizza. The users were then asked to comment about their experience ordering pizza from this site, and if it was what they expected. 7

8 Quantitative Data The table below shows the tasks that the participants were asked to perform, and their errors in accomplishing the task. The errors are rated by the users level of difficulty on a scale of 1-5: 1. Very difficult 2. Difficult 3. Moderately difficult 4. Slightly difficult 5. Not difficult Time on Task User 1 User 2 User 3 User 4 User 5 Order three pizzas 5:19 8:10 5:02 5:17 2:53 User User User User User Types of Errors and Level of Difficulty Could not locate specialty pizzas Could not locate mushroom topping Could not locate screen to create a pizza

9 Qualitative Information Participant 1 Age: 45 Gender: F Computer skill level: Intermediate The user looked at the Order Now screen, scrolling up and down, for about 30 seconds before trying to click on an item to try to order it. The user had difficulty finding mushrooms until she finally saw Baby Portabello and realized that was the only mushroom choice. I don t see mushrooms Baby Portabellos. I guess I ll choose that. The Add & Customize screen showing Baby Portabellos as the only mushroom choice. It s taking a little longer than I would think it would Just finding everything that you want Just the word Add to Order; does that mean that s what I want it sounds like I m adding something else rather than ordering the pizza. Because I don t know what to do, I m going to click Add to Order. 9

10 The user was not sure what Add to Order would do. She was unsure if that would add the pizza to her order. The Add & Customize screen showing the Add to Order button. Once she clicked the Add to Order button, she saw that the pizza was in her cart. The user was not sure if the specialty pizza was under Special Offers or Order Now. I actually feel that I want to go under Special Offers just to see what else they have available. She looked at both pages to see that the item she was interested in was available on both. The Special Offers screen showing the Double Bacon Six Cheese Pizza. 10

11 The Order Now screen showing the Double Bacon Six Cheese Pizza. When asked if the experience was what she expected, she said that it s different; You can customize it simpler than the other web site that I ve used before. She said it was very easy; easier than the other site, once she got used to it (once she had ordered the first one). 11

12 Participant 2 Age: 35 Gender: F Computer skill level: Intermediate The user didn t seem to have much trouble finding the Add & Customize button to order the first pizza. When she clicked the Add & Customize button under Build Your Masterpiece, an error screen indicated a problem with the browser plugin which the tester had to fix for her. (This time has been deducted from her total time.) The user easily found how to add toppings to each half of the pizza. She found the Baby Portabello button for mushrooms very quickly. She seemed to be a bit perplexed about this button, though, and said It s not what I expected would happen but I guess it s fine. The Add & Customize screen showing Baby Portabellos as the only mushroom choice. The user said when adding toppings, Wow, this is kind of awesome. She completed the first two orders fairly quickly and did not seem to have much difficulty finding the ordering options. The user wanted to order a Hawaiian pizza and clicked the Add & Customize button but then decided that was the wrong place, and tried to leave the page by clicking on the Order Now link, then got a Confirm Navigation alert asking her if she really wanted to leave the page, which she accepted. 12

13 The Confirm Navigation window after clicking Order Now on the Add & Customize screen. She then scrolled up and down a bit looking for where she could order the Hawaiian. She didn t see that as an option, and didn t click the More Pizzas button to see the full list of Specialty Pizzas. On that screen, there is a Hawaiian BBQ Chicken pizza. The Order Now screen showing the initial list of pizza selections, and the More Pizzas link. 13

14 The Order Now screen showing the Hawaiian BBQ Chicken pizza after clicking the More Pizzas link. After all that, and not finding the More Pizzas option, she said I guess I can make a Hawaiian, and selected the Add & Customize button after all to order it. For the most part I liked it, I liked the levels of customability. I found it harder to find the normal specialty pizzas, so I guess that was disappointing. I would have liked to be able to say search for a specialty pizza. User said that when she was asked to confirm leaving the page I would expect it to leave me on that page but allow me to add more. There were options lacking on the Add & Customize page that she wanted to see (Specialty Pizzas). 14

15 Participant 3 Age: 30 Gender: M Computer skill level: Intermediate to Advanced The user reasoned that since multiple toppings are needed for the first pizza, he would go to Create Your Own which actually is the Add & Customize button. The user had a little trouble finding mushrooms, but otherwise easily added his toppings. Where are mushrooms? Oh, Baby Portabello. The Add & Customize screen showing Baby Portabellos as the only mushroom choice. He didn t seem to have trouble finding the other ordering options for the first two pizzas. For the specialty pizza, he started ordering using Add & Customize, then changed his mind and clicked Start Over several times. In regards to his confusion about finding the Specialty Pizzas, he said I think that the specialties should be on the same exact page. The page that he was on; however, was the selection page for the Add & Customize, or build your own pizza option. He wished there was a link directly to the Specialty Pizzas from this page. He said Well, I clicked Start Over three times, and nothing s happening, so at the bottom where it says Add to Order I m going to click Full Menu. He was then asked to confirm leaving the Add & Customize page, which he did. 15

16 The Confirm Navigation window after clicking Full Menu on the Add & Customize screen. He then saw that his first two pizzas were in the cart, and found the Specialty Pizzas section on the next screen, and had no problem adding one to his cart and seeing that it was there. At that point he was ready to check out. He said that the experience was about the same as ordering from Pizza Hut online, but found some aspects confusing. In the cart, he wished that the toppings for the pizzas ordered using Add & Customize would show the toppings so he could confirm that he had selected the right ones. He did see the Details link but would prefer that the cart would show him without using that link. It s a bit confusing it should list the toppings of the other two pizzas I m ordering just so I know that I m not making a mistake. It has so that you can check the Details but I think that s highly unnecessary, if it would just list exactly what you ordered. The Cart showing the menu selections but not all the toppings. 16

17 Participant 4 Age: 25 Gender: M Computer skill level: Intermediate The user looked at the ordering options and then decided to create his own pizza using the Create Your Own button. He did not hesitate to click the Baby Portabellos button to select mushrooms. He said That s the only mushroom option I see, so I assume that s the kind I m supposed to order. The Add & Customize screen showing Baby Portabellos as the only mushroom choice. The user seemed confused about the picture of the pizza after adding two toppings. He said It does look like they put cheese on top of the toppings. I don t like that; I like my pepperoni to be on the top because I like it crispy. I don t know if there s a way I can change that. 17

18 The Add & Customize screen showing the image of the pizza after adding Baby Portabellos. He then clicked Add to Order to try to change the order of the cheese, which instead added that pizza as it was and took him back to the Order Now screen. He said Well, I guess I ll deal with that at the end; maybe it allows me to do that. When asked to select sausage, he noticed the option for Spicy Italian Sausage (as opposed to the Sausage) option and chose that instead. The Add & Customize screen showing Sausage and Spicy Italian Sausage options. 18

19 He had no problem ordering the second pizza. When ordering the specialty pizza, he had already clicked on Create Your Own before checking the type of pizza he needed to order. When he didn t see any specialty pizzas, he said I probably shouldn t have done that. The Add & Customize screen showing no link to the Specialty Pizzas. He clicked the browser s Back button and received pop-up screen that said Are you sure? asking if he really wanted to leave that page. He opted to stay on the page, and then tried clicking Start Over, which didn t change the screen. He had difficulty figuring out how to find the specialty pizzas; I don t see it. (laughing) He then accidentally clicked the Proceed to Checkout button and received a window asking him to Sign In, which he closed. The Sign In screen after clicking Proceed to Checkout. 19

20 He kept scrolling up and down trying to find the way to go back. There s no cancel or anything, not what I was expecting... Start Over didn t do it. He finally clicked Full Menu and accepted the screen asking Are you sure? to navigate away from that page. Once back on the Order Now page, he checked to make sure the pizzas he had already ordered were still in his cart. On this page he saw the Specialty Pizzas and selected one, then checked to make sure it had all the toppings he wanted, then clicked Add to Order. At this point he was ready to check out. In regard to Special Offers, he said I guess I would probably prefer it being a little more noticeable. He said that he would have liked to see special deals on his three pizzas since this order was more than he is used to spending on a pizza order. He also said that he would have wanted a way to add a note for special instructions. He tried clicking the Modify button but that did not give him a way to add instructions. 20

21 Participant 5 Age: 30 Gender: M Computer skill level: Advanced When asked where he would go next to order, he said It seems a bit confusing; there s a lot of stuff. He scrolled up and down for about 20 seconds and then clicked the Add & Customize button. When trying to order the first pizza, he said, I can t find mushrooms. It doesn t seem like there are any. Oh, there they are. He then chose the Baby Portabello option. The Add & Customize screen showing Baby Portabellos as the only mushroom choice. He had no problem clicking Add to Order to order the first pizza, and selecting toppings for the second pizza and adding it to his order. He had no problem quickly finding a specialty pizza on the Order Now screen and adding it to his order. When asked if he liked the way the site was set up to add toppings, he said, It seems pretty interactive. I don t know how that would work in a mobile aspect. Regarding the interface and ordering options, he said It seemed a little overwhelming at first, it seemed kind of busy, there s a lot going on as far as the Order Now, but they seem to have it clustered into smaller categories so once you find it, it s not too hard, but at first it s kind of congested, it seems like. Despite his comments that the site was initially overwhelming, he had no problem quickly locating his options and placing his order. 21

22 Recommendations Since four of the five users had some level of difficulty locating the specialty pizzas from the Create Your Own screen, I recommend putting a prominent link to the Specialty Pizzas from that screen, both at the top and the bottom. If a user has already landed on the Create Your Own screen but has not made a selection yet, clicking on another link should not generate the Confirm Navigation pop-up screen. Since two of the users were confused by the Start Over button on the Create Your Own screen, I recommend changing it to say Clear Selection. Also, I recommend making the More Pizzas link more prominent on the Order Now screen, since one user failed to see it. Five of the users had some level of difficulty locating the Baby Portabello link when looking for mushrooms. I recommend adding the word Mushrooms to the link to select this topping. Since one user had trouble getting started, and even the user who completed fastest with the least difficulty mentioned that the Order Now screen was overwhelming, I recommend adding a menu across the top, near Order Now, with text links to Create Your Own, Specialty Pizzas, Sides, Drinks, and Desserts. I recommend an additional set of usability sessions after these problems have been corrected, to see if these areas are easier to navigate, and to discover if there are additional problems with adding a pizza and selecting toppings. 22