User Testing Report. Team 1 Anita Sekharan Daniel So Jacqueline Wong

Transcription

1 Anita Sekharan Daniel So Jacqueline Wong YSDN 3005 David Gelb March 14, 2011

2 PROJECT BRIEF We are students from York/Sheridan who are developing a web app for current and prospective York University Fine Arts students. This app will allow them to showcase their work in the student gallery, as well as encourage interaction between students in various departments to network and collaborate on projects. The app takes you straight to the main gallery which comprises of work from various departments chosen at random. In addition to the main gallery, the app consists of three features: a browse section, a user account section, and a collaboration board. Within these features, students may search galleries using key words, edit their profiles, save work in their Likes section, save and publish collaboration posts, and upload work. 2

3 Anita Sekharan ANITA SEKHARAN DECIDE FRAMEWORK I will be looking at the Collaboration feature of our app. The goal of this test will be to see how the Collaborize feature can be improved and whether the current organization is working or not. As opposed to asking the tester to find a specific post in the collaboration post, this test will encourage exploration.this can also be used to see whether the collaboration feature is engaging and help us gauge whether it is something students would find interesting or actually consider using. This will give us more ideas about what more can be done with this feature. The practical concerns would be related to the technology is the user familiar with the iphone interface? Though our app is in general designed to have a standard look and feel, the usability also depends on the user s familiarity and comfort level. The equipment required is an iphone or itouch and pens and paper for recording purposes. TEST SCRIPT You are a first year visual arts student who plays the keytar. You used to be involved in your high school s marching band. You don t have the time to be involved in something as serious but would still like to find some people to jam with. You ask around to see if anyone is interested in starting up a band or getting together to make music without much luck. Your friend suggests you check out the FA4U app. ANALYSIS In general, the testing experience was interesting and provided insight into the potential of our app. I think we went into it without really realizing how much feedback we would receive. The tests took a lot less longer than expected, meaning we had to be careful and quick to note down reactions. My test focused on the collaboration section and it was helpful as we realized that the layout made users think that it was only meant for scrolling through the existing posts. After our first test, we thought about whether the scenario should have been more specific, but realized the unstructured approach was equally useful as it helped us see things we wouldn t have really noticed with a more structured approach. These were mainly 3

4 Anita Sekharan regarding how much time or interest people were willing to invest in the app. We gained ideas about how to better organize the content and make the posting and search features clearer. In revising the app, it will be important to give certain features more prominence. Though the tool also gives users the chance to post ideas they may have for collaborations, this was not made clear in the prototype. For the most part, the app was easy to navigate. Most people took the simplest route to complete tasks by scrolling through the images and browsing through different sections of the app. The most interesting and useful test was the one performed by someone who had minimal experience with the iphone interface. Scrolling, navigation and other things that we thought would come naturally to users proved to be challenging in this test. While we were worried that the major problems during user testing would arise from features that were not fully worked out, we ended up receiving more feedback for aspects of the app that we felt we had perfected. One of the major ones was horizontal scroll. It took us many trials to figure out how to properly code this, but we realized that a major issue was that of people s patience and willingness to scroll through a large number of images. In general, participants were more likely to get impatient when trying to find something specific as advised by the scenarios. If we were to redo the tests, I think it would be interesting to see what would happen if users were given a set of options to choose from. This would help us learn more about user preferences. For example, we could let the test subjects play around with the app, and then ask them to choose a task to complete based on how easy or preferable certain functions were. It would also be useful to offer some kind of introduction to the app so people are aware of its functions and capabilities. Overall, this was a useful exercise and although some of the ideas were not directly suggested by the people testing the app, we were able to generate new ideas as a result of the dialogue during the tests. 4

5 Daniel So DANIEL SO DECIDE FRAMEWORK The whole foundation of our project is based on sharing ideas and showcasing all work from the Fine Arts faculty at York. The goal of this user test is to explore the effectiveness of both the categorization of content and the navigation system of the application. Users are unlikely to use this app in a real world situation if the app is confusing and unmanageable. It is important to minimize frustration when using the app to search for specific items (its other function, which is to casually browse new items, is not this test s main concern). How easy is it to find what you re looking for? Are the pages well organized? Is there a better way to categorize work for different purposes? Are people more inclined to using the search function or to browse when looking for something specific? Are the signs and symbols easy to read (i.e. informaton button and search function)? There are two ways that the app can be approached. First, it can be used for casual browsing, and second, it can be used when searching for specific items. Since this test aims to uncover gaps in the organization, this test proposes a fairly unstructured task that encourages the user to figure out the organizational scheme to find specific content. The test subject will not be asked to use specific functions (i.e. the search tool) to reach their goal. This will help us understand which functions are high visual and functional priority (i.e. search versus organization structure). The testing environment will be within school grounds. The test is not considered to have an urgent timeline, meaning there is no real world pressure to reach the goals in a timely manner (except of course to minimize frustration). This means that the user can be anywhere, but will most likely be in a quiet environment (during study, at home, during lunch, etc.). We will need an iphone/itouch for the user and another method of recording results (recorder, notes, etc). We don t believe this test has any ethical concerns. 5

6 Daniel So TEST SCRIPT ANALYSIS You are a first year student of the YSDN program at york. You have seen the selected work in the display cases on campus and on the faculty website, but you would like to get a better idea of what everyone else in the program is up to. In particular, you have just submitted your Story Slam poster for your Design and Image class and would like to see how your peers from other sections have approached the same project. Your task is to simply find two Story Slam projects by navigating your way through the app. Find them any way you like. GO! The participants of the tests were very cooperative and openly voiced their thoughts and actions as they accomplished the tasks set before them. In this way, I believe the testing was very successful in discovering flaws that were not immediately apparent to us. During the test, I found that I had not made considerations for users completely new to the application. Some of the test subjects had to familiarize themselves with the interface at first, as some did not have a clear understanding of the applications features (i.e. its main purpose as a gallery was unclear). This informs us that perhaps a more comprehensive briefing session is necessary before testing. As well, I was unsure of whether to keep certain information from the test subject during testing. When the test subjects became confused, I was hesitant to lend them a hand in case it was the fault of the design of the application that led to their confusion. For example, at one point, one of the test subjects forgot their objective after losing her way in the application. This could have either meant the task was too difficult for a new user, or that the app was not organized well enough to minimize confusion. Overall, I think the test met most of my expectations. I believe the results will be very helpful when making considerations for future iterations. The tests revealed that the search function went largely unnoticed. When asked after the test, all test subjects stated that they did not even realize a search function existed. The main point of my test was to discern whether users would be inclined to using the search function when looking for specific items (the Story Slam projects). The search symbol needs to be placed in a much more obvious place, such as the main 6

7 Daniel So menu, where users are likely to look when choosing their options. In terms of content categorization, the division of content into departments seemed to work out nicely (for the purposes of the task, at least). However, one of the test subjects commented that she thought that the design gallery was a repeat of the main gallery (she was confused because a lot of the content between the two galleries was shared). Users may be unaware of the fact that the gallery is actually a random assortment of images and this needs to be clarified. Also, some test subjects did not scroll through the full gallery to find the required images. Perhaps we need to indicate the length of the gallery in some way. Overall, this testing process was very successful and our project will greatly benefit from the results. Designers are often so close to the project that many considerations are overlooked. Features that we thought were self-explanatory turned out to be the opposite. In future tests, I think a more indepth briefing session would benefit the test subject and results. After all, we don t want problems unrelated to the test objectives to distract us from our goals (although sometimes these incidents yield surprising and helpful results). 7

8 Jacqueline Wong JACQUELINE WONG DECIDE FRAMEWORK In my user-testing scenario, I will explore the usability of the Like feature in our app s organizational structure. The goal of this test is to find out how participants navigate their way through our app to locate a specific piece as instructed. This is important because the Like function can be subjective either for the artist s awareness of how many people like their work or for the end user to store work for their own personal reference. Questions that need to be answered through this user testing phase include: how quickly are they able to identify and locate work in the scrolling gallery? What is their thinking process for getting there? Do they notice the extra options they can use to reach the objective? The objective of my task can be fulfilled with structured and unstructured exploration, however in a structured scenario, we can also assess how well the images are displayed in terms of cropping and how recognizable they are. Practical issues that may arise during user testing depends on the environment and the participant s knowledge of mobile devices. Poor internet connection is a possibility if we will conduct the tests at York. It may interfere with the loading of some images. I don t see any ethical issues that may arise during user-testing. However, in general, since the concept of the gallery is for students to showcase their work, they are in control of what they post. Unfortunately, there may be some juvenile delinquents who may post inappropriate or irrelevant material on the app, either out of immaturity or their subjective notion of art. Ideally, there would be uploading guidelines or moderators who will be able to delete inappropriate work. 8

9 Jacqueline Wong TEST SCRIPT You and a friend are brainstorming concepts for your film project. Your friend describes her idea and you are reminded of an 8-bit video that you found the other day on York University s FA4U web app. You want to share what you found in hopes that it will help inspire and give her more ideas for her own project. Unfortunately, you forgot to save it to your Likes and have to look for it. Now, go to the website on your ipod to look for this piece. Then, to make sure you won t have to backtrack to find it again, save it to your Likes. ANALYSIS Anita and I conducted the first tests in the YSDN work room. She observed the participants, while I made notes. We began with reading the test scripts to our first participant, Sannah. We found that it was not the best way to do it because she was having problems grasping the objective. She was able to browse through the app easily and found the video she was asked to look for. She looked through the main gallery before using the browse feature to look through the film category, where she found the video. Our second participant was Dana. This time, we had our participant read the scripts herself. At first, we had some technical difficulties connecting to the internet, but Dana was also able to find the video through the film category quickly and easily. Our third participant was Kelvin. Kelvin said he had very little experience with mobile apps compared to our previous participants, so it was a different experience for him. Even though my test seemed fairly simple to the others, he browsed through the entire app before deciding where to look for the video. He was also unaware of the horizontal scrolling. Due to Daniel s absence, he conducted his own tests with his sister, Jennifer. When she was trying to find the video, she did not know what an 8-bit video was (Daniel was not sure if he 9

10 Jacqueline Wong should have explained it to her, although I think he could have since the script says user has seen the piece before.) She could not find it in the main gallery, so she went to the MyFA4U and Collab pages, realized they were the wrong places to look, tried the Browse > Film section, found it, and added it to her likes. In terms of our design, none of our participants noticed the search icon on the header. Although none of our tests required they use it, the option was not explored at all. The three main components (Browse, MyFA4U, and Collab) and their purposes were understandable for the most part, but since the header and footer moves as you scroll vertically, they easily missed the extra options on the header. During this testing period, I think an error on our part was some of the wording in our test scripts and not fully explaining the functions of our app. Some of our participants were not completely clear on some of the tasks we had set before them. But expectations were met, the participants were able to figure out what category the piece belonged to and identifed the piece as they were scrolling through the app. Overall, I thought our user testing experience was successful. I thought we had a good variety of participants from people who were very familiar with mobile apps to those who were not. It has definitely helped direct us to features that need more attention and need to be further developed as we work toward refining the design and structure of the app. 10

11 NEXT STEPS Since the tests showed that the search function went unnoticed, we have decided to exaggerate its presence by including it in the bottom menu. We hope this will encourage users to use this handy function when looking for specific content or when browsing becomes inconvenient or frustrating. Many of the test subjects were confused when the application started up with the random gallery. We realize that a landing page is necessary in order to anchor the user experience, instead of forcing heavy visual content upon them immediately. This will allow them to take in their experience and choose the path best suited to their needs. It will also allow us to expand the brand further with the inclusion of a logo. The landing page will have brief statements about the application s features, and a link to a more in-depth user guide. This way, new users may familiarize themselves with the system, while returning users are not bombarded with the heavy text of detailed guidelines. We also want to test different thumbnail sizes for the galleries. Some test subjects were slow to click on the thumbnails, and this may indicate that the images already seem quite large. We also want to consider cutting off images so that the horizontal scrolling function is more apparent to users. One test subject commented that he was not aware of the horizontal scrolling in the main gallery because they were perfectly aligned within the screen. Making the header and footer stationary on the screen will also make extra options on the header constantly visible to users. Right now, the header and footer get lost during vertical scrolling on pages such as the Collaborize board, where it is expected to have numerous posts. Users have to scroll all the way to the bottom in order to get to the footer s menu if they want to navigate elsewhere. Another complaint was that there was no organizational system on the Collaborize page. Right now, it is supposed to be organized in chronological order (most recent posts at the top, oldest at the bottom), but if there were to be many posts, they would have to scroll all the way to the bottom in order to get to the bottom menu. 11