White Paper. User Testing. By Adrian Westlake UX Design Team Lead Intelligent Environments

Transcription

1 White Paper User Testing By Adrian Westlake UX Design Team Lead Intelligent Environments

2 Introduction All products should be tested by the people that will use them, regardless of whether that audience is your clients customers or the general public. You can be the best designer in the world, or think you know your users inside out, but you still won t know how your users will behave until you observe them using your product. User testing is the process of evaluating how your product performs against users. This enables you to spot problems throughout the development process. There are several types of user testing you can do at various stages of development. The three discussed in this paper are Guerilla testing, Remote testing, and Expert Reviews. This paper will detail each of these types of user testing and analyse their effectiveness. User testing doesn t have to be expensive. While you can spend a lot of money on large laboratory facilities and usability companies, to test your product and get a lot of valuable insight, usability problems can also be uncovered by a much more informal environment, using cost-effective and simple techniques. You re almost always wrong about your users Manik Rathee User Experience Engineer, Google The selection of users is important to get right as it gives you a more valuable output. The users should be closely matched to your real target audience. Use any data that you have on your users to determine this. Also consider those that you wish to aim the product towards in future. Think beyond simple metrics like age and gender, but also factors like technical competence, familiarity with certain software, or hobbies. Sometimes you may want a more specific target audience of testers depending on what you want to test. Other times you will have a much broader range of testers. Testing of early versions of release Brief Design Development & QA Release Testing on mockups / prototypes Testing on fi nal release User testing should happen at every stage of development 1

3 Guerilla testing Guerilla testing is the cheapest and easiest style of testing to set up. The word Guerilla in this context explains the in the wild style of testing meaning that it can be held anywhere. Common places for this type of testing are coffee shops, offices and university campuses. Participants are normally the general public, colleagues, or friends and family. You normally only need a laptop or device, and a pre-prepared list of things you want to test. It usually takes from 5 15 mins to quickly gauge opinion on a particular aspect of your product. You may also choose the record the sessions, with permission from the person you are testing. This form of testing is quick and inexpensive. The result can be a list of common usability problems that you can immediately iterate the design with, and that suits an agile development process. It s advisable that someone other than the designer carries out this testing, to reduce any confirmation bias and give a fresh perspective. 2

4 Remote / Crowdsourced testing In order to get a broader spectrum of participants you might consider testing people from a wide geographical area, and carry out testing remotely via software such as Skype. The easiest way to carry this out is through crowdsourcing methods. Crowdsourcing in this context uses a large community of users to test your product. There are many services which provide such a solution, for example usertesting.com, and UsabilityHub. These companies have a wide network of testers. You can simply upload a design or prototype and get fast feedback, usually in the form of a heatmap, or recording of a test session. A global community means you have a much more diverse set of testers to choose from. You can usually narrow down your audience by specifying simple metrics such as age, location, and language. Some services will also allow you to define more specific requirements. As well as having a broad reach, remote testing through a crowdsource provider is relatively cheap compared with outsourcing the testing. It s also easy and quick to set up and results normally start to appear within a few hours. You will need to leave some time to analyse the results and identify the common usability problems, so that you can iterate your design. Click tests can highlight a design s effectiveness 3

5 Case Study: Account improvements In Interact 5.10 we made some layout changes to the way we show accounts and transactions. We utilised crowdsourcing to regularly test and iterate our designs until we had something we were comfortable with and were confident would work well for our customers. One of the changes we made was to move the main contextual actions for an account over to a more central part of the page, because we had found in previous testing that users suffer from banner blindness (meaning that anything in the sidebar is considered secondary content, and often advertising). Moving it to this primary area also had the benefit that it would be more visible and be displayed in a more logical order on smaller screens and screen readers. In order to test this we carried out two click tests. A click test is a static screen that can be shown to a group of users (in this case crowdsourced), who are then asked to click on a certain area. The question is most often framed Where would you click to. In this case we asked our users to click on the place to make a transfer. Doing this across several variations with different groups of users allowed us to have a comparison. While we didn t see a huge difference in accuracy, we did see a big improvement in time. It took users an average of 29 seconds on the old design, whereas it took 18 seconds on the new design. Another series of tests we did was around transactions. We knew that tables proved difficult on smaller screens, and there is a trend towards using cards to present data (cards are much more mobile friendly and don t mean losing any data). We tried several iterations of these cards using a mixture of icons and labels to highlight incoming and outgoing transactions. A simple preference test, asking users to choose the one they thought more understandable, gave us some guidance. A preference test gives the same group of users several options to choose from, with a question to guide them e.g Which is more understandable?. The order of the options is randomised so as to not bias the results. Along with some additional click testing and guerrilla tests we had a fairly good indication what worked best for our users. A preference test gives you an idea of a user s choice 4

6 Expert testing There are a variety of third parties who provide user testing as a service. The usual process includes: An evaluation of your current product in order to generate test cases Recruitment of test participants using a defined set of criteria Laboratory based testing with a moderator, and a set of observers in another room watching the testing (usually via monitors or one-way glass) Analysis and reporting of the results by way of a report, presentation or workshop. Major problems are discussed and solutions advised. Using outside experts to analyse your site gives you advantage of an unbiased expert review and solid recommendations. It is, however, the more expensive of the user testing options, and is best done on a near finished product in the final stages of development. Observing people using your product gives you an invaluable insight into what users are thinking when they use it. 5

7 Summary User testing is vital if you want to understand whether your design enables the best use of your product by your target audience. Without it you are designing blind and will only find out when it s too late. Testing at regular intervals throughout the process will ensure that you reduce the risk of usability problems being present in your final product and will give you the most customer insight. Use a combination of remote and guerrilla testing at early stages in order to iterate design quickly and not go down the wrong path. Once you have a near complete product you should also think about some third party expert testing to fully analyse your product as a whole and test it with your defined audience. User testing method Advantages Disadvantages Guerilla testing Cost effective Can observe first hand Once initially setup, quick turnaround per participant Hard to control target audience Narrow geographical area of participants Potential for confirmation bias * Crowdsourced testing Expert testing Quick to set up Cost effective Wide target audience and geographical area Expert analysis and feedback Little self-effort Can observe first hand Can t observe first hand Non disclosure agreements harder to obtain for confidentiality Limited communication with participants Expensive Delay in feedback Long lead-in time to setup * Confirmation bias is the tendency of a person to favour information that confirms their preconceptions. 6

8 About the author Adrian Westlake, UX Design Team Lead Intelligent Environments Adrian has driven the user experience for Intelligent Environments and their clients since He has a deep passion for making technology more accessible to everyone through intuitive user experiences, and has previously worked on UI projects for leading digital marketing companies, with clients including work for MyVoucherCodes.co.uk, Panasonic, Coca Cola, and Unilever. Adrian leads the Design Studio and manages the user testing process at Intelligent Environments, owning the overall user experience for Interact and working with their clients to ensure that experience is best customised for their users. About Intelligent Environments Intelligent Environments is an international provider of innovative mobile and online solutions for financial services providers. Our mission is to enable our clients to always stay close to their own customers. We do this through Interact, our single software platform, which enables secure customer acquisition, engagement, transactions and servicing across any mobile and online channel and device. Today these are predominantly focused on smartphones, PCs and tablets. However Interact will support other devices, if and when they become mainstream. We provide a more viable option to internally developed technology, enabling our clients with a fast route to market whilst providing the expertise to manage the complexity of multiple channels, devices and operating systems. Interact is a continuously evolving technology that ensures our clients keep pace with the fast moving digital landscape. We are immensely proud of our achievements, in relation to our innovation, our thought leadership, our industrywide recognition, our demonstrable product differentiation, the diversity of our client base, and the calibre of our partners. For many years we have been the digital heart of a diverse range of financial services providers including Atom Bank, Generali Wealth Management, HRG, Ikano Retail Finance, Lloyds Banking Group and Think Money Group. For all your user testing requirements, speak to one our team by calling us on 44 (0) or visit our website at 7