UX analytics to make a good app awesome

Transcription

1 UX analytics to make a good app awesome White Paper Copyright 2014 UXprobe bvba

2 Table of contents Executive summary Building a good app is a challenge How is UX of apps measured today? Web analytics Surveys and user feedback What UXprobe delivers UX & Usability focused Tracking task-based user behavior Surveys and user feedback A good app is not good enough anymore The rising power of apps - it s happening now The opportunity cost Show your customers that you care for them... 9 Summary References We will be happy to give a live demo!

3 Just an app is not enough, customers demand amazing products UX analytics to make a good app awesome For many businesses, applications have now become more and more important. As the mobile web is growing, they turn out to even be a crucial component in the success of companies. At the same time, customers expectations have grown just as fast. As they get used to better and better technology, they also want great design. Customers do not want just an app, they want a great app. More than that, what they experience when using an app - how successful they are and how they re feeling - will influence and define their relation with the service, the product and even the company as a whole. This is why User Experience has emerged as a must-have area of concern. User Experience (UX) is a subtle notion that combines user behaviors, attitudes and emotions. In order to manage and improve the UX of apps, it is important to collect metrics and insights that are specific to that subtle notion. Today, Product Managers are left with a plethora of tools, like web analytics, surveys, reviews, etc. that explore each of these dimensions separately, without any means to combine or aggregate the results. Moreover these tools were mostly developed to measure websites, not apps. They fail to analyze a much more complex behavior that goes beyond clicks and pages. This is where UXprobe comes in, a software service that combines user behavior tracking and sentiment collection that helps you pin-point the problems and opportunities in your app. 3

4 Trying to combine all these tools together is very complicated and time consuming task. 1. Building a good app is a challenge When building an app, Product Managers face a great challenge: find a clear fit between the business, engineering and users objectives. Their main goal is to build a product that solves the right problem with the right solution for the right market. In order to make their company s app that successful product, Product Managers will convince their stakeholders that they are going in the right direction. They will give trustworthy directions to the development team and they will follow users needs. To understand users needs, Product Managers, most of the time, use their own experience; their gut feeling or they go for the easy enough for Moms principle. Of course they also perform market research, they have focus group meetings, and do usability tests. All of this is good in the early stages of app development where mostly qualitative data is needed to better understand and improve concept ideas until they can be put into a first version of the app. Once you have an app that can be tested or used by a larger group of people, e.g. beta testers, it should be complemented with analytics to understand how this larger group uses the app, what they are expecting and how satisfied they are. Product Managers use web analytics services to measure the user experience of their apps, but these web analytics services were actually meant to measure marketing and business goals of web sites. Their metrics are very high-level and do not deliver any actionable insights to improve the design and usability of apps. Web analytics services just record/count what screens or pages users look at, content or features they select. They don t track what users do as a whole and certainly not why they do it or how satisfied they are. Product Managers use different kinds of feedback gathering tools for asking their users how they feel about their app. Often they use surveys to learn more about their users habits, attitudes and feelings. Unfortunately these tools only focus on one side of user experience, either on user behavior or on user feelings and satisfaction. Trying to combine all these tools together is very complicated and time consuming task. Sometimes it is even impossible to cross over the data from the different types of services. None of the current systems offers them a single platform that measures behavior, shows feedback and collects answers to surveys. Today the only real solution available is to recruit UX experts to conduct usability tests or to create a home-made system that would allow app builders to capture their own data. As Jared Spool analyzes it: We ve found the teams that get the most out of their analytics are the ones that build their own systems. Instead of relying on the http logs provided by the web server, they capture their own events ones they believe are the most important activities to study. 1 Wouldn t it be great to avoid the trouble of building a complete home-made system and have a service that does that for you? 4

5 understand the detailed flow the user experienced while using an app PIC 1. Conversion funnel, but why did people abandon the flow? 2. How is UX of apps measured today? Today, when one wants to measure the User Experience of an app, often usability testing is part of the process, especially at a very early stage. As for the available tools out there, the state of art in measuring User Experience is split between different actors. Each of them is dealing with one specific aspect of UX: Web analytics: Google Analytics, Adobe Analytics, Webtrends, Mixpanel, etc Surveys: Surveymonkey, Zoomerang, etc Feedback: Uservoice, Usabilla, etc 2.1. Web analytics Web analytics tools are about: counting page views, clicks calculating abandon and exit rates showing conversion rates and funnels Counting is what web analytics tools do by default: page views, clicks, abandons and exits. In apps, these behavioral metrics are of course needed, but only show you what people did with an app on a macro level. If you want to improve the user experience of apps you also need to understand the user behavior on a micro level, understanding what they did throughout the whole app. It is not just about measuring user interactions, but you should understand the detailed flow the user experienced while using an app. Only a detailed flow can show when things went wrong and where in the app they went wrong. When a user gets e.g. an error it is more important to know how the user s interaction resulted in that error than just knowing that the error occurred. Conversion rates today are very important, people use it as a barometer to see how well the app and hence their business is doing. Also, product teams use it to check whether the improvements they made to the app have an impact, but, unfortunately, it is not the right metric for validating design improvements. A conversion rate is a ratio, the number of people who reach the conversion goal divided by the number of users who used the app. Ratios are a problem because it consists of the numerator (people who meet goal) and the denominator (number of users) and you don t have control over which side changes. Changing the design and then looking at the conversion rate will not give you the evidence that your design was improved. It could be that you had more people achieving their goal but it could also be that this did not change and that just less people used the app. Conversion funnels show you which flows people follow while they use your app. You can see where people abandon a flow or even exit an app, but you cannot clearly deduce why they abandoned SEE PIC 1. Was it because they eventually were not interested in that piece of the app or was it because they could not work their way through the app because of UI or technical problems? 5

6 UXprobe deliberately chooses to deal with the whole User Experience in one single service 3. What UXprobe Delivers 3.1. UX & Usability focused UXprobe enables enterprises to understand the user experience and usability of their apps as opposed to the marketing performance and conversion of their websites. UXprobe is a different kind of metrics Surveys and user feedback measuring satisfaction and users feelings while they use an app implementing polls e.g to understand which features engaging with customers to get ratings and comments are preferred the most and which are not understanding users expectations Satisfaction questionnaires are good and necessary, but a rating is not enough. Users should be able to tell you why they liked or did not like your app. They should have the possibility to give you specific insights leading to improvements. Communicating with your users should be a 2-way process. Users should be able to tell the product team directly how satisfied they are with the app and what functionality and features they need. Combining user feedback and satisfaction surveys with user behavior, will give a much clearer picture of what your users are doing and why they are doing it. With current tools it is almost impossible to bring the different data sources together and deduce the actionable data with which you can improve the design of your app. User Experience (UX) involves a person s behaviors, attitudes, and emotions about using a particular product, system or service. User Experience includes the practical, experiential, affective, meaningful and valuable aspects of human-computer interaction and product ownership (Wikipedia) 2. UXprobe deliberately chooses to deal with the whole User Experience in one single service by offering a combination of behavior tracking and feelings collection. Usability is the effectiveness, efficiency and satisfaction in which specified users achieve specified goals in particular environments (definition - ISO 9241) 3. The metrics in UXprobe, the way they re collated, the way they re displayed, are focused on making it easy to understand the usability of your app. Effectiveness (completion rate / error rate): the accuracy and completeness with which specified users can achieve specified goals in particular environments»» Can users complete their tasks?»» What is the error rate? 6

7 PIC 2. Task success: effectiveness, efficiency and satisfaction Efficiency (avg. duration): the resources expended in relation to the accuracy and completeness of goals achieved»» How much effort do users require to perform a task, measured in time Satisfaction (Happiness): the comfort and acceptability of the work system to its users and other people affected by its use In the UXprobe dashboard, the graph Task (user) success will not only show you whether or not people could successfully reach your conversion goal, but it will also show you why SEE PIC 2. UXprobe allows you to drill down into the user s individual sessions and really understand what they did and where they had problems. It helps you understand by showing you e.g. the errors that occurred in a particular task, how a user went back and forth between specific screens, etc. Task success also incorporates user satisfaction collection, so you can ask your users why they left the task allowing you to get better insights about what users think is happening. With the feedback panel users can even directly send feedback about their experience and/or expectations Tracking task-based user behavior Moving to an activity-based logging system opens up a world of opportunity for the analytics writes Jared Spool 4. UXprobe provides task-based analytics which helps you understand what your users are doing with the app: which tasks they perform, the features they use, the errors they encounter, etc. Instead of just looking at pageviews, the product team can capture the complete activities which they believe are the most important activities to measure. Since it is no longer tied to pages, but to tasks, you can collect both the state change and the data that was involved (e.g. values that were filled out, etc.) Conversion funnels show where people abandon the flow but they don t show you what happened to the users that made them abandon the task. Because UXprobe is a taskbased analytics system you can look at the specific task at hand and visualize why people left. User Interface (UI) and system errors for instance can be made visible from the session details, something you miss from a conversion funnel. UXprobe s data tracker captures errors that are displayed together with the corresponding data that generated the error: Detecting frequent errors helps to understand which parts of the UI should be redesigned. 7

8 A very pragmatic approach to capture users attitudes and emotions PIC 3. Drill into sessions Detecting an increase of error messages with a new version of the app might mean that a recent change PIC 4. Surveys broke the task SEE PIC Surveys and user feedback UXprobe uses a very pragmatic approach to capture users attitudes and emotions. Micro surveys, presented at the right moment can help you understand your users satisfaction SEE PIC 4. It can be more than just a rating When your users are not satisfied you should ask them why! But at the right moment, when they have finished a task or about to exit the app. In the real world we don t interrupt others when they are in the middle of a task and so it can be when choosing the right moment to pop up a one question survey. Communication with your users should be a 2-way process, particularly when things are not going well. That s why UXprobe also offers a feedback panel that users can use no matter where they are in the app. With this panel users can tell you themselves how they feel about the app at the moment that matters to them. 8

9 If you can improve the design of your app based precisely on the direct input of your users, you will, certainly, make their lives much easier. 4. A good app is not good enough anymore 4.1. The rising power of apps it s happening now Given the overwhelming range of apps available across multiple platforms today, looking back to their early days shows just how quickly they have evolved. As recently as three to five years ago, apps were just beginning to carve out a niche in our digital world. Today, apps are everywhere. For many of us, they re an essential part of everyday life every kind of service has an app and we expect no less. Banks, energy providers, telecommunication companies, car rental firms, hotels & airlines for enterprises across all sectors, the app is now a vital asset to outperform their competition. Where will apps be in a few years? They will be essential to businesses and customers alike, a crucial method of communication between the two, helping to establish a customer base in a user-friendly format. Companies will have to compete extremely hard to secure, and maintain, a place on people s smartphones; without an app, enterprises will not even exist. Investing resources into creating innovative, engaging apps will be a key aspect of running a successful business The opportunity cost among those who run to catch up, you know you have to be vigilant and make the right choices. If you want to be among those who are leading the industry, you have to have an app that performs really well and that satisfies your customers. Once you ve missed your chance, there won t be any back button Show your customers that you care for them In this fast evolving digital world, keeping your customers happy is one of the great challenges you will face. UXprobe gives you the means to listen very carefully to them. Literally when it comes to feedback and surveys, but also in a more visual way by looking directly at what they do through behavior tracking. If you can improve the design of your app based precisely on the direct input of your users, you will, certainly, make their lives much easier. They will thank you for that. If you can identify the most common errors and the successions of events that lead to them, you might find solutions to improve the design of your app. Solutions that will effectively help your users perform better. They will thank you for that. In this digital world which changes rapidly, not missing the train has become a real challenge. If you don t want to be When feelings are listened to and understood, your users will reward you with trust and advocacy. 9

10 A great User Experience, the one that will allow users to not even think twice when using an app, is a must-have Summary For most enterprises, applications are about to be the mandatory step in their online presence. To achieve that goal, a great User Experience, the one that will allow users to not even think twice when using an app, is a must-have. If users can easily access a product or a service through the related app, they will definitely keep a spot for that app on their smartphone; they will regard it as part of their lives. User Experience (UX) involves behavior, attitude and emotions; all together. Many techniques and tools have been developed around these notions (usability testing, web analytics, feedback and survey tools), but none of them until now is a combination of the three and none of them is fully focused on apps. UXprobe aggregates the three notions of behavior, attitude and emotions in a all-in-one service. The behavioral metrics are humanized by the collection of feedback and answers to surveys. The emotions are contextualized by the behavioral metrics UXprobe s behavioral metrics focus on explaining the usability of apps by analyzing Task based success. This gives a fast and high-level understanding of how effective and efficient users are within the app. UXprobe s dashboard provides all the necessary means to understand users. With these insights, making decisions to improve the design and User Interface, to drive apps in the right direction, to make users happy, will be a clear way forward. 10

11 References 1. Jared Spool, Activity Analytics for Web Apps, User Interface Engineering, Wikipedia User Experience, 3. ISO Standard., For a full article about UX and Usability definitions: 4. Jared Spool, Activity Analytics for Web Apps, User Interface Engineering, We will be happy to give a live demo! Call us +32 (0) Send us an happy@uxpro.be Check our Website Watch our video Follow us on Facebook Look for UXprobe Google+ Look for UXprobe