1 For: Application Development & Delivery Professionals Measuring Mobile Apps by Jeffrey S. Hammond, November 18, 2013 Key Takeaways Fast Feedback Is A Natural Complement To Continuous Delivery Building five-star mobile apps requires frequent releases, but it doesn t matter how fast you release if you can t capture and process customer feedback. Together, continuous delivery and feedback from performance management tools and analytics framework define the new app development and delivery cycle. Focus On Collecting Three Types Of Mobile Measures Focus on metrics that measure your app s technical fitness, level of user engagement, and business results. Make the information you collect widely available, and present it in a format that s appropriate for consumption by developers, business leaders, and other intelligent applications. The Faster You Can Act On Feedback, The Better Feedback has a half-life. Its value degrades with time, so the faster you can act on it, the better. While it makes sense to start slow by using historical feedback to drive development and business decisions, in the long run you should collect real-time user feedback on local devices and use it to predict the next best action your app should take. Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA USA Tel: Fax:
2 November 18, 2013 Measuring Mobile Apps Performance Management: The Mobile App Development Playbook by Jeffrey S. Hammond with Phil Murphy, Julie A. Ask, Michael Facemire, and Rowan Curran Why Read This Report As you begin to build and deploy mobile apps, you ll quickly discover that your development teams need to deliver new versions and updates much faster: eight to 12 times a year or more. Mobile performance management provides an essential feedback loop to guide the content of your updates. By collecting historical, operational, and real-time data across technical, engagement, and business-aligned metrics, you can gather the information you need to analyze your app s performance, gain insight into what s working and what needs to be fixed, prioritize new features, and repair defects. This report details a number of valuable, commonly collected metrics and describes the tools and frameworks that can help you collect them. Table Of Contents Five-Star Mobile Apps Require Effective Performance Management Holistic Mobile Measurement Focuses On Three Classes Of Metrics recommendations Metrics Matter Prioritize Fast Feedback WHAT IT MEANS Developers Need To Improve Their Mobile Measurement Savvy Supplemental Material Notes & Resources Forrester interviewed 20 companies, including Adobe, Apsalar, Artisan Mobile, Bango.net, Capptain, ClickTale, Crittercism, Distimo, EPAM Systems, Flurry, HP, IBM, Kontagent, Localytics, Medio, New Relic, Perfecto Mobile, Shunra Software, and Webtrends. Related Research Documents Mobile Needs A Four-Tier Engagement Platform October 18, 2013 Forrester s Shopping Guide To Mobile Analytics Vendors August 20, 2013 Use Analytics To Build Mobile Advantage July 16, 2013 Build Five-Star Mobile Apps November 7, , Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To purchase reprints of this document, please For additional information, go to
3 Measuring Mobile Apps 2 Five-Star Mobile AppS Require Effective Performance ManagEment Mobile developers seem like they re on top of the world right now: Mobile apps are opening up new frontiers in application design, placing experienced developers in high demand. The work is fun, too; it s easier than ever to take a good idea, build it, and sell it directly to consumers via app stores. But talk to mobile developers over a few beers and you ll find that the grass on their side of the fence isn t necessarily any greener. In fact, they might just confess that they have very little idea what they re doing a large part of the time! New devices, new form factors, and new interaction mechanisms like touch, push messaging, and SMS all present new design challenges and a fair degree of chaos. As an industry, we re still learning how to design for mobile, even as tablets, phablets, glasses, and smart watches are all queuing up for attention. So what should one do when working in an environment with no established best development practices? One solution is to shift to a probe-sense-respond process: Probe the problem to discover how to design a useful app; sense the success or failure of your efforts via feedback loops; and then respond by fixing the things you ve done wrong and amplifying the things you ve done right. 1 When it comes to mobile app development, you need two important things to get this type of rapid feedback loop up and running: 1) An effective delivery process that gets changes and new capabilities into the hands of customers quickly and 2) a way to quickly assess the real-life impact of these changes (see Figure 1). Mobile performance management feeds back into the assessment step: Without an effective measurement capability, it really doesn t matter how fast you can release you re still shooting into the darkness. Previous Forrester research has already established the value of fast feedback when building five-star applications. 2 Successful mobile development shops use testing tools, feedback management tools, performance management tools, and mobile analytic frameworks to collect critical performance data that allows them to assess their efforts. This report will take a deeper look at what these shops measure and how they measure it. We ll also catalog some of the tools and techniques they use to close the feedback loop.
4 Measuring Mobile Apps 3 Figure 1 Mobile Flow Depends On Continuous Delivery And Fast Feedback Beta test Probe Develop and test Deploy Continuous delivery Respond Flow Fast feedback Develop hypotheses Measure Analyze Sense Source: Forrester Research, Inc. Effective Mobile Measurement Is Personal, Actionable, And In-Context When putting a mobile measurement strategy in place, start with the end in mind: Your goal is a convenient, easy-to-use, well-regarded app that customers like and are quick to recommend with a five-star rating. If you re building apps for employees or business partners, it s tempting to write off convenience and ease of use but resist that thought! It s not only a bring your own device world now, it s also a bring your own app world. If you don t focus on convenience and ease of use for internal apps, you re likely to find consumer-focused alternatives popping up anyway especially for tasks like document storage and collaboration. You also don t want your teams to develop bad design and coding habits. If you treat all of your constituents like customers, it s easier to apply a consistent measurement strategy that s organized around the following principles: Focusing on measuring individual experiences. With mobile apps, the customer decides whether to install, use, and recommend an app. Focusing your measurement strategy on how individuals find, use, and rate apps will help you understand different usage patterns, including how customers buy, when they interact, and who they tell about their experiences. For employees, measuring individual experiences will tell you which employees are power users and which need assistance and encouragement adapting to a mobile-first world.
5 Measuring Mobile Apps 4 Analyzing feedback in the right context. Metrics need to be available not only where they provide the most value, but also in a context that s understandable. If you ve introduced a new version of an app with a regression that causes crashes, you want to quickly alert developers and ops that something s going wrong. But business stakeholders aren t going to pore through log files; they prefer a dashboard that summarizes data like customer aging trends, bounce rates, and what customers are buying. Meanwhile, a self-managing application needs data in a terse, machine-readable format, so raw text or XML works just fine. Gathering actionable data. You can measure a wide variety of things about your app. While the resulting data may be extensive, and some of it may be interesting, it often isn t very actionable. For example, it s pretty clear how data on app crashes can lead to a better user experience, but what use is collecting data about in-app purchase habits if you can t use it to test purchase elasticity or change your product mix? Holistic Mobile Measurement Focuses On Three Classes Of Metrics In our conversations with leading mobile app developers and infrastructure providers, we ve identified three major classes of mobile metrics that app developers should collect. In descending order of strategic importance, these are business metrics, engagement metrics, and technical metrics. We ve compiled a list of business, engagement and technical measures that five-star application development shops collect and present those below. While you can always collect a more extensive set of performance data, these commonly used measurements will help you get started. Business Metrics Are Hard To Measure But Are Critical To Long-Term Success While you might be able to get short-term R&D funding or discretionary budget to experiment with mobile apps this year, at some point your business peers are going to want you to tie the expense of building and maintaining an app to their goals. Here are some common business measures that you should gather to keep them satisfied: Total revenue generated by the mobile channel. This measure rolls up all points of mobile revenue, including app sales, in-app purchase revenue, and revenue generated from ads placed in the app and viewed by users. Average revenue per user. ARPU is calculated by dividing the total revenue by the number of app users. It s helpful to establish a baseline value of an active customer over a period of a month or a year. A low ARPU might indicate an engagement problem lots of customers, not much revenue or indicate that there s a significant opportunity to grow total revenue by offering a new value-added service and scaling it up to a large number of customers.
6 Measuring Mobile Apps 5 How customers purchase inside the app. Analyzing in-app purchase data can offer a wealth of enrichment opportunities just ask the team that built Candy Crush Saga. 3 Mixing in app purchases with multivariate testing offers a powerful capability to assess customer price elasticity. Will an item that no one purchases when priced at $4.99 become a top-10 seller if it goes on sale for $0.99? Will platinum members book a weekend getaway if you target them with a time-sensitive SMS offer? More advanced shops can use conjoint analysis to assess the part worth of individual component offers. 4 Which stores drive app downloads and revenue. Do you drive more revenue from ios users downloading from the Apple App Store, or does the Android version of your app perform better? If Android users are a good chunk of your revenue, are they getting your app from Google Play, the Amazon App Store, or from another third-party store? Is it worth using the Samsung Apps store to target all of the Galaxy users out there? Unless you re monitoring your channel results, you could be doing more work than you need to or missing out on additional revenue streams. Reduced time or cost per transaction. While revenue-oriented metrics are good for measuring top-line improvement, don t ignore the bottom line! Measuring bottom-line improvement is particularly useful when measuring the performance of employee-facing applications and when there are significant differences in the cost of serving customers through different channels. The effectiveness of ads served to customers. If you re driving revenue by serving ads, it s important to maximize your reach, targeting, and transaction initiation. The ad networks will measure your effectiveness for their advertising clients, so you should measure their return to you through effective cost per mile and click-through rates. Aligning ads with relevant content, localizing content, and tying ads to premium content are all commonly employed tactics used to boost ad effectiveness. Benchmarking against other category apps. You should regularly compare your app with the best mobile apps in the same category. Analytics tools like Distimo App IQ and App Annie offer insight into the general performance of mobile apps and serve as a good starting point. Detailed teardowns of competitors apps are also a useful exercise. In addition to the measurements detailed above, you should also consider additional key performance indicators that are uniquely tied to your business strategy. Examples include industryspecific measures like reducing the number of patient readmissions for insurers or reducing the cost of point-of-sale upgrades in retail.
7 Measuring Mobile Apps 6 Engagement Metrics Show How Different Customers Use Your App (Or Don t) It s pretty common for marketing organizations to own the strategy for business-to-consumer mobile apps. In addition to business metrics, these organizations spend a considerable amount of time measuring the level of customer engagement. Why? Engagement metrics help measure increased product awareness, brand identification, and customer loyalty by providing feedback on how different groups of people use a mobile app and what can be done to match development resources to the best possible customer experience. Common engagement metrics include: The customer s initial app experience. Getting a customer to download your mobile app is a feat, but, according to mobile analytics provider Kontagent, it s only the start of the engagement process. According to the firm s data, about 10% of consumers that install an app actually use it on a regular basis. There are many reasons that a customer might abandon or delete your app, including poor performance or a confusing user interface. 5 Engaging with beta users, carefully scripting and instrumenting first-time user scenarios, and monitoring the progression toward regular interaction are three best practices that five-star mobile developers use to get customers over the hump to regular engagement. Customer retention. How long do customers continue to use your app? Look for time periods in which engagement drops off or points at which customers tend to uninstall your app. Mobile development teams will typically set a number of filters based on elapsed time (e.g., one week, 30 days, 60 days). Once you set up these filters, monitor changes closely against historic data. Rolling retention displays the percentage of users still active N or more days after they first install and launch your application. How an average user changes over time. Average users provide a good indication of how sticky your app is, especially if you re seeing good ratios of daily average users (DAU). A monthly average user (MAU) metric is a good indicator of more casual app usage and is useful for apps designed to support infrequent tasks like travel planning and monthly online bill payments. Combining these metrics (DAU/MAU) can provide insight into flagging engagement; you might need to update features or launch a marketing campaign to breathe new life into your customer base. The length of user sessions. It s important to track how much time your customers actually spend using your app once they launch it. Do they use it to glance at information and then move on, or do they engage for minutes at a time? What usage patterns did you expect and how does actual use compare? Do they use it multiple times a day whenever they have downtime, or is their usage more intentional? Knowing how much time a customer or employee spends with your app can inform decisions like where to place advertising or notifications; it can even inform user experience design like using larger text, and less of it, in glanceable apps.
8 Measuring Mobile Apps 7 How your users differ from each other. Examining user behavior patterns can yield valuable insights to drive additional engagement and revenue. One technique recommended by mobile analytics provider Flurry is to compare your hard-core customers those that use your app 25 times more than normal with your baseline population. Look for statistically common traits like geography, age, job class, or carrier. Over time, you re likely to find a number of discrete cohorts like active users, new users, hard-core buyers, and so on. Use data mining techniques to understand the differences between user cohorts and predict movement triggers between them. How and when customers buy. It s important to track when and how customers actually buy and, just as importantly, when they don t. How often do customers look at a product or offer before actually buying? (A coupon might be enough to push them over the edge.) It s also important to track influenced revenue: Does a customer look at a product on their mobile device and then consummate the purchase hours later on a laptop browser? Finally, how does time of day or location affect purchase behavior? Is it best to make mobile offers during commuting hours or geofence your brick-and-mortar stores to detect loyal customers as they come into range? App ratings. While you can collect the above metrics at your discretion, it s wise to keep in mind that apps in public stores will inevitably get customer ratings good and bad. These ratings are an overt, but anecdotal, measure of customer engagement. Pay attention, but don t overreact to a few bad comments. Look for patterns of positive and negative feedback and use them to influence decisions, backed up by the quantitative data you ve already been gathering. Expect that your competitors will use this publicly available data to assess your strength in the mobile channel, even if you ignore it. Technical Metrics Provide Feedback On Performance, Platforms, And Crashes It s hard to engage customers and employees with a mobile app that doesn t work properly on their device or is slow to respond when they use it. And with all the different types of devices out there, it s impossible to adequately test all of the different combination of devices across carriers that you ll face. Collecting technical metrics will allow you to zero in on problems in production and prioritize the worst offenders as part of a continuous improvement process. Here s what to do: Gather information on platforms, form factors, and devices. Collect data on what devices, operating systems, versions, and carriers your customer, employees, and partners use. It will help your development teams prioritize features and testing effort. For example, if 60% of your user base runs on ios devices, then it might make sense to prioritize it in your testing effort. Detect form factors as well so you can make decisions about screen resolution based on your customers adoption of 3-inch to 4.5-inch smartphones, 5-inch to 6-inch phablets, and 7-inch to 10-inch (and larger) tablets.
9 Measuring Mobile Apps 8 Prioritize reporting on crashes and errors. Nothing kills an app s rating like bugs and crashes. Collect information on where they happen, and how frequently. Examine your crash rates by device, operating system, and carrier to discover whether they are caused by a few specific bugs, a newly popular device that you didn t test, or the unique configuration of a carrier network. Use breadcrumbs and transaction tracing to mark a trail through a user s session that way, you can see what they did before the error appeared. Determine how responsive your app is. Slow app performance is another common source of poor user ratings in app stores, so make sure to measure stats like your app s time to first action, average load times for key application flows, and the time it takes to complete purchases. Unusual patterns in your engagement metrics, such as unexpected exit points in your app or increasing bounce rates, may indicate a performance problem. 6 Also, understand the difference between actual performance and the user s perception of that performance; you may be able to manage the perception with background loading or data caching. It s important to measure performance from endpoint to data store and in between to understand where your opportunities for improvement lie. Monitor sessions to scale elastically and anticipate app usage. It s important to track the number of unique user sessions in order to deliver a consistent level of performance without overdeploying infrastructure. Monitor how sessions ebb and flow on a daily, weekly, and monthly basis and how this relates to your anticipated audience. Combine session data with platform data to allow you predict capacity needs: Do you need more infrastructure running on weekends or evenings, or is that when usage drops off? Monitor sessions across mobile platforms, device types, carriers, and network connections (3G, LTE, Wi-Fi) to gain insight into how new features might be used. Assess whether you re a good steward of your user s resources. Track how much memory and storage your app requires and measure the amount of battery and network bandwidth it consumes while it s in use. Measure your resource consumption levels relative to the overall amounts available on the device and track these differences by region and user cohort. If your engagement measures make you confident that your app provides users with regular value, it s OK to consume more device resources as long as you play nice with other apps. Above all, manage your network data consumption as efficiently as possible. Don t burn up your customer s data plan it s the quickest way to get purged from a device. Storage is another area to watch out for, as you ll become a target for deletion when space gets tight on 8 GB or 16 GB devices. Collect data on the app s current context. There are a multitude of uses for contextual data such as location, speed, temperature, and direction. Such data can also help you predict your infrastructure provisioning needs; for example, you might want to spin up more servers in an Amazon availability zone that s close to a sudden surge of client sessions to optimize response time. And of course contextual data can influence real-time decision-making, like generating location-specific offers when a user crosses a geofence.
10 Measuring Mobile Apps 9 Timing Is Everything Late Metrics Are Useless Metrics It s also important to determine the information half-life of the feedback that you intend to collect. An individual point of data is most useful the instant it is collected; its value then tends to degrade over time. On the other hand, a stream of data points may still have value days or weeks after collection, as they will still indicate long-term trends in app performance and usage. Organizations that we ve spoken with suggest that there are three major classifications you should use to guide your strategy: Real-time feedback guides adaptive apps to the next best action. Real-time feedback offers the possibility to create breakthrough experiences that change the way business gets done. Consider Waze, which uses real-time feedback to alert Wazers to possible speed traps. Google Now uses real-time location data in concert with historical search data to suggest nearby locations that a Galaxy Nexus user might want to visit while traveling. Real-time feedback must be processed in seconds or subseconds, in keeping with the device user s expectation of responsive app performance. Accordingly, it often makes sense to analyze and act on real-time feedback locally in an app to minimize the latency involved with off-device data processing. Operational feedback warns of impending problems. Operational data indicates that some sort of problem is affecting the your app s performance. Maybe a carrier just released a new device that causes your application to crash; maybe there s an outage at your public cloud provider due to a lightning strike and you need to reroute your API load balancers to backup infrastructure. The value of operational data varies, but in general you should be thinking in terms of minutes, not hours. If there are 10 customers who are frustrated because their new devices don t work with your app, it might not seem like a big deal but if they go to an app store and give your app a one-star rating, things can snowball quickly. Historical feedback guides business decisions and development sprints. Historical data can be classified as any feedback point that s more than 24 hours old. Some metrics just aren t that important to the ongoing operation of an application, because there s really not that much you can do to change the data that s being produced. For example, it takes time to see how introducing a new app feature affects customer retention, or if customer engagement is moving from smartphones to tablets. With historical data, second-order information, such as trends and rates of change, is often the most valuable. A historical data approach may also be required when it takes time to acquire, manipulate, aggregate, correlate, and visualize the data in question. As you develop your mobile measurement strategy, we recommend an approach that starts by collecting technical data on a historic basis and then moves out to engagement and business metrics, while gradually pushing the information half-life down to operational and real-time levels (see Figure 2). This strategy will make it easy to get early wins, demonstrate the value that feedback provides to your decision-making process, and whet your developers and business sponsors appetites for more and better information.
11 Measuring Mobile Apps 10 Figure 2 Start With Technical Data, Then Move To Business Data And Real-Time Collection Technical metrics Historical data (>1 day) Start here Device stats; crashes Operational data Battery; session performance Real-time data (<5 seconds) Location; speed/direction; proximity Engagement metrics MAU/DAU; segmentation; cohort analysis Bounce rates; A/B testing Offers; next best actions; recommendations Business metrics Revenue; productivity; channel influence Part-worth analysis; SKU availability Upsell revenue; real-time pricing Source: Forrester Research, Inc. Use Mobile Management Tools And Analytic Services To Aid Your Effort There is an ever-expanding list of ISVs, frameworks, and services available for mobile measurement (see Figure 3). The sheer number and variety of choices at your disposal can be a bit overwhelming, but the options generally break down into the following types of solutions: Testing tools forecast how apps will work. While you won t get good business or engagement metrics until your app goes into production, you can still use mobile testing tools to perform basic diagnostics and collect technical metrics. Make sure that you test your apps on different devices and different carriers, or that you have a solution that can effectively virtualize different carrier network configurations. Examples of quality management solutions include Keynote Systems DeviceAnywhere, Perfecto Mobile MobileCloud, and Shunra NV. Feedback management tools collect and manage beta user feedback. Professional ISVs have long used beta testing as a way to get prerelease feedback on new versions of apps. Mobile feedback management tools help manage this process by allowing development teams to set up
12 Measuring Mobile Apps 11 small groups of users, distribute applications, collect feedback, and collect technical analytics, including crash reports and sessions. Examples of feedback management solutions include App47, HockeyApp, and TestFlight. Application performance management tools measure operational health. As you shift your mobile apps into production, you ll want to begin collecting technical and operational metrics across all tiers of your engagement architecture. 7 Existing application performance management (APM) tools can help, but dedicated mobile APM tools will add a layer of insight, especially when it comes to collecting device-specific technical measurements. Examples of mobile performance monitoring tools include Compuware APM, Crittercism, HP Real User Monitoring, and New Relic Mobile Application Monitoring. Mobile analytics solutions collect device, engagement, and app store data. Many specialized mobile analytics vendors collect a wide range of engagement, financial, performance, technical, and on-device metrics for mobile apps. 8 The key difference with these tools is that they are targeted at nondevelopers. Measurement data is packaged and organized at a higher level than APM tools or testing tools and delivered with prepackaged reports and dashboards. Many mobile analytics solutions are delivered as cloud base services and are designed to just drop into existing mobile apps. Examples include Adobe Analytics, Apsalar In-App Analytics, Artisan Optimize, Bango Analytic, Capptain Analytics, ClickTale, Crittercism, Distimo App Analytics and AppIQ, Flurry Analytics, Foresee Google Universal Analytics, IBM Tealeaf CX Mobile, Localytics, Medio Measure, and Webtrends Mobile Measurement.
13 Measuring Mobile Apps 12 Figure 3 Vendor Landscape: Mobile Management Tools And Analytic Services Adobe Systems App47 AppDynamics Apsalar Bango.net Capptain ClickTale Crittercism Compuware Distimo Flurry ForeSee Google HockeyApp HP IBM Keynote Localytics Medio New Relic Perfecto Shunra Software Test Flight Webtrends Technical Testing Beta testing On device Infrastructure Engagement Business Source: Forrester Research, Inc.
14 Measuring Mobile Apps 13 Recommendations Metrics Matter Prioritize Fast Feedback Mobile development forces you to increase your release cadence to satisfy the demands of your business sponsors and customers. Pair faster delivery with quick feedback to get measurable value out of the apps you build. To do that: Instrument apps early to get to regular feedback ASAP. Many analytics frameworks are simple to add; a few lines of code imports them into your app and you ll get a set of basic technical and engagement metrics that you can then build on with each successive app release. ARM your mobile teams to measure advanced behavior. When it comes to engagement, focus on measuring customer acquisition, customer retention, and the monetization of your most actively engaged customers. If you can t show business value by improving each of these measures, then you may need to rethink the entire premise of your mobile app. Create guardrails that monitor your app s operation. Aggregate operational metrics, app ratings, and defects into real-time dashboards that developers and executives alike can view easily. Consider implementing big information radiators viewable by all when you walk into the team s development space. By setting operational bounds and transparently measuring against them, you ll get immediate notification of events and put developers and operations pros into a mode where they re proactively watching for early signs of distress and actively working the backlog of feedback. Track second-order metrics and measure improvement over time. It s good to know how many crashes you have, but it s better to see that number getting smaller. It s also good to see DAU and MAU going up, as they re a good predictor of better days to come. Make sure to track the long-term trends in your metrics and use them to plan investments in future features, advertising, and staffing. Reconcile mobile metrics with data from other devices. As users work across an increasing number of devices, cross-device journeys are inevitable. Understanding when, why, and how customers put down one device and pick up another is key to building an omnichannel modern app strategy. Seek to employ the concept of a unique user who you can track across devices, even if that means gathering registration information from customers or business users.
15 Measuring Mobile Apps 14 What It Means Developers Need To Improve their Mobile Measurement Savvy Many application development professionals have little to no skills in collecting real-time and operational metrics; that s usually been the province of infrastructure and operations pros. But such thinking is both antiquated and dangerous. As requirements elicitation shifts from long textual requirements documents and lengthy analysis phases into a feedback-driven, probe-sense-respond way of working, analytics and multivariate tests become critical concepts that developers need to master. Experiment with analytic frameworks to open developers eyes to the value of feedback; in the process, you ll set the stage for faster releases and better results. Supplemental Material Companies Interviewed For This Report Adobe Apsalar Artisan Mobile Bango.net Capptain ClickTale Crittercism Distimo EPAM Systems HP IBM Kontagent Localytics Medio New Relic Perfecto Mobile Shunra Software Webtrends Flurry Endnotes 1 For more information about the value of a probe, sense and respond approach to complex problem resolution, we suggest learning about the Cynefin framework. Source: The Cognitive Edge (http://cognitiveedge.com/library/more/video/introduction-to-the-cynefin-framework/). 2 Differentiating your app from the hundreds of thousands of others on the market requires rethinking many of the development processes use in previous applications. See the November 7, 2012, Build Five-Star Mobile Apps report.
16 Measuring Mobile Apps 15 3 While you don t have to buy any in-store purchases to complete the Candy Crush Saga, many players do to the tune of hundreds of thousands of dollars every day. Source: Stuart Dredge, Candy Crush Saga: 70% Of The People On The Last Level Haven t Paid Anything, The Guardian, September 10, 2013 (http://www. theguardian.com/technology/appsblog/2013/sep/10/candy-crush-saga-king-interview). 4 Part-worth analysis, also known as conjoint analysis, is often used to measure preferences for product features in order to learn how changes to price affect demand for products or service. While it s most often done before introducing a product with user surveys, it can also be done in production through dynamic pricing. 5 Crashes and a confusing user experience are frequently cited as factors for app uninstalls. Source: utest, Five Reasons Users Uninstall Apps, VentureBeat, June 3, 2013 (http://venturebeat.com/2013/06/03/fivereasons-users-uninstall-mobile-apps/). 6 Your bounce rate is the percentage of visits that go only one page before exiting your app or mobile website. Source: Google Analytics Support (https://support.google.com/analytics/answer/ ?hl=en). 7 Forrester has developed a four-tier reference architecture for mobile app engagement. See the October 18, 2013, Mobile Needs A Four-Tier Engagement Platform report. 8 Mobile analytics are underutilized and often not well-sourced. Forrester has designed a guide to help you shop for the best mobile analytics solution for your organization. See the August 20, 2013, Forrester s Shopping Guide To Mobile Analytics Vendors report.
17 About Forrester A global research and advisory firm, Forrester inspires leaders, informs better decisions, and helps the world s top companies turn the complexity of change into business advantage. Our researchbased insight and objective advice enable IT professionals to lead more successfully within IT and extend their impact beyond the traditional IT organization. Tailored to your individual role, our resources allow you to focus on important business issues margin, speed, growth first, technology second. for more information To find out how Forrester Research can help you be successful every day, please contact the office nearest you, or visit us at For a complete list of worldwide locations, visit Client support For information on hard-copy or electronic reprints, please contact Client Support at , , or We offer quantity discounts and special pricing for academic and nonprofit institutions. Forrester Focuses On Application Development & Delivery Professionals Responsible for leading the development and delivery of applications that support your company s business strategies, you also choose technology and architecture while managing people, skills, practices, and organization to maximize value. Forrester s subject-matter expertise and deep understanding of your role will help you create forward-thinking strategies; weigh opportunity against risk; justify decisions; and optimize your individual, team, and corporate performance. «Andrea Davies, client persona representing Application Development & Delivery Professionals Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice to global leaders in business and technology. Forrester works with professionals in 13 key roles at major companies providing proprietary research, customer insight, consulting, events, and peer-to-peer executive programs. For more than 29 years, Forrester has been making IT, marketing, and technology industry leaders successful every day. For more information, visit
FOR: Interactive Marketing Professionals Win The Social Marketing Measurement Game by Nate Elliott, November 21, 2012 KEY TAKEAWAYS Avoid The Temptation To Oversimplify Your Metrics Blended single engagement
Best Practices and Tools for the Mobile App Analytics A white paper on Fundamentals and Implementations of Analytics for your Mobile Application Mobile App Analytics Accelerate your Mobile App Monetization
October 27, 2009 Case Study: ING Delivers Personalized Product Offers Across Channels In Real Time by Alexander Hesse for Customer Intelligence Professionals Making Leaders Successful Every Day October
Proactive Outbound Notification Saves Money And Provides A New Level Of Personalized Services To Customers by Elizabeth Herrell with Robert Whiteley and Margaret Ryan Executive Summary Proactive customer
July 15, 2013 The Enterprise Information Management Barbell Strengthens Your Information Value by Alan Weintraub with Leslie Owens and Emily Jedinak Why Read This Report Businesses increasingly rely on
A Custom Technology Adoption Profile Commissioned By BMC Software January 2015 Digital Business Requires Application Performance Management Introduction Digital is transforming the rules of business success.
Mobile Apps: What Consumers Really Need and Want A Global Study of Consumers Expectations and Experiences of Mobile Applications The Difference Between a Mobile App and a Mobile Website Before we evaluate
The Changing Face of Mobile Customer Engagement Sheryl Kingstone, Research Director, Yankee Group Mobile Research Summit: Data & Insights 2014 June 19, 2014 Page 1 Copyright 2014 Agenda Why Mobile is Changing
Drive Business Further Faster With RetailNext Built especially for retailers, RetailNext is a scalable in-store analytics platform that makes it easy for you to collect, analyze, and visualize data about
Crittercism 2 Key Takeaways Mobile App Complexity App Responsiveness App Uptime (Crash) Mobile app performance is challenging with over 100M permutations of variables to manage. To be competitive, your
1 Introduction Web applications today are part of every IT operation within an organization. Independent software vendors (ISV) as well as enterprises create web applications to support their customers,
A Forrester Consulting Thought Leadership Paper Commissioned By PubMatic August 2015 Real-Time Data Analytics Empowering Publishers To Make Better, Faster Decisions Table Of Contents Executive Summary...
The Five Essential Metrics For Managing IT by Craig Symons with Alexander Peters, Alex Cullen, and Brandy Worthington EXECUTIVE SUMMARY CIOs frequently ask what IT should measure and report to business
OMNI-CHANNEL MARKETING Top 9 Questions TABLE OF CONTENTS 2 3 5 7 8 9 10 11 12 13 14 15 16 Table of Contents Introduction Top 9 Questions How does omni-channel differ from multi-channel marketing? Why is
For: Customer Experience Professionals The Business Impact Of Customer Experience, 2014 by Maxie Schmidt-Subramanian, March 27, 2014 Key Takeaways Customer Experience Correlates To Loyalty Forrester once
A Forrester Consulting Thought Leadership Paper Commissioned By TIBCO Software February 2016 Improving The Retail Experience Through Fast Data Overview Customers expect better-individualized experiences
The Right CRM Metrics For Your Organization by William Band with Sharyn C. Leaver and Mary Ann Rogan EXECUTIVE SUMMARY Forrester interviewed 58 executives about their best practices for getting more value
For: Customer Insights Professionals Gauging Web Analytics Practices In The Age Of The Customer by James McCormick, June 10, 2014 Key Takeaways Web Analytics Alone Is Insufficient To Deliver Digital Intelligence
FOR: Customer Experience professionals The State Of Customer Experience, 2012 by megan Burns, April 24, 2012 key TakeaWays Companies have Lofty Customer experience goals More than 90% of respondents to
Tapping into Mobile App Installs Building a Valuable User Base for Your App Introduction If your business has an app, or you re planning to launch one, you ve probably spent a lot of time thinking about
October 28, 2014 Brief: Forrester s Shopping Guide For Mobile Application Performance Management Solutions Mobile Application Performance Management Is A Critical BT Agenda Solution For Optimizing Mobile
2014 Digital Marketing Optimization Survey results Top lessons learned from the leaders Table of contents 1: Introduction 2: Five lessons from the top 20% #1: They test to make decisions 3: #2: They put
Where s my real ROI? White Paper #1 February 2014 expert Services revenue costs The growing confidence of advertisers and agencies in mobile marketing requires an effective and veridical sort of ROI metrics.
Case Study: AMERICAN SYSTEMS Demonstrates The Value Of Business Service Management From Reactive To Proactive: Using Service Management To Leverage Integrated Event Correlation Executive Summary by Evelyn
5 Big Data Use Cases to Understand Your Customer Journey CUSTOMER ANALYTICS EBOOK CUSTOMER JOURNEY Technology is radically transforming the customer journey. Today s customers are more empowered and connected
A Forrester Consulting Thought Leadership Paper Commissioned By Google DoubleClick August 2014 A Faster Pace For Retail Paid Search Real-Time Insights Are Critical To Competitive Advantage Table Of Contents
A Forrester Total Economic Impact Study Commissioned By SAS Project Director: Dean Davison February 2014 The Total Economic Impact Of SAS Customer Intelligence Solutions Intelligent Advertising For Publishers
Case Study: Reliant Security Innovates With Open Source Software by Jeffrey S. Hammond with John R. Rymer and Justinas Sileikis Executive Summary Retailers that want to accept credit cards need to demonstrate
WHITE PAPER: WEB PERFORMANCE TESTING Everyone wants more traffic to their web site, right? More web traffic surely means more revenue, more conversions and reduced costs. But what happens if your web site
Adobe Analytics What s New in Analytics: Fall 2015 Adobe Analytics powers customer intelligence across the enterprise, facilitating self-service data discovery for users of all skill levels. The latest
8 CRITICAL METRICS FOR MEASURING APP USER ENGAGEMENT Contents Measuring the Success of Your Mobile App...01 1. Users...04 2. Session Length...07 3. Session Interval...12 4. Time in App...15 5. Acquisitions...18
Retailers Plan To Expand Online Customer Service Channels In 2010 But Disconnection Between Customer Service And Brand May Spell Trouble by Diane Clarkson with Carrie Johnson, Elizabeth Stark, and Kate
A Forrester Consulting Thought Leadership Paper Commissioned By Tapad July 2014 The New Path-To- Purchase The Connected Consumer s Cross- Device Journey Table Of Contents Executive Summary... 1 Today s
MODERN MARKETING ESSENTIALS GUIDE MOBILE MARKETING A Prescriptive Guide Fortified to Build Stronger Marketing INTRODUCTION The sheer volume of marketing information available today is nearly impossible
For: Customer Experience Professionals The Business Impact Of Customer Experience, 2013 by Maxie Schmidt-Subramanian, June 10, 2013 Key Takeaways Customer Experience Correlates To Loyalty Forrester once
Optimizing Customer Retention Programs by Suresh Vittal with Christine Spivey Overby and Emily Bowen Executive Summary Managing customer churn continues to flummox direct marketers. They struggle to define,
FOR: ebusiness & Channel strategy Professionals Mobile Measurement Is An ebusiness Imperative by Joe stanhope, May 15, 2012 Key TaKeaWays Mobile ebusiness Measurement faces Three hurdles from The start
A Forrester Consulting Thought Leadership Paper Commissioned By Gainsight April 2014 How To Get Started With Customer Success Management Table Of Contents Four Actionable Steps To Setting Up Your Customer
THE ULTIMATE WEB PERSONALIZATION PLANNING GUIDE HOW TO PUT YOUR PERSONALIZATION STRATEGY INTO ACTION TABLE OF CONTENTS Why Personalization is Important...2 Getting Started: Defining Personas and Journeys...3
FOR: Interactive Marketing Professionals The Forrester Wave : SEO Platforms, Q4 2012 by shar VanBoskirk, October 31, 2012 key TakeaWays seo isn t Just about agencies anymore Search marketers have traditionally
NetQoS Offers An Experience Monitoring Solution For Global Performance Management The Forrester Wave Vendor Summary, Q2 2007 by Jean-Pierre Garbani with Thomas Mendel, Ph.D., and Reedwan Iqbal EXECUTIVE
White Paper Explode Six Direct Marketing Myths Maximize Campaign Effectiveness with Analytic Technology Table of contents Introduction: Achieve high-precision marketing with analytics... 2 Myth #1: Simple
A Forrester Consulting Thought Leadership Paper Commissioned By AT&T August 2013 Table Of Contents Executive Summary... 2 The Profile Of Respondents Is Across The Board... 3 Investment In Collaboration
Includes Forrester research panel data, Client Choice topic Social Media Playtime Is Over Applications by Jeremiah K. Owyang with Josh Bernoff, Tom Cummings, and Emily Bowen EXECUTIVE SUMMARY The recession
6 A/B Tests You Should Be Running In Your App Introduction Everyone in mobile believes in A/B testing. But if you re wasting your days testing nothing more than various colors or screen layouts: you re
2015 Mobile App Testing Guide Basics of Mobile App Testing Introduction Technology is on peek, where each and every day we set a new benchmark. Those days are gone when computers were just a machine and
Defining Mobile App Intelligence 1 Defining Mobile App Intelligence: Ensuring Superior Mobile App Responsiveness to Build Business Advantage Mobile apps transform daily life The explosion of mobile devices,
Qualcomm Retail Solutions Inc. GIMBAL PLATFORM DIGITAL INSIGHTS INTO THE PHYSICAL WORLD The Advantages of Gimbal for Retailers, Brands and Application Developers Revision 1 November 2013 1 Table of Contents
Enterprise Services Whitepaper 3 Reasons Why You Need Real User Measurements Contents There s Money in the Metrics 2 Real User Measurements: The New Revenue Drivers 3 RUM catches problems synthetic monitoring
For: Customer Insights Professionals The Forrester Wave : Cross-Channel Attribution Providers, Q4 2014 by Tina Moffett, November 7, 2014 Key Takeaways AOL/Convertro, Google, And Visual IQ Lead The Pack
For: Application Development & Delivery Professionals Big Data In Banking: It s Time To Act by Jost Hoppermann and Martha Bennett, April 15, 2014 Key Takeaways Words, Rather Than Deeds, Currently Dominate
For: Marketing Leadership Professionals Make The Most Of Analytics To Meet Your Mobile Objectives by Thomas Husson, September 25, 2014 Key Takeaways Lack Of Clear Objectives And Defined KPIs Hinders Mobile
A Forrester Consulting Thought Leadership Paper Commissioned By SAP Delivering New Levels Of Personalization In Consumer Engagement A Guide For Marketing Executives: Strategy, Capabilities, And Technologies
And How To Calculate The ROI On Fixing It INTRODUCTION If you re a consumer-facing business of any size, you ve probably got a mobile app. You may well take a close interest in how well it functions, and
An AppDynamics Business White Paper HOW MUCH REVENUE DOES IT GENERATE? Why Alerts Suck and Monitoring Solutions need to become Smarter I have yet to meet anyone in Dev or Ops who likes alerts. I ve also
Hyper-Personalization with MNO Subscriber Data Approaches and Recent Trends Regarding the Use of MNO Data for Hyper-Personalization By Zach Cohen, Shahzad Zia and Carly Christian Private and public-sector
Mobile Games in the App World 1 What is an App: The term "app" has become popular over the years and in 2010 and was listed as "Word of the Year" by the American Dialect Society. App, in the sense that
FOR: Customer Experience professionals Contextualization by Ron Rogowski, stephen powers, and Anjali Yakkundi, november 19, 2012 key TakeaWays knowing user Context is Critical For delivering Relevant experiences
Adobe Digital Publishing Suite, Analytics Service Analyze and optimize content for greater business impact Table of contents 1: Business benefits 2: Key features 2: Standard baseline analytics reporting
IBM Global Business Services Microsoft Dynamics CRM solutions from IBM Power your productivity 2 Microsoft Dynamics CRM solutions from IBM Highlights Win more deals by spending more time on selling and
Personalized User Journeys By Kevin Jackson Global Sales Director Gravity R&D 12/15/14 Table of Contents Omnichannel and Retail 2.0... 3 Moments of Truth (MOTs)... 4 ibeacons, MOTs, and Big Data... 5 Personalized
THE MOBILE SHOPPER IS HERE 88% of Respondents Use Retail Mobile Apps HOW RETAIL MOBILE APPS HAVE SHAKEN UP THE INDUSTRY The way customers shop, browse, and purchase is evolving. Retail sales have become
Research Report Charging and Billing for the Digital Economy Copyright Openet Telecom, 2013 Index Introduction 3 1. The Future for Traditional Billing and Charging 4 2. Innovating and Accelerating Time
A Forrester Consulting Thought Leadership Paper Commissioned By Purolator November 2014 Canadian Ship-To-Store Programs Benefit Both Consumers And Retailers Table Of Contents Executive Summary... 1 Canadian
Business Process Services White Paper Personalizing E-Commerce: Improving Interactivity to Increase Revenues About the Author Subramaniam MV Subramaniam is a Delivery Manager at Tata Consultancy Services
For: Customer Insights Professionals The Forrester Wave : Web Analytics, Q2 2014 by James McCormick, May 13, 2014 Key Takeaways Adobe, AT Internet, IBM, And Webtrends Are Leaders In Enterprise Web Analytics
A Forrester Consulting Thought Leadership Paper Commissioned By hybris October 2013 Online And Mobile Are Transforming B2B Commerce Firms That Act Now Will Gain Appreciably, Companies That Don t Will Fall
January 27, 2012 The Forrester Wave : US Digital Agencies Mobile Marketing Strategy And Execution, Q1 2012 by Melissa Parrish for Interactive Marketing Professionals Making Leaders Successful Every Day
Includes data from Business Data Services, Client Choice SaaS Economics Will Change ISVs SI And VAR Channels Channel Managers: Retool Partner Programs To Prevent SaaS Delivery Disaster by Michael Speyer
TECH CHOICES Allant: A Market Leader Among Database Marketing Service Providers The Forrester Wave Vendor Summary, Q1 2006 by Eric Schmitt with Chris Charron and Jennifer Joseph EXECUTIVE SUMMARY The Allant
A Custom Technology Adoption Profile Commissioned by Riverbed A Tidal Wave of Dynamic Web Content Is Coming How Will You Respond? March 2012 Introduction As enterprises shift from desktop apps to web and
App user analytics and performance monitoring for the business, development, and operations teams CA Mobile App Analytics for endto-end visibility CA Mobile App Analytics WWW.OVUM.COM Written by: Michael
For: Customer Insights Professionals The Forrester Wave : Loyalty Program Service Providers, Q4 2013 by Emily Collins, October 30, 2013 Key Takeaways Loyalty Service Providers Don t Just Support Points
Case Study: InterContinental Hotels Group Consolidates Email And Data Management by Shar VanBoskirk with Kate van Geldern and Angie Polanco Executive Summary InterContinental Hotels Group (IHG) revamped
Accenture Business Intelligence for Fashion and Luxury Creating a Differentiated Customer Experience for Long-term Brand Loyalty Fashion is inherently an ever-changing industry. Customer preferences fluctuate
A Forrester Consulting Thought Leadership Paper Commissioned By Zebra Technologies October 2015 Is It Time To Refresh Your Wireless Infrastructure? Table Of Contents Executive Summary... 1 The Age Of The
Six Signs you are ready for BI WHITE PAPER LET S TAKE A LOOK AT THE WAY YOU MIGHT BE MONITORING AND MEASURING YOUR COMPANY About the auther You re managing information from a number of different data sources.
A Forrester Consulting Thought Leadership Paper Commissioned By Trend Micro Key Strategies To Capture And Measure The Value Of Consumerization Of IT Enterprises Achieve A Wide Range Of Benefits By Deploying
Accelerating SharePoint Performance The guys at Application Performance quickly understood our SharePoint problem. They demonstrated that the combination of WebTuna and Riverbed SteelApp Web Accelerator