1 WHITE PAPER + MOBILE MEASUREMENT The Mobile Measurement Framework Making Sense of Your Mobile Efforts in the Context of Your Business by Eric T. Peterson, Web Analytics Demystified, Inc. OpinionLab, Inc. 600 Central Avenue, Suite 265 Highland Park, IL This document briefly outlines the challenges companies face today when working to measure their mobile efforts and describes how Web Analytics Demystified s mobile measurement framework leverages qualitative feedback provided by the paper s sponsor, OpinionLab. The result is a rich set of key performance indicators focused on user experience, interaction, engagement, and costs that can be applied to mobile sites, mobile applications, and traditional web sites easily, efficiently, and inexpensively. EXECUTIVE SUMMARY The mobile era has clearly arrived for consumers and businesses alike. Phones and mobile devices provide engaging and delightful experiences, rapidly transforming from telephonic tools to a complete, always-on connection to the digital universe. Taking advantage of device evolution, smart business owners are deploying increasingly robust digital media properties and applications designed to further promote user engagement, create entirely new revenue channels, and drive buzz in the marketplace. And despite all of this activity, one has the feeling that the mobile revolution is only getting started. Unfortunately, measurement in the mobile realm is far from easy for businesses. Between fragmentation of devices, platforms, and programming languages, and the lamentable response of some platform providers that choose to limit the ability of their development partner s business efforts, mobile analytics currently fails to benefit from the nearly fifteen years of work put into measuring web sites and applications. Whereas analytical specialists focusing on traditional fixed web sites have gained valuable experience regarding what to measure and how to use that output, their experience is proving difficult to apply to the multitude of emerging mobile channels. Web Analytics Demystified has gained a great deal of experience working with businesses extending their reach through a variety of mobile efforts, and as a result of this work we have established a surprisingly simple methodology to help companies begin to get their arms around measurement in the mobile world. More importantly, our methodology recognizes that mobile is essentially just another channel, one that needs to be examined in the context of all other digital efforts, and provides a comparative framework that relies on both quantitative and qualitative input. We focus our measurement framework in large part on qualitative input for one simple reason qualitative data and direct customer and user response delivers significant value to companies willing to listen. All of the companies interviewed for this document as well as Web Analytics Demystified s many clients working to measure mobile efforts tell largely the same story as Joe Megibow, Vice President of Global Analytics and Optimization at Expedia.com, who commented, We get five times more feedback from mobile users than we do on our traditional web properties. [What we love is] this feedback is far more actionable than the ratings that appear elsewhere in social media and online.
2 WHITE PAPER + MOBILE MEASUREMENT PAGE 2 Table of Contents EXECUTIVE SUMMARY 1 THE MOBILE MEASUREMENT CONUNDRUM 3 1. Multiple Platforms Make for Massive Headaches 3 2. Multiple Devices Make for Mechanical Difficulties 5 3. Multiple Privacy Concerns Make for Modern Nightmares 5 THE WEB ANALYTICS DEMYSTIFIED SOLUTION 7 Come to Terms with Fundamental Limitations (and Move On) 7 Strip Measurement Down to The Essentials 7 Apply the Essentials Equally Across All Platforms and Channels 8 THE MOBILE MEASUREMENT FRAMEWORK 9 User Experience: Percent Positive Feedback 9 Interactions 13 Engagement Rate 15 Adding Costs and Revenue to the Essential Measures 17 SPECIFIC EXAMPLES 19 For Online or Multi-Channel Retailers 19 For Online Travel Sites 19 For Financial Services Institutions 20 CONCLUSIONS 21 About the Author 22 About OpinionLab 22
3 WHITE PAPER + MOBILE MEASUREMENT PAGE 3 THE MOBILE MEASUREMENT CONUNDRUM Few debate the opportunity facing businesses today when it comes to mobile devices and smartphones. Gartner reports that despite a small (0.9%) decline in total mobile phone sales between 2008 and 2009, smartphone sales to consumers grew nearly 24 percent during the same period and 41 percent in the fourth quarter when compared to the same period in Applications designed to run on these devices are predicted to generate over $6.7 billion in 2010 and increase to nearly $30 billion by the end of led by sales in money transfer, locationaware, search, and mobile browsing applications 3. While sales and adoption of mobile technology grows at a breakneck pace, the ability of business owners to measure the interaction, use, and enjoyment of mobile web and mobile applications is unfortunately not keeping up. Unlike the growth in Internet use and adoption which is largely powered by a small set of platforms (Microsoft, Apache), development environments (.NET, PHP) and browsing devices (Internet Explorer, Firefox, Safari, Chrome) that exhibit a strongly similar set of characteristics, the mobile ecosystem has a far greater diversity of platforms, environments, and browser/devices which are each more different than not. What s worse is that mobile is really two opportunities presenting two marginally different user engagement models--mobile web and mobile applications. The mobile web is functionally similar to the fixed Internet, leveraging HTML, CSS, and a traditional client-server architecture. Mobile applications are essentially software developed for specific platforms using diverse programming languages and application development frameworks like Cocoa, Android, and Windows. The differences between the two manifest in a variety of ways including development, distribution, and most important in the context of this paper, their measurement. Regarding development and distribution, it is important to keep in mind that mobile web and mobile applications are really two completely different creatures. Mobile web sites can be changed in real-time simply by publishing a new version of the site and the same version of the site will be seen by all users. On the other hand, mobile applications are software projects that are essentially shipped and live in isolation on the devices they re installed upon. Making changes to mobile applications therefore requires the changes to cascade through the point of distribution (e.g., Apple s App Store) and end users need to install/upgrade to see the changes manifest. What s more, because mobile applications are installed software they have greater visibility into the platform on which they re running. The most obvious example is location, the ability of a GPSenabled phone to report back on its physical location, hopefully after asking the user s permission to do so. This functionality simultaneously imparts great power to mobile applications and creates great risk and consternation among the mobile community. More discussion of this issue follows, but based on comments that Apple s CEO Steve Jobs made at the D8 Conference 4, it appears that the exact functionality afforded mobile applications may have been the reason Apple added language to their developer agreement banning the use of third-party tracking systems without their express permission. From a measurement perspective, the mobile ecosystem creates three primary challenges arising from the multitude of platforms, devices, and underlying privacy concerns: 1. Multiple Platforms Make for Massive Headaches The platform challenge mobile developers face is similar to that faced during the early days of the Internet when Netscape and Microsoft were competing for hegemony over the browser market. During the browser wars site designers were more or less forced to develop and tweak site designs to accommodate the nuances exhibited by both browser types
4 WHITE PAPER + MOBILE MEASUREMENT PAGE 4 Fortunately at the time, despite having to support two browsers, the majority of the market was surfing the Internet on PCs (as opposed to Apple or Unix) so the problem was a headache but one more or less contained. Business owners wishing to provide a satisfying experience to their visitors could build an IE version of their sites that worked well on a PC and then tweak that slightly to accommodate Netscape (or visa versa, depending on their individual preferences.) With mobile there is little hope for containment. Consider the distribution of smartphone sales by operating systems from Q2/2009 (Figure 1). Figure 1: Global Smartphone sales by operating system, provided by Canalys via Wikipedia (http://en.wikipedia.org/wiki/mobile_operating_system) This data does not include Palm s WebOS which likely has negligible market share. Since Palm was purchased by HP in April 2010 this is likely to change over time. While the data presented in Figure 1 is certainly subject to change given the fluidity in the market and the ongoing investment on behalf of behemoth companies like Apple, Google, Microsoft, Nokia, and now HP, the challenge should be clear: business owners wishing to provide a satisfying experience need to consider as many as six or seven different development platforms for both their mobile web and mobile application efforts. But wait, there s more. While each of these platforms is bundled with a core web browser, many are capable of running multiple or alternative browsers as well. Wikipedia s page on mobile browsers lists nearly two dozen applications and references several others 5, and while deployment data for these alternative browsers is somewhat difficult to come by, Opera s press release declared that over one million people downloaded their Opera Mini browser for the iphone in the first day of its availability alone 6. Unfortunately, platforms and browsers aren t the full extent of the challenge. We still need to consider the devices themselves. 2. Multiple Devices Make for Mechanical Difficulties 5 6
6 WHITE PAPER + MOBILE MEASUREMENT PAGE 6 Web Analytics Demystified believes these restrictions are much like throwing the baby out with the bathwater and represent an unfortunate trend in the industry today. Hopefully calmer heads will prevail and device manufacturers and carriers will do the right thing, opting to provide site and application developers incremental access to device-collected user data (based on a series of permissions granted by the user) rather than arbitrarily blocking access or worse, blocking access to this valuable data altogether. Despite all of these challenges, Web Analytics Demystified believes that mobile-site and -application tracking is a fundamental requirement for any business moving into this exciting emergent channel.
8 WHITE PAPER + MOBILE MEASUREMENT PAGE 8 All three concepts are easy to understand given their widespread and colloquial use in society and technology: Experience covers the qualitative aspects of a user s interaction with the mobile offering. Does it solve their problem or create frustration? Does it work as advertised or not at all? Will they tell their friends to try or buy or encourage them to blow you off? Interaction deals with the quantitative aspects of the relationship without getting hung up on semantics and technical terms. People don t visit their Facebook mobile application any more than they deal significantly in page views when using Google Maps on any platform, mobile or otherwise. Engagement bridges the gap between qualitative and quantitative measures. While seemingly complex, measuring engagement is actually surprisingly simple; all that is required is agreement on the level of attention that justifies calling someone engaged. These essential measures are fully explored in the next section, but first, we want to point out what is perhaps obvious: these measures transcend devices, platforms, and channels and have the potential to introduce for the first time ubiquitous measures of success across the digital landscape. Apply the Essentials Equally Across All Platforms and Channels When you come to terms with the limitations inherent in traditional web analytics and understand the essential measures we propose, you will quickly recognize that these measures can be applied to any channel, any device, and any technology. This is powerful for many reasons, but perhaps the most useful is that this approach allows businesses to evaluate their efforts in any one channel in the context of all other efforts. Imagine having the ability to directly compare your mobile applications across multiple platforms and technologies, to compare mobile web and mobile application initiatives, and to compare your mobile efforts to your fixed web efforts? What if you could then compare those efforts directly to your call center? What if you could compare all digital channels to your efforts in your physical stores? If you re like the majority of our clients today, you certainly would, creating a new level of understanding about investment and return in an increasingly complex world. Furthermore, these three simple measures can also be readily applied to your social channels. Facebook, Twitter, YouTube, and others are all trending toward a more measurable state, one where interaction, engagement, and sentiment can be relatively easily scored and measured. For example, applications like Twitalyzer 8 are already being used by businesses around the world to calculate interactions, impact, and engagement: Figure 2: Twitalyzer metrics for this paper s author, highlighting Engagement which can be used in much the same way as the other metrics described in this paper. While additional work is required to integrate your social efforts (which is beyond the scope of this paper), we recommend that you consider this integration and point you to our free white paper called Social Marketing Analytics, co-authored with New York Times bestselling author Charlene Li and respected social media analyst Jeremiah Owyang (Twitalyzer is a product of Web Analytics Demystified) 9
9 WHITE PAPER + MOBILE MEASUREMENT PAGE 9 THE MOBILE MEASUREMENT FRAMEWORK As mentioned previously, our essential measures deal with experience, interaction, and engagement. While these three terms are intuitive and somewhat obvious, in our experience they can oftentimes be confusing because of their simplicity. To prevent any confusion by readers, following are concise and concrete definitions of each measure and its application. User Experience: Percent Positive Feedback Our first essential measure quantifies the overall quality of user experience. Because traditional clickstream data is incapable of quantifying quality in the absence of complex statistical models, Web Analytics Demystified recommends using the right tool for the job and providing the audience with a direct mechanism for expressing their joy as well as frustration. Because it is our belief that the best strategy for collecting this type of data is also the simplest possible, we recommend taking the approach popularized by Apple s App Store rating service and deployed by a number of vendors including this paper s sponsor, OpinionLab. The core recommended functionality is two part: 1.1. A dead-simple rating system to allow the user to quantify their opinion about the quality of their experience 1.2. A free-text area that allows motivated individuals to expand on the rating they provided, offer suggestions, or simply vent their frustration While there are a variety of different ways to use the quantified opinion scores, ranging from a raw average to some more complex calculation like the Net Promoter Score (NPS), our universal approach is to simplify whenever possible, so we recommend the following: using any numerical or visual scoring system, group the possible results into three categories: negative, neutral, and positive. For example, on a ten-point scale (with ten being the best possible score) the three groups would be: Negative = scores between 1 and 3 Neutral = scores between 4 and 7 Positive = scores between 8 and 10 You may choose to use a broader range for negative and positive, compressing the neutral scores, which is fine. You may also be tempted to somehow weight higher scores, for example: counting 10 scores twice, which we strongly recommend not doing because it makes additional assumptions. Regardless, the important thing is to be consistent and be in agreement. Then, do not change the categories after the fact to improve your results. With the positive grouping defined, all that is left is to count the number of positive scores and divide by the number of total scores collected. For example, if you collected 100 ratings, 30 of which were positive, your Percent Positive Feedback score would be 30 percent: 30 positive scores / 100 total scores = 0.30 * 100 = 30 percent This extends easily to other solutions, such as that provided by Opinonlab, who uses a slightly different scale that has negative, neutral, and positive inherent to the methodology. On the surface, this scoring certainly resembles Net Promoter, and certainly if your organization has already standardized on the Net Promoter Score, you should use that instead of our recommendation. Either way, the goal is to quantify the relative amount of delight your mobile applications are able to generate among your users, focusing on the positive rather than the negative.
10 WHITE PAPER + MOBILE MEASUREMENT PAGE 10 Figure 3: Example of OpinionLab visualization of score distributions. In this case you would simply count the number of feedback that received a + or ++ from your audience and divide by the total number of feedback collected. Some may argue that negative feedback needs to be taken into account, perhaps by also reporting the Percent Negative Feedback gathered. On this point, we caution that no company sets out to create negative experiences so focusing on them is somewhat counterproductive. Also, research has repeatedly shown that negative feedback is quite easy to gather, suggesting that the positive feedback you do get is even more valuable. Still, our goal is not to change your approach toward collecting qualitative feedback but rather to strongly encourage you to do so, especially in mobile channels where user needs are often more immediate. When you report Percent Positive Feedback in your regular reporting on mobile websites and mobile applications, we strongly recommend putting those values in context. For example: If you have multiple sites and/or mobile applications, you should consider providing interested stakeholders a next level down view that details the results for each site and application.
11 WHITE PAPER + MOBILE MEASUREMENT PAGE 11 Furthermore, this exact same data-collection strategy can be extended to fixed Internet sites by adding site scores to the same reporting for additional context: Regarding sample size (reported as n = 200 in the examples above), we strongly recommend working with whatever vendor you select for your data collection or a statistician if you collect the data using your own technology to determine the point at which you have a robust sample suitable for extrapolation across your entire audience. In situations where the time for regular reporting has come but your sample is insufficient, we recommend calling this out visually. For example: This elegant scoring system provides a useful summary of the perceived quality and value of your mobile offerings and is suitable for presentation to management. More importantly, it provides you with an apples-to-apples comparison between your multiple digital and mobile investments an objective summary of the relative amount of delight you are able to provide your different audiences. And we haven t even begun to talk about the open-text responses you ll get! The second distinct advantage our solution provides is the ability to get into your users heads, figuratively speaking. Using a system like OpinionLab, you provide your users and audience a direct (and private) mechanism to communicate with you about their joys and frustrations. We emphasize private because of a phenomenon that many companies that have deployed this solution have started to notice: frustrated consumers, when given a direct feedback mechanism, appear less likely to complain in public forums such as Apple s App Store, on Twitter, or in blogs.
12 WHITE PAPER + MOBILE MEASUREMENT PAGE 12 How great is that? By providing your mobile audience an easy-to-use mechanism to give you feedback and vent when frustrated, you simultaneously gather great and useful insights and prevent potential PR and product-support challenges. We loved how simple OpinionLab was to deploy in our iphone application and mobile site, says Joe Megibow, Vice President of Global Analytics and Optimization at Expedia.com. But what was really great is that relative to traffic, we get five times more feedback from mobile users than we do on our traditional web properties. Perhaps the best news about Percent Positive Feedback and the collection of qualitative data in your digital channels is the potential for ubiquity. This paper s sponsor and other companies in the feedback market are working on making their solutions available across mobile, web, , and even offline channels. Having a single strategy tied to a single metric of success that spans all of your digital efforts is as close to the Holy Grail as we get in web analytics, although you won t be successful with one metric alone. In considering how you use the feedback you get, it s really up to you and your data-collection vendor. Some businesses have incredibly sophisticated text-mining algorithms designed to surface problems, while others simply focus on presenting the feedback in an easy-to-read fashion. At Web Analytics Demystified, we recommend a two-pronged approach that begins with a word cloud and then filters or categorizes feedback based on those results. A word cloud is a brilliant text-visualization strategy that simply sizes each word in a body of text based on the number of occurrences. For example, here is a Word cloud made from the text of this white paper: Figure 4: Sample word cloud created from the text of this white paper. The fact that this document is about mobile, feedback, applications, and interactions immediately stands out. Not only is the cloud a great way to present open-text feedback, it has the potential to help focus your analysis of the text by highlighting specific words of concern that may be part of an emerging pattern. Word clouds aren t the only visualization that can help focus analysis efforts.
13 WHITE PAPER + MOBILE MEASUREMENT PAGE 13 For example, here is OpinionLab s Circles visualization that categorizes each piece of feedback and allows for rapid drill-down into key themes and ideas: Figure 5: Example of categorization and category visualization provided by OpinionLab. The circle on the left provides visual cues similar to a word cloud allowing the analyst to immediately see what themes are dominating the open-text feedback collected. Regardless of how you choose to focus your efforts to move beyond a numerical analysis of sentiment, Web Analytics Demystified recommends ensuring that you have some methodology in place. The ideal situation is one where feedback can be collected, categorized, and then efficiently distributed within the business for follow-up (if necessary.) The best companies are already doing this, and we expect that vendors like OpinionLab and others will begin supporting moderately robust workflows for customer feedback in the very near future. Interactions As discussed earlier in this paper, page views and visits do not apply nearly as well to the mobile channel as some would hope. While they certainly can be applied to mobile web initiatives, comparing page-view counts browsed on large screens and small screens is likely an unfair comparison. When mobile applications are added, all bets are off: nobody visits an application on their phone; rather, they click a button and launch it. With this in mind, and with a goal of creating a truly useful metric that can span multiple connected channels, Web Analytics Demystified set about to define a measure that has true utility when presented in context. What we arrived upon is again surprisingly simple: a measure of the number of interactions users have with a given site or application, without regard for the actual mechanism through which that interaction is delivered. We refer to this measure simply as an interaction.
14 WHITE PAPER + MOBILE MEASUREMENT PAGE 14 Being a generic catch-all, interaction can be used to describe and measure: Visits to a mobile website (like m.expedia.com) Uses of a mobile application on any device Requests and responses to and from SMS Calls to a sales or customer support organization Face-to-face communication in stores The only truly important thing you need to keep in mind when measuring interactions is their definition, which is specific to the application, channel, device, or situation being measured. For example, a website interaction is traditionally referred to as a visit, which is usually defined as any number of page views bounded by 30 minutes of inactivity on the measured site. 10 Given the similarity between web and mobile sites, this definition is fine for use on the mobile web but breaks down completely for mobile applications. We suggest that for this channel, an interaction be defined as any measurable launch application and close application messages from the application. Other interactions include: SMS interactions, starting when a message is received and continuing through multiple responses until there is no further response for 30 minutes Calls, starting when the phone is answered and continuing for as long as the conversation lasts and both phones are hung up Face-to-face interactions, starting when two or more individuals begin a conversation and continuing until that conversation is complete and the parties walk away Note the flexibility inherent in what is otherwise simple semantics: by removing the explicit attempt to use web-analytics terms to describe actions not occurring on web pages, we create a framework where a single word can be used to describe multiple types of events. All that is required is A) reasonable mechanisms for measuring each type of interaction and B) agreement about the interaction s individual boundaries and definitions. When presenting interactions to the business, given the term s colloquial use, Web Analytics Demystified strongly recommends providing additional context whenever possible. For example, if you are presenting the number of interactions in your web, mobile web, and mobile applications, you might want to present that data like this: 10 Web Analytics Association,
15 WHITE PAPER + MOBILE MEASUREMENT PAGE 15 Another way to present interactions is as a percentage of all interactions measured, which simply gives the reader additional context. For example: For the first time business owners are now able to compare their efforts in fixed web and multiple mobile channels using the same criteria. The value of this type of comparison is immensely important for companies that are aggressively experimenting in mobile on multiple platforms and devices, although the value is perhaps tied more to the utility of the interaction measure than the measure itself. Engagement Rate Similar to the challenges facing web analytics professionals when attempting to work with the canonical page views measurement, the measure of interaction relies on context to become truly useful. While it s a good start to present interaction as a rate rather than a measure, as described above, we propose that our measure of interaction can also be used to determine the user s rate of engagement. Engagement is a complex term to be sure, one that seems to mean something different to almost everyone you ask. Web Analytics Demystified has provided a great deal of thought leadership on the measure of engagement in online channels, culminating in the publication of our white paper Measuring the Immeasurable: Visitor Engagement 11 in which we propose that engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals. Given the proceeding definition of interaction spanning multiple channels, our measure of engagement simply puts those interactions into meaningful business context. For example, if you spend thousands of dollars developing an Android application with multiple functions, a reasonable engagement goal is to have visitors interact with at least one or two of those functions. The same applies to a mobile web site, SMS campaign, call session, or traditional web site: in almost all instances, site owners have specific goals in mind for visitor and user interactions, and those goals can be used to measure engagement. 11 Measuring the Immeasurable: Visitor Engagement
16 WHITE PAPER + MOBILE MEASUREMENT PAGE 16 Other examples of how engagement can be scored in mobile channels include: Engaged visits can be counted after some number of page views, a set amount of time, or after specific interactions have been recorded; Engaged application uses can be counted after a set number of clicks or during any use where a specific functionality is executed; Engaged SMS sessions can be counted when the requestor clicks a link in the message response body; Engaged calls can be counted when the caller asks a question and the agent begins to provide an answer; Engaged face-to-face interactions can be counted when the customer exhibits specific psycho motor behaviors demonstrating attention (e.g., eyes focus, movement stops, conversation ensues, etc.). All of the specific acts measured in the examples above are the interactions we described in the previous section, so what we are really saying is that engaged users are those users completing a defined set of interactions specific to the channel they re interacting in. Given that this number will never be the same as the total number of interactions you record, the percentage of engaged interactions to all interactions can be thought of as your engagement rate. For example, a large retailer s engagement goal for their iphone application is to have users either A) view product details, B) search for a product and click on a search result, or C) browse the weekly specials content. The engagement rate for the application is therefore: Using basic segmentation tools, if this retailer were to discover that A or B or C happened during 100,000 interactions with their application, and the application had 1,000,000 total interactions during the same time frame, their engagement rate would be 10%. Presented in tabular form, engagement rate might look something like this:
17 WHITE PAPER + MOBILE MEASUREMENT PAGE 17 Armed with three essential measures of sentiment, interaction, and engagement, all that is left to create a robust multichannel reporting package for management is to tie all three metrics to their relative cost and value back to the business. Adding Costs and Revenue to the Essential Measures At least in the ideal world, all of our efforts, regardless of channel, are designed to lower costs while simultaneously increasing revenue. No matter if you re a retailer selling online, a travel site trying to use a mobile application to lower customer-support costs, a bank trying to facilitate mobile commerce, a media provider trying to increase ad-based revenue, or a business site trying to generate leads at the lowest possible cost, at the end of the day, it is always about optimizing your return on investment. While this paper doesn t explore the specific actions you would take in your own business, we are happy to share the single most useful thing we have ever discovered regarding reporting data to management: it needs to be about money, period. The best executives will understand the value of a high percentage of positive feedback, strong growth in your interaction rate, and a high level of user engagement; but all executives will understand the value of decreasing costs per interaction and increasing revenue per engaged user. So do that. Put your percent positive feedback, interaction, and engagement measures into the context of costs and revenues. The latter can be difficult, especially if you re not selling directly in a connected channel, but the former is as simple as coming to agreement on the costs associated with developing and supporting the web site, mobile site, or mobile application. For example, if your mobile app development costs were $100,000 plus an ongoing $10,000 per month for support with an expected lifespan of two years, then your 24-month amortized costs would be $340,000 ($100,000 + $120,000 per year for two years) or around $14,000 per month. If during the month you generate 30% positive feedback, have 80,000 interactions, 30,000 of which are counted as engaged then your associated costs would be: $466 per positive feedback percentage point $0.18 per interaction $0.47 per engaged interaction Considered in a vacuum, these amounts are hardly interesting and raise more questions than they provide answers. But, like everything else in analytics, the real value arises from presenting this data in context.
18 WHITE PAPER + MOBILE MEASUREMENT PAGE 18 Suddenly, the conversation with management is all about what is driving changes in costs across all digitally connected channels supported by the business, a decidedly different conversation than most analytics managers are having with senior leadership today. In our experience, this is the most important conversation to have because it leads to requests for analysis, budget for optimization efforts, and increased resources to correct areas of concern.
19 WHITE PAPER + MOBILE MEASUREMENT PAGE 19 SPECIFIC EXAMPLES While these are certainly still the early days of businesses measuring their mobile efforts in the context of their larger business, a handful of companies have already established themselves as category and thought leaders. Two of these companies a large retailer and Expedia.com are well-known and have a long history of using digital measurement to create competitive advantages. The third, a well-known Financial Services firm that has requested anonymity for competitive reasons, has recently started to really take advantage of both qualitative and quantitative insights gathered via small screens to guide the development of their current and future mobile offerings. For Online or Multi-Channel Retailers One retailer working with this paper s sponsor OpinionLab, while wishing to remain anonymous, is well known for their innovation, customer centricity, and leadership in the digital channel. Under the leadership of their Vice President of Emerging Platforms, the retailer has emerged as the company to watch when it comes to innovative uses of mobile technology. Having multiple mobile web sites, numerous mobile applications for iphone and Android, and a variety of other efforts including location-specific, QR codes, and SMS campaigns, the company is pushing what is possible in mobile and emerging channels. Our goal is to help customers shop, learn, and buy while mobile says their Vice President of Emerging Platforms. We want to create engaging experiences for our customers wherever they are, in our stores, in lines, on the couch, or in front of their computers. And by nearly all measures this company is succeeding their iphone application alone has been downloaded over 1,000,000 times and enjoys a high overall rating in Apple s online App Store and their mobile site and mobile applications are generating millions of dollars of revenue each year. All of these efforts, of course, require measurement, and on this point the company s Manager of Online Customer Experience recently oversaw the deployment of OpinionLab s mobile feedback technology on both their primary mobile site and within a handful of the company s mobile applications. We like OpinionLab for three reasons. First, it is incredibly easy to deploy across our multiple properties and applications. Second, it provides great consistency of measurement across these customer touch-points. And third, the open-text response model has proven critical to our ongoing development and optimization efforts. Supported by OpinionLab technology and the company s internal digital analytics team, the retailer s mobile teams are able to make increasingly tactical decisions about where the next opportunity lies. These insights allow the company to focus their efforts on those applications, sites, and technologies that contribute not only revenue but also interaction, engagement, and delight. According to the company s Vice President of Emerging Platforms, We will always innovate in emerging platforms. Our mobile analytics efforts allow us to be smart about where that innovation occurs and to visualize the intelligence the collective data suggests. For Online Travel Sites Few sites understand the power of the voice of the customer like Expedia.com. For years under the guidance of Joe Megibow, Vice President of Global Analytics and Optimization, the company has leveraged a variety of feedback and satisfaction solutions, including OpinionLab, to create a rich tapestry of customer likes and dislikes, all in an effort to remain the travel industry category leader. More recently, the company expanded heavily into mobile sites and applications, and consistent with their character, brought voice-of-customer measurement right along. We see providing a clear and useful feedback mechanism on our sites and in our applications as critical to doing our job right, says Mr. Megibow. Our customers expect a lot from us and will always let us know when we can do better. We owe it to them to make it easy to provide feedback, and more importantly, easy for the business to act upon.
20 WHITE PAPER + MOBILE MEASUREMENT PAGE 20 To this end, Mr. Megibow has deployed OpinionLab s feedback solution into the company s iphone application and mobile site. This simple inclusion in the application and on every page of the mobile site gives customers direct access to all of Expedia. Every comment is not only read by customer support channels, but by supply groups, product groups, marketing groups, and even senior leadership. To this end, Expedia has internally built a voice-of-customer workflow tool that ensures that feedback is correctly tagged, routed, and where possible, followed up on. According to Mr. Megibow, We loved how simple OpinionLab was to deploy in our iphone application and mobile site. But what was really great is that relative to traffic, we get five times more feedback from mobile users than we do on our traditional web properties. Feedback and insights captured via the company s TripAssist iphone application and m.expedia.com site have surfaced critical, actionable insights, and Expedia has already deployed changes based directly on their learnings. This feedback is far more actionable than the ratings that appear elsewhere in social media and online, says Mr. Megibow. For Financial Services Institutions The financial services sector has perhaps the most to gain from the continued expansion of mobile channels. Banking, insurance, credit, and investment firms have a tremendous opportunity to provide anywhere, anytime services to their customers, especially when those services can cut expensive, inefficient paper delivery. To this end, there are dozens, if not hundreds, of financial services firms currently providing and rolling out mobile sites and applications designed to allow customers to check balances, make payments and transfers, apply for loans, and more. One OpinionLab financial services customer has already implemented many of the recommendations made in this white paper and is actively tracking both qualitative feedback and engagement across both their fixed web and mobile channels. Early on, an analysis of the company s integrated OpinionLab and Omniture dataset highlighted mobile visitors using the fixed web site and providing feedback on the need for a mobile-specific site. The mobile group s manager told Web Analytics Demystified that the verbatim feedback was very clear, and since the company was already adept at using qualitative data to prioritize investments, the mobile initiative was launched. Now, roughly two years later, the company has a robust mobile offering that is yielding great insights about the needs of mobile visitors. Not only do we get a dramatically higher rate of feedback per visitor in the mobile channel, said the group s manager, we also find that over 60 percent of raters using the OpinionLab service also provide valuable direct feedback compared to only five percent providing feedback on our traditional site. Given that engagement is tremendously important for the company and that their definition is nearly identical to the metric we proposed earlier in this paper having direct feedback is invaluable in helping them create more valuable customer interactions and drive engagement. By focusing on creating engaging experiences for small screens and ensuring that customers are able to complete on the go activities, the company will continue to provide anywhere, anytime service and hopefully increase positive sentiment about the applications, sites, and entire corporate brand.