Data-DrivenDesign. DigitalExperienceTeamsAreFocusedonWebsite MetricsThatDon tdemonstratebusinessvalue

Transcription

1 Data-DrivenDesign DigitalExperienceTeamsAreFocusedonWebsite MetricsThatDon tdemonstratebusinessvalue

2 A Forrester Consulting Thought Leadership Paper Commissioned By Extractable Digital Experience Teams Are Focused On Website Metrics That Don t Demonstrate Business Value April 2012

3 Table Of Contents Executive Summary... 2 The Current State Of... 3 Design Improvements Are Not Focused Enough On Business Results... 5 Marketers Are Missing Key Tools And Skills... 7 Key Recommendations: Leverage Partners To Adopt...11 Appendix A: Methodology...12 Appendix B: Demographics/Data...12 Appendix C: Endnotes , Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. For additional information, go to [1-JCQKOA] About Forrester Consulting Forrester Consulting provides independent and objective research-based consulting to help leaders succeed in their organizations. Ranging in scope from a short strategy session to custom projects, Forrester s Consulting services connect you directly with research analysts who apply expert insight to your specific business challenges. For more information, visit Page 1

4 Executive Summary We ve entered the age of the customer an era where a focus on customers matters more than any other strategic imperative. Companies are waking up to the fact that customers perceptions have a profound impact on business metrics ranging from brand equity and customer loyalty to increased revenue and cost savings. But while every executive knows that customers matter, most companies simply don t approach their interactions with customers in a disciplined way. Firms work to improve the experiences they provide their customers but lack the tools and processes needed to do this thoroughly and consistently. This is particularly true with digital touchpoints, where rapid proliferation has increased the need to adopt tools and processes that allow digital marketers to better design, optimize, and measure experiences that meet customers needs and drive their business goals. Marketers have to not only deliver experiences across touchpoints but also deliver those experiences rapidly in order to keep pace with the changing nature of those touchpoints. In September 2011, Extractable commissioned Forrester Consulting to study the adoption of and benefits and challenges associated with data-driven design processes. Forrester s surveyed 209 marketer, ebusiness, and customer experience professionals tasked with designing and improving their firms digital experiences. As suspected, most firms don t adhere to a structured data-driven design process. Forrester instead asked them about the tools and techniques used in their digital design and optimization tasks and the barriers that prevent them from adopting a more structured design process. The result is a lack of process and a focus on select quantitative tools that stems from organizational and business challenges related to how they measure success of digital touchpoints. Key Findings Forrester s study yielded three key findings: Firms lack a coherent set of design and optimization tools and techniques. Firms need a portfolio of tools to evaluate and improve the experiences they provide to their customers. 1 This includes quantitative and qualitative tools, tools for research and testing, tools that monitor the experience over a period of time, and those that test specific interactions in greater depth. While survey respondents reported using a breadth of techniques to design and optimize their websites, when asked specifically about which techniques they use, Forrester uncovered a preference for evaluative techniques methodologies that are typically used to measure existing interfaces as opposed to ones that identify drivers of behaviors. 2 Sites owners are overly focused on misleading metrics. Websites are often measured on traffic, brand recognition, and time spent on the site not metrics that alone could demonstrate how well the site is performing for the business or its customers. It makes sense then that the tools teams put in place help them monitor and drive these metrics as opposed to evaluate opportunities to improve more complex metrics like conversion rate or quality of leads. Digital teams need partners that complement their skills within their budget. Teams report that although they want to improve their digital experiences and are frustrated with the tools at their disposal today, they face challenges, including a lack of understanding, budget, and skill to implement techniques that may provide more value. This presents a clear opportunity for user experience partners to help round out team skill sets and encourage adherence to a process that improves the quality of digital touchpoints as well as the timeline and budget with which improvement projects are delivered. Page 2

5 The Current State Of Creating and maintaining a successful digital experience that drives business results requires the right research insight, design, technology, and ongoing optimization. A data-driven design process helps codify those key activities into a repeatable set of tasks that firms can use to create and maintain their website, mobile or tablet site, application, or any other digital experience. It typically begins with deep research so that teams can understand users, the business, and the current environment. Using that data, they can design the best possible experience for users and deliver improved and measurable outcomes for the business. Finally, to ensure a site s ongoing viability, a data-driven process requires the definition of appropriate and actionable behavioral and outcome analytics to continually optimize the user experience. To understand the current research tools and design processes that companies use, Forrester conducted an indepth survey of 209 marketer, ebusiness, and customer experience professionals who oversee the design, optimization, and measurement of their company s digital touchpoints. Our survey found that: Sixty percent have seen their website improve due to use of data. More than half of all companies surveyed have seen improvement in their websites due to the use of user data. When we look solely at the firms that say they have a repeatable design process, 71% of those firms report seeing an improvement in their sites due to the use of user data. Similarly, when we look at just those firms that report regularly consulting multiple sources of data, again, 71% report seeing an improvement in their sites due to the use of user data. These results imply that while a repeatable design process and use of multiple sources of data improve output for many firms, almost 30% of firms still aren t getting the most out of their design process or the various sources of data they gather. Three-quarters of firms are looking for ways to optimize their website(s). Most firms launch a website or mobile website and make incremental additions to it over a period of several years. Forrester s research shows that this process slowly deteriorates the quality of their site, often resulting in diminishing brand or business metrics. 3 Not surprisingly, 75% of our respondents reported that their teams are continually looking for ways to optimize their websites (see Figure 1). More than half report regularly consulting multiple sources of data. Fifty-seven percent of firms report that they regularly use multiple sources of data in their design and optimization process. This, according to them, includes qualitative and quantitative user data, competitive market data, business and strategic data, and data from testing. But just more than one-quarter are satisfied with the tools they use today. Despite consulting multiple sources of data, only 28% of firms reported being truly satisfied with the tools they use today. As high as 52% believe that there are other tools that could provide them with better insight than they currently get. In the case of tools like ethnography, social media monitoring, web satisfaction measurement, eye tracking, as well as website session recording and playback, it s because most feel that the techniques could provide new information that they do not have today. With techniques like user interviews, usability lab tests, focus groups, card sorting, and behavioral analytics, it s more likely because they feel that the technique could provide better quality of information than what they gather today (see Figure 2). Just more than half have a systematic and repeatable design process. Despite the fact that many firms are looking for ways to optimize their websites, only 52% report having a systematic and repeatable design process. Twenty-three percent feel that their design process is not systematic and repeatable. This indicates Page 3

6 that nearly half of companies responding either do not feel that a design process is a good way to optimize their website or have trouble implementing such a process. Figure 1 The Current State Of How would you rate the following statements? (On a scale of 1 [strongly disagree] to 5 [strongly agree]) Disagree Strongly disagree Agree Strongly agree We are continually looking for ways to optimize our website(s) We regularly use multiple sources of data in our website design and optimization process Our website has improved due to systematic use of user data There are other tools available that could provide us with better or more information than we are able to get today Our design process is systematic and repeatable We are satisfied with the tools we have at our disposal today We spend more time measuring the success of past website changes than planning new improvements -6% -4% 31% -3% -8% -100% -75% -50% -25% 0% 25% 50% 75% 100% -17% -8% -8% -5% -6% -28% -35% -9% -15% -17% 31% 34% 39% 24% 42% 11% 1% 4% 44% 26% 18% 13% 18% Figure 2 There Are Other Tools That Could Provide Better Insight You indicated that your company does not use but is considering or wants to use the following tools and techniques in the website design and optimization process. What is the primary reason why you want to use these tools and techniques? Feel it could provide new information we do not have today Feel it could provide better-quality information than we gather today Ethnography (observation, diary studies, cocreation) Website session record/playback tools Eye tracking studies Social media insight/monitoring ROI models A/B and multivariate testing tools Behavioral segmentation (design personas) Clearly defined key performance indicators (KPIs) Website satisfaction (or other experience quality) measurement tools Customer journey maps Metrics dashboards Expert evaluations Market segmentation On-site keyword search term analysis Search engine search term analysis User interviews Focus groups Usability lab testing (remote or in-person) Generic online survey and customer feedback tools 72% 62% 60% 60% 58% 57% 56% 54% 54% 52% 51% 51% 51% 50% 48% 45% 44% 43% 39% 28% 38% 40% 40% 42% 43% 44% 46% 46% 48% 49% 49% 49% 50% 52% 55% 56% 57% 61% 0% 25% 50% 75% 100% Base: Digital experience professionals who are considering or want to use each tool/technique N = Page 4

7 Design Improvements Are Not Focused Enough On Business Results Our survey indicates that teams are irregular at best in their use of multiple sources of data and a repeatable process. Why? To learn more, we asked site owners about their site optimization goals and challenges. What we found indicated that problems stemmed not just from a lack of data or consistent design process but also from business and organization issues. Respondents told us that: Sites are primarily measured on metrics that don t clearly demonstrate business value. Two-thirds (66%) of teams report that they measure their site on traffic (impressions), and nearly half said that they do so on brand recognition (47%) and average time spent on the site (46%) (see Figure 3). It s surprising that those metrics top the list because alone they don t demonstrate the value a site might provide to either customers or the business. Simply driving more traffic to a site, unless that site is supported solely by advertising, doesn t indicate the quality of the traffic, audience, or leads. Similarly, average time spent on the site doesn t always indicate whether visitors to the site are having a positive experience. Instead, it could be a sign that users are lost on the site while trying to find value. Finally, while brand recognition, too, is not a bad metric to measure on a site, it doesn t indicate the degree to which the site reinforces the key brand attributes and is a metric that tends to lag behind the current site experience. But more informative metrics are not far behind. Although sites are primarily measured on metrics that don t demonstrate business value, those metrics are followed closely by those that do indicating that a change in the way sites are valued may not be far behind. Metrics like customer satisfaction (45%), conversion rate (43%), and leads (42%) that demonstrate how well a site is delivering on user and business goals tend to be more informative when considering site performance and improvements. Teams are goaled on speed and budget. When we asked what metrics the teams themselves were measured on, the ability to complete a project on time or faster and under budget topped our list at 65% and 64%, respectively, followed closely by the ability to bring to market new ideas at 62% (see Figure 4).When time and budget are the primary concerns, it can be tempting to cut corners on the research and testing phases of a design process especially when the goal for the site is to drive quantity and stickiness as opposed to quality of lead and ability for users to complete their goals. Stakeholders don t take existing data into consideration. Site owners also highlighted some of their key challenges with their existing site design and optimization processes. More than one-third reported that key stakeholders didn t take into consideration the data they uncover (37%), and, closely related, they said that they gather data but do not use it (34%) (see Figure 5). In other words, the data they are collecting gets wasted. Anecdotally, teams tell us often that they struggle to be more customer-centric and not overly focused on the highest paid person s opinion. 4 When considering new tools, the focus is market leadership above all else. What do teams consider when evaluating new tools, techniques, or changes to their design processes? Oddly, their primary concern is neither how well the new tool or technique will affect the performance of their website or team nor the time or cost it will take to implement. Rather they re primarily concerned with the extent to which the technique will make the team market leading () something that is difficult to measure and has no bearing on the potential for the tool s success (see Figure 6). For example, 66% of respondents indicate that they use social media insight and monitoring that s more than 20 percentage points more than those using user interviews. While both techniques help gather first-person feedback or insights from customers, social Page 5

8 media monitoring is overly focused on individuals who are active online (and index high in their social behaviors), an attribute that isn t always relevant to every company. Figure 3 The Top Three Most Common Site Metrics Don t Demonstrate Business Value What website metrics are you and your team measured on? Traffic/impressions Brand recognition Average time spent on-site Customer satisfaction Conversion rate Sales leads Social media "likes" or followers Sales (online or offline) Sign-ups for trials, downloads, or test accounts Likelihood to recommend Sales assists/lead nurturing Average order value Customer self-service Task completion rates Service costs Other None 22% 21% 14% 13% 12% 8% (multiple responses accepted) 1% 47% 46% 45% 43% 42% 37% 34% 66% Figure 4 Teams Are Goaled On Speed And Budget What team performance metrics are you and your team measured on? Our ability to complete projects on time or faster Our ability to complete projects on or under budget Our ability to bring to market new ideas Our ability to meet project specifications 65% 64% 62% 56% Our ability to improve internal processes 49% The evaluation of our team performance by other teams 26% Other None 2% 5% (multiple responses accepted) Page 6

9 Figure 5 Stakeholders Don t Take Existing Data Into Consideration What are your biggest pain points with your existing website design and optimization process? Key stakeholders don't take into consideration the data we uncover We gather data but do not use it 34% 37% We don't have the right data to make decisions 28% We have multiple, conflicting sources of data 24% We have so much data we can't make decisions We have no pain points with our existing website design and optimization process Other 11% 11% 21% Not sure (multiple responses accepted) 3% Figure 6 When Considering New Tools, The Focus Is Market Leadership When considering new tools, techniques, or changes to your website design and optimization process, which of the following is the most important to you and your team? The extent to which the technique will make us market leading The cost of the tool, technique, or process change The ability for the tool or technique considered to address a specific problem we're having 19% 20% Our ability to affect the performance metrics our team is measured on 13% Our ability to affect the metrics our team website is measured on The availability of training, professional services, or agency partners to maximize our use of the tool, technique, or process change 9% 10% Other 1% Don't know/none of the above 5% Marketers Are Missing Key Tools And Skills Most site owners report using a balanced set of tools and techniques in their website design and optimization process. But their sites are goaled on metrics that aren t well aligned with driving business value, and teams are encouraged to minimize cost and complete projects more quickly. As a result, they favor methodologies that uncover quick hits over those that help sites evolve to better experiences which of course require an investment in skills, time, or budget that they may not or no longer have today. Survey responses reveal that: Page 7

10 Teams say that they use a balanced set of research techniques. Survey respondents tell us that they use a variety of data sources as input to their design process. Fifty-three percent report regularly using qualitative data, 64% regularly use quantitative data, 59% regularly use competitive market data, 47% regularly use testing data, and 59% regularly use their internal vision and road map to guide site optimization (see Figure 7). But in reality they ignore major qualitative research and testing methodologies. Many firms report that they don t use techniques like card sorting (46%), ethnography (44%), website session recording and playback tools (45%), eye tracking studies (44%), and usability lab testing (30%) (see Figure 8). 5 Instead, they focus on techniques like search engine search term analysis (75%), behavioral web analytics (72%), and online survey and feedback tools (67%) (see Figure 9). While these tools are beneficial, they focus more on the behavior users exhibit on websites today and the immediate breakdowns in that existing experience. They are less focused on identifying drivers of that behavior in order to drive long-term innovation. These techniques that firms rely on are reports churned out by systems that require one-time implementation. The ones they ignore require investment of time and resources to execute and analyze each time the technique is used and, as a result, provide different results. Most see some value in more strategic qualitative methodologies. Notably missing from the list of frequently used and not currently used methodologies were strategic tools like customer journey maps and behavioral segmentation (personas). But they did top the list of techniques that firms were currently considering using in their website design and optimization process. Thirty-two percent of firms are considering customer journey maps, while 27% are considering behavioral segmentations. Twenty percent and 32% of firms, respectively, are already leveraging these techniques today (see Figure 10). But most don t have the skills or budget to leverage qualitative tools. When asked the primary reasons why they don t use techniques like focus groups, usability lab testing, eye tracking, and tools like website session recording and playback, personas, and customer journey maps, firms cited a variety of reasons. Some felt that those techniques would not be useful, but most also cited lack of internal infrastructure, budget, and skills among their concerns (see Figure 11). Figure 7 Teams Claim That They Use A Balanced Set Of Research Techniques To what degree would you say you use the following types of data in your website design and optimization process? (On a scale of 1 [rarely use] to 5 [regularly use]) Occasionally use Rarely use Frequently use Regularly use Quantitative user data -6% -8% 26% 38% Competitive market data -6% -11% 31% 28% Internal vision/road map -4% -12% 36% Qualitative user data -8% -8% 26% 27% Testing -9% -19% 20% 27% -50% -25% 0% 25% 50% 75% 100% Page 8

11 Figure 8 Firms Largely Neglect Qualitative Research Top 10 tools and techniques that respondents do not use in the website design and optimization process N = Card sorting Ethnography Website session record/playback tools Eye tracking studies 46% 44% 44% 44% Usability lab testing Website satisfaction measurement tools Behavioral segmentation Customer journey maps A/B and multivariate testing tools Focus groups 30% 28% 25% 22% (multiple responses accepted) Figure 9 Search Term Analysis, Analytics, And Surveys Are The Most Frequently Used Tools Top 10 tools and techniques that respondents use in the website design and optimization process Search engine search term analysis 75% N = 156 Behavioral web analytics Generic online survey and customer feedback tools Social media insight/monitoring On-site keyword search term analysis Market segmentation Metrics dashboards Clearly defined key performance indicators (KPIs) User interviews (multiple responses accepted) 44% 72% 67% 66% 65% 61% 56% 54% Page 9

12 Figure 10 Journey Maps, Behavioral Segmentation And User Interviews Are Being Considered Top 10 tools and techniques that respondents are considering using in the website design and optimization process N = Customer journey maps Behavioral segmentation User interviews Social media insight/monitoring Website satisfaction measurement tools ROI models A/B and multivariate testing tools Expert evaluations Ethnography Focus groups 27% 22% 21% 21% 21% (multiple responses accepted) 32% Figure 11 Respondents Don t Have The Skill Or Budget To Leverage Many Techniques You indicated that your company does not use the following tools and techniques in the website design and optimization process. What is the primary reason why you do not use these tools and techniques? Do not think it would be useful Get the same data by other means Lack of infrastructure, skills, or too expensive Focus groups 20% 9% Stakeholders don't see the value 51% 11% N = 45 Usability lab testing 21% 2% 47% 8% 62 Eye tracking studies 27% 1% 45% 9% 91 Website session record/playback tools 18% 4% 43% 6% 93 A/B and multivariate testing tools 24% 4% 34% 10% 49 Behavioral segmentation 33% 13% 32% 12% 52 Customer journey maps 18% 2% 32% 10% 49 0% 25% 50% 75% 100% Base: Digital experience professionals who do not use each tool/technique Page 10

13 KEY RECOMMENDATIONS: LEVERAGE PARTNERS TO ADOPT DATA-DRIVEN DESIGN Firms tell us that they re constantly looking for ways to improve their websites. As digital touchpoints proliferate to mobile devices, tablets, and beyond, the need becomes even more pressing. In order to effectively understand user needs and frustrations, monitor existing designs, and identify opportunities for improvements, firms need to use a variety of tools and apply them in a consistent manner. To overcome existing challenges with the data-driven design process, firms should: Align website and team metrics to business goals. Metrics like impressions and average time spent on the site fall down in two ways. First, they don t indicate or drive the behaviors that firms are actually after: greater conversion, more leads, or larger share of wallet or cart size. They also fail in helping site owners optimize the experience they provide because they simply measure performance and don t give insight into how to drive future behavior. Good website customer experience metrics fulfill two purposes tracking overall site experience quality and pointing to opportunities for improvement. In order to support their efforts more directly, firms should prioritize three types of metrics: 1) measures of how customers perceive their visits to the site; 2) events that signal potential problems even if customers don t complain; and 3) data about what customers do or intend to do after the visit that can help show how experience quality affects the business. 6 Select tools and techniques based on their ability to address both strategic and tactical problems. Our survey respondents reported an unsurprising trend: When it comes to selecting website optimization techniques, firms chase the shiny new object. Unfortunately, this leaves a long list of tools and techniques that firms have tried, didn t find useful, and then discarded. In some cases, this is because the tool wasn t helpful. In other cases, it s because the firm didn t have the skills or knowledge to leverage the technique properly. While firms should consider new techniques, it s more critical to ensure their tool kit consists of a breadth of techniques that are exploratory, evolutionary, and evaluative in nature. 7 Techniques of each type help firms ensure that they re uncovering relevant customer needs and evolving experiences to adapt to new technologies, while identifying and improving current interfaces. Partner with vendors that complement your team skills. Teams report that their biggest challenges in broadening their use of research and optimization techniques are budget and skills. This is the case if they try to implement a tool or technique themselves. Instead, they should look to partner with agencies or user experience firms that offer these services to a broad range of clients. The advantage? It leverages the partner s expertise in the techniques, a track record of success with similar efforts, and the scale an agency is able to provide its clients. This helps ensure that the tool is used in the right way, to maximum efficiency, and raises the likelihood that it s implemented in the most cost-efficient way. Looking for an agency that specializes in techniques that your team has not already implemented also helps ensure that you re hiring a complementary set of skills as opposed to one that overlaps closely to what your firm already staffs in-house. Page 11

14 Appendix A: Methodology In this study, Forrester conducted an online survey of 209 digital experience professionals in the US to evaluate current practices around website monitoring and digital experiences. Survey participants included marketer, ebusiness, and customer experience professionals tasked with designing and improving their firms digital experiences. Questions provided to the participants asked what tools and metrics they used to monitor their websites. Respondents were offered rewards points as a thank you for time spent on the survey. The study began in September 2011 and was completed in October Appendix B: Demographics/Data Figure 12 Company And Team Size 20,000-plus employees 24% 5,000 to 19,999 employees 18% How many employees work for your company worldwide? 250 to 999 employees 27% 1,000 to 4,999 employees 31% How many full-time equivalents are on your team including you? I am the only person on my team or more 7% 7% 9% 18% 29% 30% Figure 13 Respondents Were Mostly Managers And Directors Which of the following best describes your job level? Executive management 3% VP 9% Which of the following most closely describes the team you're on? Marketing Other, please specify 7% 79% Manager 51% Director 36% ebusiness/ecommerce Corporate communications/pr IT or web development Customer experience or user experience Market research or customer intelligence Customer service 4% 3% 2% 1% 1% 1% Page 12

15 Figure 14 Respondents Were Mostly From Technology, Industrial Products, And Media And Entertainment Verticals Which of the following best describes your company s industry? Technology (software, consumer electronics, online services, etc.) Industrial products 12% Media and entertainment 9% Professional services 9% Healthcare, pharmaceuticals, and life sciences 8% Financial services 8% Retail 6% Travel, leisure, and hospitality 5% Consumer packaged goods 5% Education 4% Nonprofit 3% Transportation 2% Telecom carriers 2% Government 2% Other 6% 15% (percentages do not total 100 because of rounding) Figure 15 Respondents Were Primarily From B2B Firms To whom does your company sell primarily? Mostly consumers 26% Mostly businesses 59% Consumers and businesses roughly equally 14% (percentages do not total 100 because of rounding) Page 13

16 Appendix C: Endnotes 1 Evaluating the effectiveness of these interaction channels requires a portfolio of tools, each with strengths and weaknesses in terms of the types of insight they provide, the resources and skills they require, and their usefulness across channels. Source: Companies Need A Portfolio Of Tools To Evaluate The Customer Experience, Forrester Research, Inc., June 2, Research techniques, while plentiful, fall into three key buckets. Exploratory research methods uncover behaviors, emotions, and desire that can seed new concepts. Evolutionary research methods can help define or validate a concept. Evaluative methods help improve existing interfaces. Source: How To Craft Your Customer Research Plan, Forrester Research, Inc., December 20, Clients frequently ask Forrester how often they should redesign their Web site. The answer? Redesign your site when it no longer supports business goals, brand attributes, or user goals. How will site owners know that their sites fall short in one of these three missions? Forrester recommends that customer experience professionals watch for one or more of seven indicators that it s time for an online overhaul. Source: When To Redesign Your Site: Seven Indicators That It May Be Time For An Online Overhaul, Forrester Research, Inc., August 17, Design personas are tools that organizations use to create consensus around who they re trying to serve. Without them, stakeholders often mistake themselves as the customer and input opinions based on their personal preferences and needs. Source: Executive Q&A: Design Personas In 2011, Forrester Research, Inc., February 22, While it s not necessary that firms use every qualitative research technique, select techniques can be helpful regardless of project to determine drivers of customer behavior and design solutions that support that behavior as opposed to solutions that simply mirror marketing objectives. 6 For more guidance on identifying the right web customer experience success metrics, see Forrester s recent research report. Source: What Are The Right Web Customer Experience Success Metrics? Forrester Research, Inc., July 20, Source: How To Craft Your Customer Research Plan, Forrester Research, Inc., December 20, Page 14