The Three Pillars of Web Performance Monitoring: How to measure the business and

Transcription

1 The Three Pillars of Web Performance Monitoring: How to measure the business and operational impact of web performance SPRING 2009 Executive Summary The performance of a website may be the most important aspect of its success because if the store is not open people cannot browse, buy and book. As the Internet matures, organizations rely more and more on the web for businessand mission-critical processes, and users are demanding faster, richer and more consistent online experiences. These factors put increased pressure on business, IT and web development teams responsible for creating and managing web applications. Key challenges include: Ensuring your most important web pages and transactions perform properly Protecting every dollar of revenue Rapid identification and notification of problems impacting your customers Accelerating troubleshooting and root cause analysis Ensuring that all of your customers have quality web experiences Today s web 2.0 applications are more feature-rich and complex than ever. Applications have become composite, meaning that their multiple elements assemble for the first timewithin the end-user s browser. For instance, some of these services come from within your datacenter; some are delivered through content delivery networks (CDNs) and others from 3rd parties providing services and content such as ratings and reviews, ads, newsfeeds, ecommerce, and more. In fact, the average web application consists of 6 separate services each of which impacts end-user experience, site availability and response time. So how can businesses thrive in this environment and have the same control over user experience online as store managers have over the experiences of visitors at brick and mortar locations? The answer is by monitoring performance from the outside-in.

2 The 3 pillars of outside-in monitoring include: Active monitoring from the internet backbone Active monitoring from the last mile Passive monitoring from actual end users Applying the three monitoring pillars delivers a comprehensive and realistic view of site operations and their impact on business success. These three monitoring approaches are distinct, complementary, and equally important, each addressing core challenges you face. According to The Aberdeen Group, companies using outside-in web performance monitoring: Improved application response times by 50% Reduced mean time to repair performance issues by 42% Reduced labor cost to manage application performance by 69% Improved customer satisfaction by 43% Outside-in monitoring has the additional benefit of being delivered as a software-as-a-service. This means that there is no hardware to install, licenses to pay for and no agents to deploy. It is fast and easy to implement, with payback measured in weeks. This whitepaper addresses best practices for web performance monitoring, and provides guidance on selecting the best approach for your business. The information contained in this document represents the current view of Gomez, Inc. on the issues discussed as of the date of publication. Because Gomez must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Gomez, and Gomez or its respective suppliers cannot guarantee the accuracy of any information presented after the date of publication. This white paper is for informational purposes only. GOMEZ AND ITS RESPECTIVE SUPPLIERS MAKE NO WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Gomez, Inc. Gomez may have patents, patent applications, trademarks, copyrights or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Gomez, the furnishing of this document does not give you any license to these patents, trademarks, copyrights or other intellectual property Gomez, Inc. All rights reserved. Gomez and the Gomez logo are registered service marks of Gomez, Inc. in the United States and/or other countries. The names of actual companies and products mentioned herein are the trademarks or service marks of their respective owners. Gomez, Inc., 10 Maguire Road, Lexington, MA USA This document contains information of a proprietary nature. All information contained herein shall be kept in confidence and shall be for the original recipient s use only. Any unauthorized reproduction by any other party shall constitute an infringement of copyright. 2

3 How can you monitor web performance from the outside-in? The two basic approaches to outside-in web performance monitoring are: Active/Synthetic Monitoring which uses geographically dispersed agents to collect performance data from scheduled tests that simulate the way users interact with websites and view video content. Passive Monitoring/Real User Monitoring which collects data on real users actual experiences and behaviors as they interact with web pages and videos. Now, more than ever, you need website insight from your customers perspective: the outside-in In a challenging economy, every organization needs to do more with less: more work with fewer employees, better web performance with less infrastructure investment, more revenue with fewer dollars spent. The need to protect every source or revenue is especially paramount and certainly so for your most productive online channels. Correctly executed, your website represents one of the most cost-efficient ways to connect with customers. But even as the economy declines, customer expectations are rising. And the web applications that customers take for granted are becoming more complicated to build, test, monitor, and manage. Any compromise in web performance has a real impact on your business. Consider: It s seven times more expensive to add a new customer than retain an existing one you can t afford dissatisfaction with your website experience 1 Every second of response time delay decreases conversions by seven percent 2 Web-based customer support averages just $0.10 per incident that same request costs an average of $5.50 when handled by a call center; any down time that increases call center volume dramatically increases costs 3 And with competitors eager to scoop market share, no one can risk compromises in web performance that lead to damages in brand reputation. To protect customers, look outside of your firewall This whitepaper is all about the most important perspective on your website: your customers. Internal monitoring of your IT infrastructure web, database and application servers, load balancers, storage and more is very important but it is also not enough and often misleading. Your internal systems view may show a green light even as customers are experiencing failures with the fundamental activities that drive your business: placing orders, making purchases, checking account balances and more. This is especially damaging if your competitors are outperforming you; if your site i s failing to meet customer needs and expectations, they can switch to a competitor s with a mouse click. That s why it s essential that you look at your website the way customers (the source of your revenue) do from the outside-in. Your goal? To understand web performance in a way that reveals the real status of your site and its business impact by collecting data along all points on the information highway. By looking from the outside-in, you see all of the variables that contribute to your customers web experiences including content and 1 Bain Consulting 2 Aberdeen Research, The Performance of Web Applications, November Forrester Research, Need to Cut Costs? Improve the Web Site Experience, December

4 functionality from your datacenter and all third-party partners or vendors such as CDNs, advertising networks, web analytics providers, cloud-based services, etc. In, we ll examine the following three crucial monitoring perspectives: Active web performance monitoring from the Internet backbone Active web performance monitoring from the last mile Passive web performance monitoring from actual end-users Each of these monitoring approaches can reveal important information about your site. By combining these three methodologies into one monitoring strategy, you achieve a whole perspective greater than the sum of its parts insights that can help you diagnose issues, improve web performance and, ultimately, drive operational and business success. When to use Active Backbone Monitoring: To be immediately alerted when key pages or transactions are slow or unavailable and diagnose root causes before customers are impacted To continuously monitor availability and end-to-end performance across key geographies To measure Service Level Agreements (SLA) with vendors To benchmark performance against key competitors Cruising the Major Highways: Active Web Performance Monitoring from the Internet Backbone Active monitoring, also known as operational monitoring and/or synthetic monitoring, ensures your key business pages and processes perform properly by running scheduled tests or transactions 24 hours a day, from the locations you choose. Backbone refers to the first level of Internet activity outside your firewall, the Tier 1 connections located at the major hubs. The Backbone Perspective: Active Backbone Monitoring gives you precise data on availability (your site s accessibility to users) and response time how long it takes your site to fulfill a user s request from their perspective, not from inside your datacenter. In addition, it enables you to: Accelerate problem identification with early awareness: active testing finds problems and notifies you before your end-users complain Trouble-shoot precisely: even as users sleep, active tests can quickly isolate performance issues and their root causes to solve problems before they escalate when users come online Protect critical business processes: active tests ensure that your most important pages and transactions perform properly, even when user traffic is low Benchmark the competition: with active testing you can set benchmarks of your site s performance that can be compared to your competitors The key advantage of Active Backbone Monitoring is control. Of the three methodologies, Active Backbone Monitoring is the closest to laboratory conditions: fast, commercial-grade servers and Internet connections are used, and you can determine what s monitored, where it s monitored from, and when. By using the same active traffic repeatedly, you get a clean data set that allows you to compare apples to apples over time. 4

5 What Remains to be Seen: The clean room control is what makes Active Backbone great for operational monitoring. But just outside of that clean room are your real customers who don t connect to your web applications through datacenter nodes; they connect through residential-grade PCs connected to local ISPs. Their experiences are conditioned by factors such as geographical location, ISP, connection speed and web browsers that remain invisible to backbone monitoring. Active Backbone Monitoring is a great first step, but when you rely on it in isolation, you risk missing performance issues that are impacting your real customers experiences at the edge of the Internet. When to use Active Last Mile Monitoring? To find issues in specific geographies or with specific regional ISPs To measure the performance of 3rd party providers (such as CDNs) in key markets, at the edge of the network To identify issues with specific customer segments such as broadband or dial-up users To measure performance from the perspective of end-users the source of your revenue Exploring the Byways: Active Web Performance Monitoring from the Last Mile Like active monitoring from the Internet backbone, active monitoring from the last mile applies synthetic traffic to test your site. But now you re shifting your perspective from the lab-like environment of the backbone toward a viewpoint at the edge of the Internet. In addition to regularly generating activity to your website from hundreds of backbone, commercial-grade servers, you select from tens of thousands of consumer-grade desktops across the globe to generate this activity. With this last mile approach, you monitor your site from a huge variety of real-world PCs, network configurations and geographic regions, generating information that s much closer to your customers true experiences. This type of monitoring is impossible to do any other way. Last Mile Perspective: Active Last Mile Monitoring couples the benefits of controlled monitoring with insight into the native habitat found on users computers. This desktop perspective helps you protect customer experience and measure differences in the quality of service you deliver around the world in the following ways: Identify local problems: finds and notifies you of potential issues in local and regional geographies Troubleshoot ISPs: uncovers peering and other issues with local and regional ISPs that may negatively impact users Objectively measure CDN performance: quantifies the benefits that different CDNs provide in your key geographies. By incorporating these and other local factors, you can validate the reach of your applications. Active Last Mile Monitoring allows you to weigh the impact of application, infrastructure, hardware and content acceleration investments on your ability to drive performance at the end of the delivery chain. 5

6 Taking the Customer s Perspective Taken together, the two active approaches to performance monitoring backbone and last mile give you an effective way to measure availability, response time and consistency in a controlled, automated fashion from outside your firewall. But to fully ensure your users are getting a quality experience, you also need to see through your real users eyes to expose the actual experience of your site. This experiential monitoring takes a step beyond operational monitoring and answers questions like: How does application performance vary by different user types (e.g. IE7 vs. Firefox 3.0 users)? Did the application change made last month improve user satisfaction?, and How did that performance issue that lasted for 4 hours last Monday affect our business? That s where passive web performance monitoring from actual end-users discussed in the following section comes in. While active monitoring provides operational data on site performance and availability, passive monitoring takes that next step and collects performance data on real users actual experiences and behaviors and gives you an extra dimension: information on the consequences and business impact of compromised performance. When to use Passive Real User Monitoring? To monitor real-world application performance by location, browser, OS, etc. To measure the delivery of and experiences with Rich Internet Applications To focus troubleshooting efforts based on the greatest impact to customers and business To ensure key customer segments can complete transactions and are satisfied Bringing It All Back Home: Passive Web Performance Monitoring from Actual End Users With Passive Monitoring from actual end-users, you collect data reflecting actual website activity, not synthetic transactions. The approach is similar to web analytics tools like Omniture or Google analytics; you place a tag on the pages you want to monitor and as real users interact with your site, you collect data about technical performance of their activities. This enables you to precisely understand the business impact (customer satisfaction, conversions, etc.) of poor end-user experience. The Customers Perspective: Passive Monitoring connects the dots, allowing you to see how technical performance issues relate to crucial business issues. This helps you answer questions like How many users were impacted by that last performance incident? Is it a segment of users I care about? or Do customers experiences and conversion behaviors change depending on where they are or the browser they use? Passive Monitoring provides real insight on the business consequences of performance and helps you to: 6

7 Drive revenue: protect revenue-generating pages and transactions, and identify the impact of slow response time on business goals like conversion and page views Reduce costs: know immediately when your customer satisfaction, conversion, and page view goals are threatened by performance issues Optimize client-side performance: monitor and improve the behavior of browser-based Rich Internet Applications using technologies such as AJAX, Flash, Flex, etc. Retain customers: keep customer satisfaction levels high by optimizing performance for key users across their browsers, operating systems, locations, ISPs, and devices. Observe 3rd party providers: see how their contributions (ads, analytics, CDN, search, etc.) perform for your real users Better preparation for Web 2.0 The advanced applications common to the Web 2.0 world have shifted from servers to end-user browsers. Passive Monitoring addresses this technological shift by collecting real-user data, including their technical characteristics from browser types to window sizes, from operating systems to ISPs regardless of complexity. All for the users you care most about, your own. Combined Benefits: The Power of a Complete Outside-in Monitoring Approach With so much at stake, organizations with business and mission critical web applications that drive revenue, lower costs and promote brands must proactively manage web performance and customer experiences. Outside-in web performance monitoring provides visibility to the performance of your most important web pages and transactions as your user experiences them you cannot fix what you cannot see. Where you start monitoring and which approach to take obviously depends on your objectives. This paper has outlined key reasons why both IT and business groups are implementing outside-in end-user monitoring as part of a complete monitoring strategy. On the next page is a summary of the primary benefits companies receive from each approach: 7

8 Combined Benefits: The Power of a Complete Outside-in Monitoring Approach, continued Active Backbone Monitoring Active Last Mile Monitoring Passive Real User Monitoring Monitoring Vantage Point Geographically distributed nodes at large ISPs (datacenter-grade connectivity, machines and servers with guaranteed uptime) Geographically distributed consumer-grade hardware across a diverse set of ISPs, machine configurations and networks Tagged pages within an application that collect performance data based on real user interactions Number of Testing Locations ~100 Tens of thousands Hundreds of thousands based on all, or a subset, of real visitors to your website Key Benefits for Business Stakeholders ecommerce Directors, Marketing and Sales Executives, Product Managers, etc. Monitor the health of key transactions on website Manage Service Level Agreements (SLAs) with vendors Baseline website performance across key geographies Benchmark performance against key competitors Measure the differences in the quality of service for existing and new markets Quantify the benefits that different CDNs provide in your key geographies Proactively test performance from the end-user s perspective at multiple stages of the development process Understand the business impact of customer performance problems Track and notify when customer satisfaction, conversion, and page view goals are NOT met Measure that key customer segments can complete transactions and are satisfied Key Benefits for Technical Stakeholders Vice Presidents of IT and Operations, NOC Managers, Quality of Service Directors, etc. Continuously monitor availability and end to end performance across application tiers Notify personnel when an application problem impacts customers Accelerate root cause analysis of application, network, and partner issues Analyze performance trends and support capacity planning Validate hardware and software changes and new features before deployment Monitor application performance and identify customer issues in specific local markets Troubleshoot ISPs and uncover peering issues with regional ISPs Identify, before deployment, problems in key localities and new regions or with real world variables like connection speeds, (dial-up, DSL, cable broadband, fiber optics, etc.) before users arrive Prioritize performance issues based on the magnitude of customer and business impact Measure changes for real users based on new infrastructure, code, website features, etc. Monitor impact of vendor changes on performance delivered to end-users Identify the impact of browser type, connection speed, and location on performance for real users 8

9 Outside-in Monitoring Across the Entire Web Application Delivery Chain Combined Benefits: The Power of a Complete Outside-in Monitoring Approach, continued The best approach to monitoring combines both active and passive approaches for a complete web performance monitoring strategy. Active testing from the backbone and the last mile alerts you to systems and infrastructure problems and accelerates problem resolution; passive real user monitoring exposes real customers problems that affect satisfaction and your business. By taking an integrated approach, you gain a 360º perspective that helps you: Protect revenue: complete web performance monitoring ensures that your most important pages and transactions perform properly and exposes the business impact of customers performance problems Reduce operational costs: rapid notification and detailed diagnostics accelerates problem identification and helps personnel perform root cause analysis to resolve application issues, fast Protect customer experience: keep customers coming to your site by identifying and resolving performance issues impacting customer satisfaction, conversion, and page views before too many customers are impacted Optimize performance: a complete perspective allows you to identify all user issues and then tune application performance and maximize availability for key customer segments and geographies Watch the competition: benchmarking your site s performance against the competition helps you understand the performance levels you must achieve to increase market share Load Balancing 3 rd Party/Cloud Web Services Local ISP 1 Web Server App Server Major ISP 1 Internet Local ISP 2 Db Server Local ISP 3 Storage Content Delivery Networks Local ISP 4 Internal Monitoring Active Monitoring from Internet Backbone Active Monitoring from Last Mile Passive Monitoring from Actual End Users 9

10 Our overall evaluation of Gomez s ExperienceFirst solution placed it in the on fire category; it is definitely a service provider to consider... Gomez has made spectacular progress in the past 18 months to become the leader in Web experience management. Jean-Pierre Garbani Vice President, Principal Analyst, Forrester Research Tech Horizons: Evaluating Gomez s Web Experience Management Services (August 2008) Gomez: A One-Vendor Solution to Web Performance Monitoring At Gomez, we recognize that the right web monitoring solution needs to be as simple as it is thorough. That s why we offer all offer three perspectives of outside in monitoring through one integrated suite that includes: Active web performance monitoring from the Internet backbone Active Network XF provides page-level and transaction monitoring from 80+ globally-located Internet backbone nodes Active web performance monitoring from the last mile Active Last Mile XF enables you to take edge-of-the-network measurements at multiple connection speeds from over 80,000 real user desktops throughout the world Passive web performance monitoring from actual end users Actual Experience XF measures the true experiences of your site s real users and the impact of those experiences on business results. Unlike device-based solutions, Gomez s unique tagging approach provides insight into the impact of elements outside your firewall (like a user s browser or 3 rd party service) on web performance. Offered as a software-as-a-service (SaaS), Gomez provides the industry s largest physical global network available to you, on-demand; you can begin at any level and scale upwards as your needs evolve. Coverage is easy: Gomez supports all browsers and geographies; there s never any hardware to install or manage, and there are no additional expenses for licensing or maintenance. With short implementation times, you achieve business value almost immediately. To learn more about Gomez visit or call us at About Gomez Gomez, Inc. is a leading provider of web application experience management services, which businesses use to test their web applications while in development and to monitor their web applications after deployment. More than 2,500 customers use Gomez s on-demand services to improve the quality of the web experience in order to increase their revenue from web applications, reduce their operating costs, and extend their brand reputations. For more information, please visit 10