mpulse Whitepaper SOASTA Real User Monitoring Best Practices How to Connect Performance Analytics to Business Results SOASTA mpulse Dashboard
Contents Digital Performance is User Focused.................... 3 Step One: Understand Your Current Site Speed.............. 4 Step Two: Determine Your Optimal Speed Metric.............. 5 Step Three: Assess Your Current Mobile Testing Strategy......... 6 Step Four: Connect Performance to Your Business............. 7 About SOASTA mpulse........................... 8 SOASTA, Inc. Connecting Performance Analytics to Business Results 2
Digital Performance is User Focused Whether it s revenue, customers, cart abandonment rates, bounce rates or session length all of the major metrics of business success share one crucial commonality: performance. A speedy, responsive site is directly correlated with higher levels of visitor engagement and favorable user experience. Real User Measurement correlation between page load time and user engagement and shows where performance can be optimized to directly impact business outcomes. Too often, however, the conversation about user engagement and experience gets bogged down in conversations that dwell on servers and back-end infrastructures and not on measuring the user s actual experience. The fact is, business managers rely on tools like Omniture and Google Analytics to look at engagement from their preferred perspectives such as bounce rates, cart size, downloads, and number of page views. On the other hand, IT pros tend to concentrate on back-end monitoring tools such as Keynote or Compuware Gomez to measure uptime, availability, and performance bottlenecks. Those monitoring tools can tell you which pages load faster and which browsers are performing well but they don t have a business context to take action on that data. As a result, performance data gets compartmentalized into a technical-only perspective or business-only silos. The business context for performance and engagement is never seen. Some pages are slower and some are faster but which are the ones that tie directly to business goals? Behind-the-scenes improvements may lower IT costs and improve security but do they improve the user experience with your site? Testing scenarios simulate and predict results but what happens to actual users on a live site? With as much as 80-90 percent of today s performance issues arising in the front-end client environment, a new discipline has emerged Real User Management (RUM). With RUM, actual events in real time can be measured as they are experienced by real users. With these timely analyses, Web performance can better align with business goals. The following pages describe a four-step framework for implementing the RUM discipline and deriving its associated value. At the core of this implementation framework is SOASTA mpulse, a breakthrough RUM solution that tracks and aggregates performance and engagement metrics from a business perspective. http://www.soasta.com/mpulse. SOASTA, Inc. Connecting Performance Analytics to Business Results 3
Step One: Understand Your Current Site Speed Begin with getting a true sense of the speed and responsiveness of your website. It is inarguable that the speed and availability of the website is strategically critical to e-commerce. The faster and more responsive the site is, the more likely it is customers will engage and complete transactions. Collect all the data. Optimizing application performance to create a is only possible when you have all the data at your disposal. SOASTA mpulse runs on a massively scalable analytics engine, putting big data at your 100 percent of all user every aspect of a user s interaction with your web and mobile applications in real time from page load times and on-page visits to exit, bounce and conversion rates. Collect all the Data Traditionally, website professionals have relied on synthetic monitoring to gauge the performance of their sites. In this model, developers create simulations of use cases against the website. For instance, an e-commerce website might script a search, with clicks/taps to add items to a cart, and complete a checkout process. Then, using software automation, testing professionals would replay that simulation perhaps thousands of times with thousands of simultaneous users - and summarize the results of the hypothetical scenario. The pitfall of synthetic monitoring is that it can also fail to account for crucial variations. Time of day can matter. Remote locations can produce unexpected lag/latency. Add in many different kinds of devices and browsers, and connection speeds. And the fact that users often take different paths through websites and processes. Each new step or variable introduces errors and complexities that are virtually impossible to anticipate the ultimate effect of which can be very costly. What s more, synthetic monitoring doesn t attempt to pin down the underlying causes of latency and delays. Most sites are increasingly dependent on content and services delivered by third-party providers from images, content-delivery networks, user-ratings, discussion threads, payment services, and more. Sites and application owners have little control over those performances and even less access to performance data. The result is a rapidly proliferating set of permutations that even the most ambitious testing plans and infrastructures will be unable to keep up with. While preproduction testing is highly valuable and an indispensable best practice, it s no longer enough. That testing must be supplemented with real user monitoring for true business success. With SOASTA mpulse, a near-zero-footprint snippet of JavaScript is added within the site. Within a unique session ID for each user, the mpulse code snippet sends a beacon or pulse to an related): timings, site locations, browser type, and more. The real user aggregates that data 100 percent of user-experience data happening live on your site and compiles sophisticated metrics and analytics to give you a real-time picture of exactly what is and isn t performing on your website. You get to see exactly how fast the pages load and how fast each component of a page is running. And for companies that have SLAs with third-party providers, this user- isolated performance data is precisely what s needed to give those agreements a level of enforceability. SOASTA, Inc. Connecting Performance Analytics to Business Results 4
Step Two: Determine Your Optimal Speed Metric essential that developers have correlated access to the underlying performance data associated with those incidents. When a payment page loads poorly, what is the typical user response? How long will they wait before abandoning the site? What does a two-second slowdown mean for conversion at checkout? SOASTA RUM drills down to see what each user did in response to realworld scenarios. transactions, and site areas based on business needs and context. SOASTA gives businesses the ability to achieve their goals by turning site speed into more conversions, increased loyalty, and greater revenue. When determining your optimal speed metric, best-practice standards and service levels from leading performance sites are helpful but may not be rooted in the same experiences and preferences of your unique demographics. RUM gives you speed goals. With RUM, for instance, you can answer crucial questions such as, What s the ROI rates from 2 percent to 3 percent? Custom Timers. What matters in relation to real user interactions varies from site-tosite, page-to-page and transaction-to-transaction. SOASTA mpulse gives you to determine what timers are important and create custom timers based on your own criteria. Measure beyond simple page speed to custom components and third-party services to data for SLA management and enforcement. Performance Metrics to Business Metrics SOASTA, Inc. Connecting Performance Analytics to Business Results 5
Step Three: Access Your Current Mobile Testing Strategy The proliferation of native mobile apps is creating a new set of development and testing challenges. While performance improvements to a website can be pushed out in a matter of hours, changes to stretching the development lifecycle to weeks or months, which is completely at odds with our era and culture of immediate updates that respond to performance issues. SOASTA mpulse has a mobile component that gives developers and testers new visibility into their native mobile apps. This is important, since mobile users often have very different behaviors. By adding this new dimension to the testing arsenal, developers can see what the mobile user does, what pages they visit, what devices they re using, what types of connections and carriers they re using, and much more. monitor user performance. Since those calls are built into the app, you don t need to change your application to change what you want to measure and monitor. cell carrier. With net neutrality now an open question, app makers will want to carefully monitor their app s performance on various carriers to ensure they are getting the speed they ve contractually purchased or to determine if a carrier is throttling their site. Custom Metrics. Application performance and its effect on real user behavior is most valuable in context to its impact on your business. SOASTA mpulse lets you measure the impact of user behavior on such key performance indicators (KPIs) as conversion rates and revenue, guiding you toward the steps necessary to improve application performance enhancing user experience and ultimately ensure that your business objectives are being met. Bounce Rate vs. Page Load SOASTA, Inc. Connecting Performance Analytics to Business Results 6
Step Four: Connect Performance to Your Business The bottom line: SOASTA mpulse gives you a view of all of your user data. There are no gaps, formulas, extrapolations, or assumptions regarding the performance of your site. RUM eliminates happen. For instance, if certain pages or page types don t load well in certain browsers, you can directly and quickly improve your performance, to meet your business goals. It s no longer a case of performance for performance s sake. It s performance improvements in direct pursuit of business goals. continually raises the bar for performance. Real Time Data Visualization. Utilizing cutting-edge visualization technologies including WebGL typically used in gaming and 3D animation mpulse creates vivid, interactive and easy-to-understand views of how web and mobile performance impacts your business, all customizable needs. To complete the view, mpulse offers multidimensional analysis that lets you drill deeply into data by page groups, browser, mobile device and carrier, geography, bandwidth and more. Real Time Data Visualization SOASTA, Inc. Connecting Performance Analytics to Business Results 7
About SOASTA mpulse The real value of SOASTA mpulse lies in delivering the data you need to enhance user experience and ensure that performance metrics are aligned with your business objectives. mpulse helps you achieve these goals by providing real-time, actionable data for all areas of your business. Using Real User Monitoring (RUM), mpulse tracks and aggregates the performance and engagement metrics most important to your business directly from a user s mobile app or browser allowing that randomly sample user interactions, mpulse captures every single user experience across every browser, platform, OS, and geographical location. It then instantly presents this data to you in an actionable format, allowing for greater control of your business outcome. About SOASTA SOASTA is the leader in mobile and web performance test automation and monitoring solutions, SOASTA CloudTest, SOASTA TouchTest, and SOASTA mpulse, enable developers, QA professionals and IT operations teams to test and monitor users with unprecedented speed, scale, precision, and visibility. The innovative product set streamlines test creation, automates provisioning and execution, and analyzes real user behavior in real time. It distills analytics to deliver actionable held and headquartered in Mountain View, Calif. For more information about SOASTA, please visit http://www.soasta.com. Headquarters 650-210-4950 444 Castro St, Fourth Floor Mountain View, CA 94041 FAX: 650-210-4957 EMEA New Broad Street House 35-42 New Broad St. London EC2M 1NH ASIA Room 11/12 25/F Central Plaza No.381 HuaiHai Middle Rd. Shanghai, China, 200000 To learn more visit: soasta.com or email us at info@soasta.com 2014 SOASTA. All rights reserved. SOASTA, the SOASTA logo, SOASTA CloudTest, SOASTA TouchTest and SOAS- TA mpulse are trademarks of SOASTA. All other product or company names may be trademarks and/or registered trademarks of their respective owners. mpulsewp.20140914.v8 Connect with us: TEST FASTER. RELEASE SOONER.