Why Real Browsers Matter

Transcription

1 White Paper Why Real Browsers Matter Why real browser measurement technologies matter to IT operations and business managers that are measuring their Web applications and properties for both operational availability and also monitoring the complete end-user experience Authors Keynote Systems Robert Hughes Ken Vodicka

2 Table of Contents Executive Summary 3 Why Testing Web 2.0 Sites Requires Real Browser Measurements 4 The Evolution of the Web Page 5 How Ajax Improves the User Experience 5 How Browsers Handle Rich Internet Applications 6 Four Web Applications That Need Real Browser Monitoring 7 1. Pottery Barn - Accurately Capturing a Web Page with a Real Browser 7 2. Borders - Measuring the impact of executing JavaScript in a Real Browser 9 3. Harrah s - Detecting Flash Plug-Ins in Real Browsers Nordstrom - Precisely Measuring Client Execution of VBScript in a Real Browser 12 Which Real Browser Do I Use to Monitor My Site s Performance? 13 Monitor End-User Performance with Real Browser Technology 14 Expand Availability Monitoring with Real Browser Technology 15 Conclusion 15 Recommended Reading 16 Why Real Browsers Matter 2

3 Executive Summary Web Performance Monitoring is a discipline for IT operations and E- commerce business management teams that includes both operational monitoring and end-user experience monitoring. Operational Monitoring helps operations teams implement dependable endto-end performance monitoring for critical Web applications as a turn-key service, and is achieved with high-frequency application monitoring that detects response time and availability issues in real time. End-user Experience Monitoring is the domain of both operations teams and performance analysts, who need to most accurately measure the performance of critical Web transactions, including those involving interactive Flash, from the true end user s perspective. This gives full visibility into the user s experience by providing real browser measurements of the responsiveness and availability of RIAs. Monitoring the performance of Web applications and properties is essential to ensuring that customers are having a good user experience when interacting with application interfaces, media and content that use Rich Internet Applications (RIA) technologies. Rich Internet Applications use the newest Web 2.0 technologies, such as Ajax, Flash, Silverlight and sophisticated JavaScript to ensure a user experience on Web sites that have some of the characteristics of desktop applications, such as interactive UIs that are also quick and responsive. Both of the performance monitoring disciplines highlighted in the sidebar are important to IT operations teams to ensure high performance responsiveness and availability of Web applications. When teams expand from operational monitoring to monitoring the complete end-user experience on the Web, both IT and business teams benefit. IT operations are able to accurately measure the performance of critical Web transactions, including those use desktop-like RIA technologies, using real browser measurement technologies. E-commerce business managers gain the most accurate and objective insights so that they can effectively benchmark and trend their Web performance against competitors. With RIAs, much of the execution of code in Web applications happens on the browser client, and not just the Web or application server, and therefore understanding what is actually happening inside a real browser has become imperative. To understand and measure an application s performance, most Web monitoring tools try to emulate a user transaction using measurement technologies that use imitation, or emulated browsers. These imitation browsers mimic, or emulate browsers like IE and Firefox to simulate a site or application visitor s behavior. While emulated browser measurements have their place when simply monitoring for availability, a complete Web performance monitoring discipline must include both operational monitoring, primarily conducted using emulated browsers, and true end-user experience monitoring, conducted using real browsers. This paper will illuminate why it is critical for developers, test and QA organizations, operations, and business managers to rely on performance monitoring technologies that use a real Microsoft Internet Explorer (IE) browser to test Web transactions, especially for Web applications that employ Web 2.0 or RIA technologies. This paper will highlight four RIA examples where the end-user experience can only be captured with Web performance monitoring technologies that use a real browser: 1. Accurately capturing a snapshot of a Web page with a real browser 2. Measuring the impact of executing JavaScript in a real browser 3. Detecting Flash plug-ins in real browsers 4. Precisely measuring client execution of VBScript in a real browser Why Real Browsers Matter 3

4 Why Testing Web 2.0 Sites Requires Real Browser Measurements We constantly hear the term Web 2.0, but if you ask someone, What is Web 2.0? you will get many different answers. Some people call it a technology, but it is really the second generation of Web development and design. This second generation includes tools that enable better collaboration, communication, and interoperability on the World Wide Web. Social networking sites such as MySpace, Facebook, and YouTube are good examples of the information sharing and collaboration that we call Web 2.0. Among Web 2.0 applications, there is a class of Rich Internet Applications that are made possible by technologies such as Ajax/JavaScript, CSS, DHTML/XHTML, and Flash/Silverlight, all working together. In today s modern Web applications, Ajax and Flash are just about everywhere. They are not just in Web sites that use Rich Internet Applications like Google Maps or rich media sites like Amazon; advanced Web 2.0 techniques are now found on even simple Web sites. Ajax and Flash redefine the role of the client and server that developers have been accustomed to in the design of traditional Web applications. In the past, Web browsers merely displayed content that was either statically or dynamically processed on a Web server, and there was very little (if any) client-side code execution in the browser. Content such as HTML, CSS, and images - was designed to reside on the Web server waiting for the server-side user session to access it, and deliver it to the client browser, which would then display the content. Nowadays, the most engaging Web applications rely on the browser to also execute code and redisplay content, often requiring the browser to process code that requires a desktop engine, such as JavaScript engine built into the browser, or a Flash player installed as a Web plug-in. The Web operating system is no longer just resident on the server, it is also on the user s desktop, in the client-based browser. Why Real Browsers Matter 4

5 The Evolution of the Web Page The Web was originally intended to help researchers share documents as static pages of linked text formatted in HTML. From there, Web pages quickly evolved to include complex structures of text and graphics, with plug-in programs to play audio and video files or to stream multimedia content. Web developers supplement the basic browser function of rendering HTML by invoking code such as JavaScript, or Perl on the user s computer. These scripts can create interface elements such as rollover effects, custom pull-down menus, and other navigation aids. They can also execute UI methods for example, to validate a user s input in an HTML form. These script capabilities enhance a user s interaction with individual Web pages. However, they do not change the fundamental model in which application logic runs on the server and executes between Web pages after the user clicks. This behavior is said to be synchronous that is, after each click the user waits while the server handles the input and the browser downloads a response page. In e-commerce, a typical user interaction involves a series of Web pages, which represent steps in a larger process that comprises a Web application or Rich Internet Application. This synchronous communication between the Web browser and the server is now becoming more asynchronous. How Ajax Improves the User Experience Ajax has almost become synonymous with Web 2.0. While the acronym refers to Asynchronous JavaScript and XML, the term Ajax can also be used to describe any combination of client-side scripting and data interchange technologies (not specifically JavaScript and XML) used to exchange data between client and server asynchronously. Each implementation of Ajax can vary, but the concept is basically the same. Code execution is now shared between the server and client, and data is typically transferred asynchronously between page loads. Ultimately, the goal is to smooth out the jerkiness of browser/server interaction inherent in the old synchronous click-and-wait model. The user experiences a faster, more nimble Web site, with the Web application now performing with the responsiveness and interactivity of a desktop application. Why Real Browsers Matter 5

6 How Browsers Handle Rich Internet Applications The browser performs the important role of downloading, rendering, and arranging Web content and enabling user interaction with the Web application. It is as important as the design of the application or the servers that are hosting it, since its features and capabilities can define the user s experience of the Web application. It should accurately render content and perform exactly as the designers and developers envisioned. As a visitor navigates to the site, the Web browser is responsible for resolving the URL, requesting the appropriate content, and requesting all of the relevant objects that it needs to render the page that the visitor requested. Additionally, content-rich sites that have animation or other dynamic components may also require that the browser execute client-side scripting or initialize plug-in components - Flash and Silverlight, for example - that allow the correct content to render and provide a rich user experience. When you need to test these more dynamic sites, you quickly realize that understanding the browser s behavior and its interaction with the Web site is critical. For example, it is necessary not just to download and display HTML, but also to initialize plug-ins that allow the display of current TV episodes or even navigation of the entire site. Other Web sites employ methods such as preloading content to minimize user wait time. JavaScript is executed by the browser to download content that will be used on subsequent pages before the user requests it. This method doesn t reduce network time, which is typically monitored by emulated browsers, but does significantly increase the performance of the application from the user s perspective. Companies like Microsoft and Mozilla have spent millions of dollars and years building real browser products such as Internet Explorer and Firefox to enhance the quality of the user experience on the Web. It would be Why Real Browsers Matter 6

7 practically impossible to emulate the behavior of these browsers to a high degree of accuracy, especially to monitor the true end-user experience e.g. emulating how IE and Firefox browsers perform multi-threading when downloading content from multiple domains. How the browser interacts with the Web application and renders the content is entirely controlled by the browser, and is not a task that can be reliably emulated. IT operations and business managers need Web performance products that monitor the true end-user experience by leveraging embedding a real IE or a Firefox browser. IE, Firefox, Safari, Chrome, and other browsers have all been designed with the same goal in mind: to provide a high-quality user experience. The browser will enable the execution of plug-ins, Flash, and JavaScript to give the user the best experience possible. As more sites adopt these technologies, the methods used to benchmark their performance must be adapted to reflect the changes in technology. Using an instrument that most closely approaches the end-user experience is the most accurate way of capturing the ever-increasing complexities of these sites. If the measurement tool uses the real browser to benchmark a site, you can be confident that the end-user s experience reflects what the tool is reporting. However, when imitation browsers are used, the performance metrics frequently fail to accurately reflect the true end-user experience and that is why such a technique is used correctly for operational monitoring only. Four Web Applications That Need Real Browser Monitoring Let s take a look at four Web applications on sites where real browsers matter. These examples highlight how the end-user experience can be captured only with Web performance-monitoring technologies that use a real browser: 1. Accurately capturing a snapshot of a Web page with a real browser 2. Measuring the impact of executing JavaScript in a real browser 3. Detecting Flash plug-ins in real browsers 4. Precisely measuring client execution of VBScript in a real browser 1. Pottery Barn - Accurately Capturing a Web Page with a Real Browser There are many reasons to do ongoing testing and monitoring with a solution that uses a real browser rather than an imitation browser. An important reason is the ability to accurately capture an actual high-fidelity bitmap of the page when an error occurs instead of a reconstructed page. A bitmap of a page is generated when you take a screenshot of the actual page exactly as it is drawn by the browser. A reconstructed page attempts to recreate the layout of the page by translating the captured HTML. Solutions that use imitation browser testing are forced to capture the HTML of a page when an error occurs, since they do not actually draw or construct the page during the test. If you use an imitation browser to try to view the page that generated the error, you will get a reconstructed page. While this works for simple apps, sites that use RIA technologies such as DHTML will not work with imitation browsers. A real browsertesting product captures the exact page that the user sees in a bitmap Why Real Browsers Matter 7

8 image; nothing is lost in the translation by reconstructing and redrawing the page. In the example above, Pottery Barn s Web site is using DHTML to display a Web page, but the page did not display correctly. The page looks washed out because the previous page s graphics and text seem to float on the page. This triggered an error when the user attempted to add the product for sale, washcloths in this example, to their shopping chart. In turn, this error caused performance problems. When operation managers see a performance issue they want to quickly resolve the problem to reduce the number of users who could be impacted. To capture the Web page exactly as the user would experience it, you need real browser technology that captures the actual dynamic behavior of the Web site. An imitation browser that captures individual objects would try to reconstruct this image by assembling objects via HTML, a process that can be error prone and can result in a misleading analysis. Screen reconstruction would have extreme difficulty displaying the DHTML behavior used by a site like Pottery Barn. Using an imitation browser is like reconstructing a crime scene from someone s memory of it. Using real browser technology is like using a surveillance camera to take a snapshot of the actual crime scene. Why Real Browsers Matter 8

9 2. Borders - Measuring the impact of executing JavaScript in a Real Browser Total user experience time (Total Time) is captured by measuring (Network Time) plus client execution time (Client Time). A detailed look at the Borders Web site shows why testing with a real browser is the best way to capture interaction on the client side. It is important to understand that you need to capture the total user experience time when measuring performance. Total user experience time is a metric reported in Web performance monitoring tools, and this includes network time and client time. Network time is the total time needed to download a Web page from the network s point of view. It starts from the beginning of the base file s DNS lookup, and it ends when the last packet of the last page element has been delivered. Client time is the time during which page element execution is taking place, such as executing JavaScript or executing and displaying Flash objects. End-user experience monitoring with real browsers gives you both the network time and client execution time. Imitation browsers cannot capture client execution time at all. Network time is the total time needed to download a Web page from the network s point of view. When a user accesses the Borders home page, the page executes JavaScript on the client side to give the user a rich interactive experience. The client-side execution time is part of the total user experience time that needs to be measured. The other component being measured is network time. Network time may include the following components: DNS lookup, initial connection, SSL, redirection, request time, first byte download, and content download. A waterfall chart then displays all the objects on a page and their response times. Below we drill into the waterfall to better understand what components could be causing the performance slowdown. This Borders transaction shows how Keynote Systems real browser approach executes a JavaScript function contained in a.js file to determine which content it will download based on a client cookie. The waterfall below shows a break (client execution time) in the download time, which shows exactly how long it takes to execute that JavaScript code. Why Real Browsers Matter 9

10 Monitoring you can perform with real browser technology Client time is time during which page element execution takes place within the browser. Real browser testing and monitoring will highlight the time it takes to execute code in the browser, and on the client computer. It is important to note that client execution time accounted for 23% of the total measurement time. Only measurement technology that embeds a real browser can capture client execution time. Next, we ll look at how an imitation browser measures the performance of the Borders page. It is important to note that the imitation browser does not capture the JavaScript client execution time. When there is a performance slowdown, operations managers or developers may think the problem lies in network time. They cannot see that the problem lies in the client execution time, since the imitation browser cannot capture the client execution time of Web 2.0 based applications. Imitation browsers cannot capture client execution time. Imitation browser technology looks similar to real browser technology, but misses critical client execution time. This Borders transaction shows how an imitation browser cannot dynamically execute a JavaScript file to determine which content it will download based on a client-side cookie. The imitation browser does not capture client execution time, leading to faulty analysis in monitoring the performance of Web 2.0 sites you might believe that the application is more responsive than it actually is for end-users. Why Real Browsers Matter 10

11 3. Harrah s - Detecting Flash Plug-Ins in Real Browsers Harrah s home page tries to detect whether the user s browser can accept a Flash object to display rich content. This example again shows that it is essential to use a real browser measurement tool to capture metrics that most accurately reflect the actual user experience. If there is some loss in fidelity, the data will not be good enough to be useful in managing Web site quality. Use of a real browser-monitoring product is especially important for business managers who are responsible for service-level agreements with vendors or other partners. The Web page code snippet below checks for a Flash-enabled browser. If the Flash plug-in is not available, it tries to load the associated plug-in to run the Flash object. Real browser technology executes logic for detecting a plugin, and then displays the ad. Why Real Browsers Matter 11

12 Below is a screenshot of Keynote KITE, a free web performance testing tool that embeds a real IE browser being used to analyze the performance of Harrah s home page. KITE is available for download from Testing with an imitation browser will not always allow the Flash plug-in to be loaded. In some cases, the Web site cannot recognize the browser type. As a result, you would either miss the Flash object or the test script would have to be hard-coded to enable it. If the imitation browser can load the Flash object, it does not always wait for it to render. Testing with a real browser ensures that client-side actions are real and consistent with the experience of end users using the same browser. 4. Nordstrom - Precisely Measuring Client Execution of VBScript in a Real Browser Nordstrom s home page detects which Microsoft Internet Explorer version the user is running to optimize user experience and stickiness. Depending on the capabilities of the browser, different client execution will be used to enhance the user experience. Why Real Browsers Matter 12

13 Note that testing with an imitation browser will not always run the Visual Basic Script (VBScript) and allow loading of the Flash plug-in. This is sometimes because the Web site cannot recognize the browser type, resulting in missing the Flash ad or having to hard-code the test script to enable it. In addition, if the imitation browser can load the Flash object, it does not always wait for it to render. Which Real Browser Do I Use to Monitor My Site s Performance? We have just explained what real browsers do and why it s important to use them in testing the performance of your Web application. You may now be asking, Which browser should I use to monitor my site s performance? Should I test for all the most popular browsers? IE still has the majority of users in the market place but shouldn t I also test with Firefox? Testing with a real IE browser is the most effective strategy because of the IE audience and because open web standards do not require you to conduct extensive testing using multiple real browsers you gain the most by testing first using IE. Another reason to test with IE is that application developers often testing their applications using Firefox, only to have the application fail or not perform well when IE users begin to use it. Since most of the other browsers support the World Wide Web Consortium (W3C) features and standards, testing for the most complex browser (IE) is recommended. The browser that is most widely used is Microsoft Internet Explorer, with over two-thirds of the market nearly 70% of site visitors use Internet Explorer. All of other browsers combined total less than one-third of the market. Why Real Browsers Matter 13

14 Ask yourself the fundamental question, Which browser is our customer base using, and is our Web performance being monitored with that real browser or an imitation browser? Many Web site developers have had to code their sites so they can detect which browser was viewing their site, and they used different code sets to accommodate the different browsers. A decade ago, Microsoft took one approach with IE, while Netscape took another. With the myriad of browsers, it was becoming a developer s nightmare to keep their sites compatible, and a code change for one browser could affect the performance and availability of the site with all other browsers. Today, this is no longer the case. In October 1998 DOM Level 1 was adopted by the Web standards body W3C. This was a joint effort by Netscape, Microsoft, and other W3C members to adopt a platform- and language-neutral interface that allows programs and scripts to dynamically access and update content on browsers. With the increased use of Web 2.0 technologies we have also seen increasing conformance to industry standards. More than 95% of the browsers today support either DOM Level 1 or DOM Level 2. Many browser developers now code their browsers to standards, so a single browser can handle many different Web sites. Monitor End-User Performance with Real Browser Technology The performance of a Web site can be monitored in several different ways. The most basic end-user performance metric comprises the entire time it takes from when a user requests the desired page until he or she can view that entire page on their Web browser. This time includes not only the network components such as DNS lookup, initial connection, and content download, but also any client-side execution of plug-ins and video, sounds, or animation. When you measure client-side execution you need to use a real browser-monitoring product if you want to capture the complete enduser experience. Why Real Browsers Matter 14

15 This user performance time also has two distinct versions: performance for a first-time visitor and performance for a returning visitor. Since returning visitors comprise the vast majority of traffic to a Web site, understanding which content is being cached and how the returning user is experiencing the site is critical. The use of a real browser allows you to see the browser s behavior for the returning user and validate that performance is being optimized for the returning user. Expand Availability Monitoring with Real Browser Technology Site availability is measured by calculating the successful percentage of site tests. Failure is defined as a site test that resulted in an undesired page being displayed or no page being displayed at all. Success is defined as correct display of all the intended page content. When a real browser is used to perform these tests, we can be confident that all content that was requested by the HTML and by any Flash, Ajax, Silverlight, or other technology was successfully downloaded. Testing availability with a real Web browser enables you to view the exact download sequence for each element on the page, whereas imitation browsers often do not download the page elements in the same order as the user s Web browser. The element download order helps pinpoint which content can cause the browser to hang, since its place in the download order prevents subsequent content from downloading and thus results in an availability issue for the end user. Conclusion Rich Internet Applications are a reality on the Web today. Web 2.0 technologies such as Ajax/JavaScript, CSS, Flash/Silverlight all working together allow better collaboration, communication, and interoperability. The browser s ability to manage asynchronous communications and the length of the browser s execution time are critical in delivering a good user experience in this evolving environment. To ensure a good Web experience for the user, you need to have testing and monitoring tools that can capture the entire Web experience from the Web server to the client s browser. The browser is no longer just a dumb terminal. Today s browser is a platform that executes various technologies such as Ajax, Flash, and JavaScript. Examples in this paper highlight client-side execution and plug-in detection based on JavaScript. The timing of these events cannot be captured accurately unless you measure performance with the actual browsers your users are using. To complement real browser measurements there are cases where an emulated browser is still very useful. For example, if you are interested in only monitoring availability you may not need the full fidelity of a real browser-monitoring product. However, with real browser monitoring, IT operations teams can expand from just monitoring availability to monitoring the complete end-user experience. Many companies find that a blend of real browser and emulated browser test solutions is a must in securing the performance they desire. Why Real Browsers Matter 15

16 Keynote s test and measurement products are designed to capture accurate and actionable information about transaction performance in the latest Web 2.0 technology environments with real browser and emulated browser technologies. Keynote provides web operational monitoring and end-user experience monitoring to meet the full range of performance measurement needs. Recommended Reading The following books and white papers were referenced in developing this white paper and are recommended for a deeper understanding of Web 2.0 and RIA technologies: Sas Jacobs, Beginning XML with DOM and Ajax: From Novice to Professional (Berkeley, Calif.: Apress, 2006). Chris Loosley, Rich Internet Applications: Design, Measurement, and Management Challenges (San Mateo, Calif.: Keynote Systems, 2006); available at Jennifer Niederst Robbins, Web Design in a Nutshell: A Desktop Quick Reference (Sebastopol, Calif.: O Reilly, 1998). Steve Souders, High-Performance Web Sites: Essential Knowledge for Front-End Engineers (Sebastopol, Calif.: O Reilly, 2007). Why Real Browsers Matter 16