Web Site Performance: Can You Handle the Traffic? Alex Shah Chief Technology Officer VelociGen, Inc.

Transcription

1 Alex Shah Chief Technology Officer VelociGen, Inc. Abstract Each week there are reports of another major site experiencing outages due to overwhelming traffic. Slow or unresponsive web sites account for hundreds of millions of dollars in lost revenue and increased cost, with estimates going into the billions for the coming year. Despite these facts, performance and capacity testing and tuning are largely ignored by web masters and developers. We will demonstrate the need for web developers to focus on the web application in addition to connection issues between the web site and end user. A guideline for testing web sites will be proposed as well as a detailed explanation of techniques used at VelociGen Inc. to guarantee reliability and quick response times under heavy traffic.

2 Introduction On February 14, 1998, Valentine s Day, America s largest reseller of flowers, FTD, with the help of their new online web site, FTD.com, was on its way to a record breaking day of flower sales. However, by day s end, millions of dollars of revenue had been lost due to a slow and unresponsive web site. In the early morning hours the web site was doing fine, but as the day progressed, more and more traffic congested the site. By 10:00am, the site had slowed down to an unusable state. Transactions were taking more than 30 seconds to respond, often not responding at all. Within an hour, the web site was completely unreachable, even by FTD s own development staff. Since FTD s developers were located in Toronto, Canada and its web server was being co-located at an Internet Service Provider (ISP) in New Jersey, it took several frantic hours for the rescue team to get organized and travel down to FTD headquarters to begin diagnosing the problem. By 3:00pm, the web site was again able to take orders, and with careful monitoring and multiple restarts of the web server, FTD s online presence was partially restored. Millions of dollars of revenue were lost that Valentine s Day due to the number of lost orders. The indirect loss from damaging the FTD brand and the overall cost of losing online consumers to other flower vendors is more difficult to calculate, but undoubtedly exceeds any direct loss. The irony of the FTD story is that it is relatively inexpensive to detect performance issues and resolve them as compared to the cost of down time and lost business. Many web sites fail to follow necessary steps when testing their own web application. A common oversight of web site management is in the preparation for failures at the ISP, or the connection between the web server and the end user while neglecting other aspects of the web site. Part of this failure can be attributed to overconfidence amongst the in-house development team. Relying on the development team to check its own work for defects is not ideal. Using a separate Quality Assurance (QA) department is a better way of bug tracking, but the QA lab will often find usability and look and feel type mistakes. Having a few QA people review a web site does not compare to having thousands of simultaneous users banging away. Most QA labs do not have the expertise or tools needed to emulate how the live web site will actually be used. The result: the web site crashes during peak usage, or on the launch day. A set of guidelines that VelociGen Inc. uses for effectively testing web sites will be discussed. In addition, we will demonstrate several common web site performance issues and our solutions. Even with the best preparation, web site disasters can happen. In fact, web site failure is nearly unavoidable and precautions must be taken to properly monitor for the inevitable crash. A plan of action will be proposed for worst case scenarios. Unavoidable Loss A recent study by Zona Research, Inc. (Nov., 1999) found that 20% of Internet users spend on average $200 each month. Given the size of the Internet (67 million active users), this equals 13.5 million online buyers, each spending $200 a month, or approximately $32 billion spent each year on online commerce. While the consumer e-commerce market is considerable in size, the business-to-business (B2B) e-commerce market is larger and is predicted to grow dramatically. Web sites are beginning to accept B2B transactions over the Internet, either as part of their existing consumer site or as a key component of their business model. According to Forrester Research, the B2B e-commerce market is estimated to be $250 billion for 2000 and will grow to over $1.4 trillion by Ideally, e-commerce sites should capture all of this trillion dollar potential. Pages should load quickly, be easily understood, guide users to the information that they need with a minimal number of clicks, and not suffer from connection issues between the web server and the consumer. B2B transactions should be secure, reliable and instantaneous. Page 2

3 Figure 1. Unavoidable Loss Due to Connection Issues In reality, the web master has little control over many connection issues that a client user faces (Figure 1). As concluded in the Zona Research study, 8.7% of the time client users are unable to connect to their local ISP due to difficulties with the connection: modem configuration, phone line, busy signals, or unreliable service. 2.2% of the time, the client user s ISP loses its connection to the Internet backbone. By carefully choosing a well-connected ISP, a web site can reduce the likelihood of a dropped connection between the web server and the Internet backbone (typically 2.2%). Unfortunately, the connection between the consumer and Internet backbone is less controllable. Thus, even with an optimally connected web solution, e-commerce sites cannot avoid losing 10.9% (8.7% modem + 2.2% ISP) of potential revenue. B2B sites can expect closer to 2.2% loss, since the business on the other end will likely have a dedicated connection. The "8 Second Rule Most site managers understand that the longer a person has to wait for a web page to load, the more likely they are to bail out to another web site. As a result, web masters choose their hosting service carefully to ensure that their site has good connectivity to the consumer. When the consumer is outside the United States, efforts are made to mirror the web site in key geographic regions so that the delay in accessing a web site is minimized. Despite these considerations, many web sites overlook the fact that the majority of consumers connect to the Internet using either a 28.8 or 56kbps modem. Usability testing is done across a high speed connection instead of simulating how the end user will access the site. Given that web sites are typically accessed via 56kbps modems which transfer at 5 kilobytes (K) per second, how long will a consumer wait for a page to load? [Note: kpbs represents kilobits per second, not kilobytes per second; there are 8 bits to a byte. Most 56kbps modems actually connect at 48kbps which results in a tranfer rate of 5K/s.] Several studies have been conducted to determine why surfers terminate their use of a particular web site. The result of one such study is shown in Figure 2. Interestingly, no matter how long a page takes to load, 6% of users will bail out regardless. When a page exceeds 35K, the number of users who exit increases dramatically, until 70K is reached at which time about half have quit the site. Once a user has waited for more than 70K to transfer, they tend not to bail out, but the majority of consumers have already been lost by then. Page 3

4 Why are pages greater than 35K exited? Taking into account that the transfer rate for most end users is 5K/s, it will take 7-8 seconds for a 35K page to load. Many web studies have labeled this observation the "8 second rule". In other words, users will not wait more than 8 seconds to access a web site. Those web sites that have large front pages can expect to lose a good percentage of their visitors, as shown in Figure 2. Figure 2. Bailout rate based on size of home page Developers should keep in mind that the "8 second rule" applies to the entire loop. The loop starts when the request is sent by the browser to the web server, and ends when the result page and all embedded images are transferred, processed and rendered on the screen. One should also calculate the seconds required by web applications to generate dynamic pages, for example: pages that access a database, or process forms. Such pages should process within 1 or 2 seconds, even under peak load conditions, to ensure that the overall response time is well within the "8 second rule". Web Application Performance Since web consumers will not tolerate a delay of more than 8 seconds, several guidelines can be established: 1. Understand the route from web server to ISP to customer. By reducing the distance between the web server and customer, consistent response times can be obtained. Internet distance is not measured geographically, but rather by the number of hops. A hop is a machine on the Internet that a request must pass through to get to its destination. The more hops that are passed through to get information to the client browser, the longer the end user must wait. Work with the ISP to lower hops between the customer and the web site. If the customer is located in another country, mirror the web site to minimize the hop count. 2. Design pages that are less than 40K. Smaller pages mean lower bail out rates. If a page is dynamically generated, the size of the overall page must be reduced by 5K for each second it takes to process the page. For example, if a search request takes 3 seconds to process the page, the total page size (including images) cannot exceed 25K, 40K (5K * 3). Page 4

5 3. Minimize the number of components on each web page. The time a page takes to load is made up of the transfer time of the HTML page plus the transfer of each image plus the time to render on the browser. Creating complicated pages with many tables and multiple images will exceed the "8 second rule". 4. Create several shorter pages instead of one large page. Studies have shown that web surfers prefer to click through several smaller pages with less content than wait for a single lengthy page. 5. Maintain a persistent, high performance connection between web server and database. Common Gateway Interface (CGI) is often used to generate dynamic pages that access database content. Several issues exist with CGI which make it unusable for even low traffic web sites: Each CGI request starts a new process. Starting a new request introduces unnecessary operating system overhead when handling requests. Under load, the number of CGI processes started will equal the number of incoming CGI requests which can result in thrashing a condition where resources are dedicated to operating system tasks and no user processing can be completed. With FTD (see Introduction), thrashing brought on by too many simultaneous CGI requests resulted in a deadlock condition. Only after the power was cycled on the machine was the web site able to recover. Each CGI request requires a new database connection. Databases based on structured query language (SQL), Oracle in particular, do several lengthy processing steps when a new database connection is created. The benefit of additional overhead during the initial link is that subsequent database requests are nearly instantaneous. Unfortunately, CGI cannot take advantage of database connection caching since the process is terminated for each request. 8 sec Figure 3. Scattered CGI Response Times CGI processing is inconsistent. Usability studies have shown that in addition to the "8 second rule", the second major factor that drives web surfers away from web sites is inconsistent response times, making CGI a nonviable solution for web applications. CGI is unable to maintain consistent response times, even under moderate load. As shown in Figure 3, CGI requests are not handled in a first in, first out (FIFO) basis. Some requests are handled within a couple of seconds, while others take half a minute. The reason for this phenomena is that the operating system scheduler determines which request is handled next. Page 5

6 For example, during the processing of an initial request (request 1), a second request (request 2) may arrive and take CPU cycles from request 1. Request 2 may finish before request 1, and another request (request 3) might arrive before the process scheduler resumes request 1. Request 3 completes, and request 4 starts its processing. By now, request 1 has been swapped out to disk and will not be loaded up for some time. 6. Separate the web application from the web server. In response to the performance problems associated with CGI, several vendors have developed high performance, proprietary application programming interfaces (APIs) for their servers. The most notable are Netscape s NSAPI, Microsoft s ISAPI and Apache s server API. Applications compiled using one of these proprietary server APIs are faster than CGI programs. By running an application within the server process, the overhead of starting up and initializing a new process for each request is removed. Requests are handled in the order received and are not subject to rearrangement by the operating system s process scheduler. Database connections can be opened once, and reused. Although server APIs perform better than CGI programs, they suffer from reliability and scalability issues. Since the web application is loaded into the web server process, a bug in the application can bring down the entire server. Code placed in the web server must be thoroughly tested under various load and failure conditions to ensure reliability. In addition, server APIs are difficult to learn and force the developer to use C or C++ to develop their application. Server APIs do not scale well on their own and require specialized software and hardware to spread processing across multiple machines. Poor Testing Practices In order to verify that no page on the web site takes more than 8 seconds to load and to ensure reliability of the site, appropriate testing must be performed. Each web site has its own method for performing quality assurance. The goal of such testing is clear: to simulate how the web site will behave when put into production. Listed below are a few of the testing techniques that have been used at major web sites. These techniques do not accurately predict the web site s quality and reliability. Creating fake usage scenarios for testing purposes. For new web sites, having the QA lab use the web site and generate fake usage scenarios may be the only option. However, for an existing web site, where a new revision of the web application is being tested, real world data should be used. Often, developers ignore valuable data contained within the web site s access log. A useful testing procedure is to simulate web usage by taking the daily access log and increasing the number of concurrent users to predict site performance and reliability under peak usage conditions. Using the web developer for QA. Often the group of developers who have created the web application are assumed to be the most qualified in an organization to test the web site for defects and performance issues. It is a valid assumption, especially since QA personnel usually don t have the know-how for proper load testing. Relying on the development team to check its own work for defects is not ideal. Overconfidence amongst the in-house staff typically results in poor testing. The best solution for web site testing is to hire consultants outside the application development group, who have the tools and expertise and are paid to find reliability and performance issues. Using developers instead of software to simulate load. The basic premise of this technique is to have several developers fill out a form on the web site and then wait for the verbal command Everyone submit now before clicking on the submit button. If the application can sustain the onslaught of 5 to 10 simultaneous requests, then Page 6

7 it is ready for production. Of course, on the live web site, the likelihood of submitting identical information ten times concurrently is low, thus reducing the usefulness of this type of testing. A more accurate test would be to let a team of ten QA personnel use the web site as the web surfer would, independent of each other. An even better test would be to use load testing software that can simulate ten or more users surfing the site for several days. Simulating web usage that is less than expected. In the frantic rush to get online, many web masters neglect to verify their web application s performance under load. Many web sites are unprepared for major traffic and quickly become victims of their own success. In several cases, performance issues arise at the most inopportune time: when the site first launches, after a major announcement, or after an expensive advertising and marketing campaign. Unfortunately, the press, influenced by good marketing and press relations, are likely to visit the web site at its most vulnerable hour, compounding the damage. With so much at stake, it is advisable to predict how much traffic one expects during peak usage and prepare for double the expectation. Keep in mind that even under heavy load, the web application and ISP must be capable of maintaining response times below 8 seconds. Going live with the web site without proper testing. Often, web masters don t realize the overwhelming need for proper performance testing until the web site has gone live. Clearly, no testing method can detect performance and reliability issues as well as the live web site. With proper forethought, this method can be used to attain some good information about how the site will be used without damaging the company s reputation. One solution is to allow guest access to the web site. Let a few limited users get a sneak peak at the site before the official release date, with the knowledge that the site is still under construction. Load testing software can then record each guest s path through the site, and use this data to simulate thousands of concurrent users. Web site performance can then be accurately predicted, before the launch date. VelociGen Inc. s Testing Practices Well designed web site testing detects problems in the development computer before such problems appear on the live web site, thus reducing damages significantly. Before VelociGen Inc. begins any testing, we work with the client to understand a few basic requirements and principles. Web site operators must adequately test the application that they have written for defects, performance and reliability, prior to placing the web site into production. Page 7 Web site testing requires at least two separate computers: one for development, and a second for the live operation of the web site. The development computer is used to create the initial web site and add new features. Also, before updating the live web site, additional testing must be Figure 5. VeloMeter s User Friendly Interface

8 conducted on the development machine. Web site testing is performed using simulation techniques that reflect expected usage and real world load as closely as possible. We verify that the web site can handle double the maximum expected traffic during peak usage. When performing testing for a client that has no existing online presence, we attempt to create scenarios that reflect how the site will be used under real world conditions. At VelociGen Inc., we use specialized software called a proxy server, which captures all information that is sent between the client browser and the web server: the requested URL, arguments, POST data, and cookies. Dan Boneh s simple Perl proxy server ( can be configured to capture information about incoming requests to a log. Unlike the typical web server access log, a proxy server is able to save all components of the HTTP request the URL, POST data, cookies. One caveat of using a proxy server is that you must configure the client browser to access the web site through a proxy, rather than directly. Both Internet Explorer and Netscape Navigator are capable of accessing the web site via proxy. Once the browser has been configured to use the proxy, several sample web sessions are generated. After capturing is complete, the data contained within the proxy log is separated into several files, one for each web session. Our load testing tool, VeloMeter Pro ( ) is then used to read and simulate users using the captured sessions. The source for VeloMeter can be downloaded at no cost from VelociGen Inc. s web site: VeloMeter has been awarded the WebTechniques trophy for Best Web Site Management Tool. Our Javabased tool is capable of simulating multiple users using Java s threading capabilities. POST, GET and cookie protocols are supported. Secure Socket Layer (SSL) support is available when used with a proxy server that has SSL enabled. During performance testing, web usage can be amplified by specifying the concurrent users to simulate (Figure 5) in order to predict how much load a web site can handle before it exceeds the 8 second rule. Results can be viewed graphically (Figure 6) or exported to an Excel spreadsheet for further analysis. After simulating user load with VeloMeter, we verify that the traffic the web site can handle is double the maximum expected. If so, the web site is given our seal of approval. Figure 6. VeloMeter s Graphical Results Page 8

9 For existing sites, the load testing procedure is slightly different. For such sites, we take advantage of VeloMeter s ability to read the daily access log from the site. If the web site accepts POST or cookie data, additional sample sessions are generated using the proxy server method as described above. During and after testing, we try to capture as much information about the web site s usage as possible. Ideally, all requests and associated data should be logged so that web usage can be fully reproduced. Unfortunately, today s web servers do not provide a mechanism for capturing all data associated with an HTTP request. Proxy servers can make up for logging features that are not provided by the web server, enabling the recording of all web traffic. Recording web usage helps facilitate the simulation of real world conditions. Also, such data greatly helps to recreate conditions leading up to a web site crash. Prepare for the Worst Even after a company has adopted good web testing practices, there is always the possibility of a server crash. The more complex the web application, the more likely some unforeseen combination of parameters will take down the site. For example, one of our clients had a small mistake in their production web server s configuration file. The mistake brought out a bug in the web server software. Unfortunately the bug did not present itself until a few HEAD requests (part of the HTTP protocol used only by search engines) were made for a CGI script. Days later, and after processing hundreds of thousands of requests, the web server eventually crashed. Only after careful analysis of the access log and further testing was the problem resolved. No software is bug free. A plan of action for the worse case scenario should be written up before the web site goes live. The steps taken in that plan of action should correspond to the amount of damage caused by a crash. Depending on the number of potential dollars lost, a monitoring service can be utilized to ensure that the web site is running. It is also advisable to choose an ISP with 24/7 support and subscribe to a monthly performance consulting service. When updating an existing web site, the plan of action should also include steps to quickly fall-back to the previous working revision. Be prepared to record all information about the web server machine when a crash happens. The more information that can be gathered, the easier it will be to recreate events that lead up to the crash. The most valuable clues will be found in the web server s access log. The log will give the list of events that caused the web application to fail. Ideally, all HTTP traffic to the web site should be recorded: POST, GET, HEAD, SSL, etc. Conclusion Over the last couple years, VelociGen Inc. has developed a number of methods to ensure web site reliability and performance. If we ignore some of the technical specifics, our web site design and testing strategy boils down to a few key principles: 1) Keep the development and production machines separate. 2) Use real world data for reliability testing. 3) Test performance at no less than double the maximum expected traffic. 4) Capture as much information on the web site s usage as possible. 5) Prepare for the worst. By following these principles, you will learn from the mistakes made by other e-commerce companies and increase your chance of becoming the next big dot com success story. Page 9

10 References VelociGen, Inc., URL: VeloMeter Pro, Java based web server load tester, URL: source: Dan Boneh, simple Perl proxy server, URL: Connie Guglielmo, Crash and Get Burned, Week, September 6, 1999, URL: Forrester Research, Inc., URL: Keynote Systems, URL: Tim Wilson, The Cost of Downtime, Internet Week, July 30, 1999, URL: Zona Research, Inc., The Economic Impact of Unacceptable Web Site Download Speeds, November, 1999, URL: Zona Research, Inc., The Need for Speed, June, 1999, URL: Page 10