APPLICATION DELIVERY LOAD TESTING TO PREDICT WEB PERFORMANCE

Transcription

1 LOAD TESTING TO PREDICT WEB PERFORMANCE

2 ABSTRACT Businesses that leverage the Web to conduct daily transactions need to provide customers with the best possible user experience in order to be successful. Often, however, these businesses lose customers because their sites are unable to handle surges in Web traffic. For example, a successful promotion that drives Web traffic can radically impact business performance and response time to end users. Online customers, tired of waiting, will simply click to competitors sites, and any opportunities for revenue growth will be lost. Whether a business is a brick-and-mortar or a dot.com, the challenges of successfully conducting business online are the same: high user volumes, slow response times for Web requests, and ensuring the overall reliability of the service. This white paper illustrates how maintaining Web-application performance is key to overcoming these e-business challenges and generating revenue. The paper then discusses the importance of maintaining a Web application TABLE OF CONTENTS Abstract 2 E-Business is Booming 3 Ensuring Optimal End-User Experience A Complex Issue 3 Application Load Testing Prior to Going Live 4 Challenges of Automated Load Testing Tools 6 Accuracy 6 Scalability 7 The Process of Automated Load Testing 7 Step 1: System Analysis 7 Step 2: Creating Virtual User Scripts 8 Step 3: Defining User Behavior 9 Step 4: Creating a Load Test Scenario 9 Step 5: Creating Network Impact Tests 9 Step 6: Running the Load Test Scenario and Monitoring the Performance 10 Step 7: Analyzing Results 10 Mercury LoadRunner 10 Step 1: System Analysis 11 Step 2: Creating Virtual User Scripts 11 Step 3: Defining User Behavior 12 Step 4: Creating a Load Test Scenario 13 Step 5: Creating Network Impact Tests 14 Step 6: Running the Load Test Scenario and Monitoring the Performance 14 Step 7: Analyzing Results 15 Summary 18

3 to ensure customer satisfaction and why load testing is critical to successfully launching and managing Web sites. In addition, it examines various types of load testing and provides a detailed discussion on the load testing process and the attributes of a reliable testing tool. In closing, this paper provides an overview of Mercury LoadRunner, our load testing solution. E-Business is Booming In the past few years, e-business has grown at an accelerated rate. Today, analysts estimate that 260 million people use the Internet and there is little sign that this growth will slow down. In fact, the International Data Corporation expects the number of online users to reach 500 million within the next two years. E-business has become a popular commercial medium for two reasons: It enables businesses to share information and resources worldwide, and it offers them an efficient channel for advertising, marketing, and e-commerce. By using the Internet, businesses have been able to improve their sales and marketing reach, increase their quality assurance to customers, and conduct multimedia conversations with customers. More important, businesses are realizing the challenges and rewards of providing customers with a positive end-user experience. After all, customers who are satisfied with their online experience are likely to engage in repeat business and provide a steady stream of revenue. As a result, businesses have become more focused on providing positive end-user experiences. Ensuring Optimal End-User Experience A Complex Issue In addition to being fast-growing, e-business is very complex. According to a December 1999 report by the IBM High-Volume Web site team, commercial Web sites can be classified into four categories based on the types of business transactions they perform: publishing/subscribers, online shopping, customer self-service, and trade/auction sites. By understanding these categories, businesses can better predict their level of user volume and understand how users prefer to access the site. Following is an overview of the different commercial Web site categories: Publishing/subscribers sites provide the user with media information, such as magazine and newspaper publications. Although the total number of concurrent users is generally low on these sites, the number of individual transactions performed on a per-user basis is relatively high, resulting in the largest number of page views of all site categories. Online shopping sites allow users to browse and shop for anything found in a traditional brick-andmortar store. Traffic is heavy, with daily volumes ranging between one and three million hits per day. Customer self-service sites include banking and travel reservation sites. Security considerations (e.g., privacy, authentication, site regulation, etc.) are high. Trade/auction sites allow users to buy and sell commodities. This type of site is volatile, with very high transaction rates that can be extremely time-sensitive. 3

4 No matter the transaction type, Web sites must enable customers to conduct business in a timely manner. For this reason, a scalable architecture is essential. A well-structured Web environment, however, consists of an extremely complex multi-tier system. Scaling this infrastructure from end-to-end means managing the performance and capacities of individual components within each tier. Figure 1 illustrates the complexity of these components. INTERNET CLIENTS ROUTERS SWITCHES WEB SERVERS INTERNET FIREWALL LOAD BALANCERS APPLICATION SERVERS DATABASE SERVERS & OTHER DATABASE SOURCES Figure 1. Pictured is the schematic of a complex Web infrastructure. This complexity prompts many questions about the integrity and performance capabilities of a Web site. For instance, will the response time experienced by the user be less than eight seconds? Will the Web site be able to sustain a given number of users? Will all the pieces of the system, in terms of interoperability, coexist when connected together? Is communication between the application server and the database server fast enough? Is there sufficient hardware on each tier to handle high volumes of traffic? Will the clients have quality experiences over wide area networks (WANs)? To eliminate these performance issues, businesses must implement a method for predicting how Web applications will behave in a production environment, prior to deployment. Application Load Testing Prior to Going Live To accommodate the growth of their sites, Web developers can optimize software or add hardware to each component of the system. However, to ensure optimal performance, businesses must load test the complete assembly of a system prior to going live. Application load testing is the measure of an entire Web application s ability to sustain a number of simultaneous users and/or transactions, while maintaining adequate response times. Because it is comprehensive, load testing is the only way to accurately test the end-to-end performance of a Web site prior to going live. 4

5 Application load testing enables developers to isolate bottlenecks in any component of the infrastructure. Two common methods for implementing this process are manual and automated testing. Manual testing, however, has several built-in challenges, such as determining how to: Emulate hundreds of thousands of manual users that will interact with the application to generate load. Coordinate the operations of users. Measure response times. Repeat tests in a consistent way. Compare results. Because load testing is iterative in nature, you must identify performance problems, tune the system, and retest to ensure that tuning has had a positive impact countless times. For this reason, manual testing is not a very practical option. With automated load testing tools, tests can be easily rerun and the results automatically measured. In this way, automated testing tools provide a more cost-effective and efficient solution than their manual counterparts. Plus, they minimize the risk of human error during testing. Today, automated load testing is the preferred choice for load testing a Web application. The testing tools typically use three major components to execute a test. These include: A control console, which organizes, drives, and manages the load. Virtual users, which are processes used to imitate the real user performing a business process on a client application. Load servers, used to run the virtual users. INTERNET VIRTUAL USERS WEB SERVER APPLICATION SERVER DATABASE MANUAL TESTERS CONTROL CONSOLE Figure 2. A single console controlling several thousand virtual users replaces manual testers. 5

6 Using these components, automated load testing tools can: Replace manual testers with automated virtual users. Simultaneously run many virtual users on a single load-generating machine. Automatically measure transaction response times. Easily repeat load scenarios to validate design and performance changes. This advanced functionality in turn allows you to save time and costly resources. Automated testing tools recently demonstrated their value in a report by the Newport Group. The report, published in 1999, revealed that 52 percent of Web-based businesses did not meet their anticipated Web-based business scalability objectives. Of this group, 60 percent did not use any type of automated load testing tool. In contrast, nearly 70 percent of businesses that met their scalability expectations had used an automated load testing tool. Challenges of Automated Load Testing Tools Primary challenges of load testing tools include the ability to be accurate and scalable and to isolate performance problems. To isolate performance problems, load testing tools monitor key system-level components and identify bottlenecks during the run of a load test. Accuracy is defined by how closely an automated tool can emulate real user behavior. Scalability relates to the product s ability to generate the maximum load using the minimum amount of resources. DID YOUR APPLICATION SCALE AS EXPECTED? FROM 48% DID YOU USE AN AUTOMATED LOAD TESTING TOOL? FROM 52% DID YOU USE AN AUTOMATED LOAD TESTING TOOL? YES 48% NO 52% NO 30% YES 70% YES 40% NO 60% Figure 3. Automated load testing enables businesses to meet scalability expectations. Automated load testing tools must address all aspects of accuracy and scalability and be able to pinpoint problems in order to ensure reliable end-to-end testing. Following are some key attributes of accuracy and scalability. 6

7 ACCURACY Recording ability against a real client application Capturing protocol-level communication between the client application and the rest of the system Providing flexibility and the ability to define user behavior configuration (e.g., think times, connection speeds, cache settings, iterations) Verifying that all requested content returns to the browser to ensure a successful transaction Showing detailed performance results that can be easily understood and analyzed to quickly pinpoint the root cause of problems Measuring end-to-end response times SCALABILITY Generating the maximum number of virtual users that can be run on a single machine before exceeding the machine's capacity Generating the maximum number of hits per second against a Web server Managing thousands of virtual users Increasing the number of virtual users in a controlled fashion Simulate the effect of scaling out to remote locations over WANs Using real-life data Synchronizing virtual users to generate peak loads Monitoring different tiers of the system with minimal intrusion Figure 4. Accuracy and scalability are key attributes in load testing. The Process of Automated Load Testing By taking a disciplined approach to load testing, you can optimize resources; better predict hardware, software, and network requirements; and set performance expectations to meet end-user service-level agreements (SLAs). Repeatability of the testing process is necessary to verify that changes have taken place. Following is a step-by-step overview of the automated load testing process: Step 1: System Analysis This step is critical to interpreting your testing needs and is used to determine whether the system will scale and perform to your expectations. Testers essentially translate existing requirements of the user into load testing objectives. A thorough evaluation of the requirements and needs of a system, prior to load testing, will provide more realistic test conditions. First, you must identify all key performance goals and objectives before executing any testing strategies. Examples include identifying which processes/transactions to test, which components of a system architecture to use in the test, and the number of concurrent connections and/or hits per second to expect against the Web site as well as clarifying which processes are to be tested. By referring to the four models of Web sites (see page 3), developers can easily classify their site s process/transaction type, allowing users to conduct transactions in a more timely fashion. For example, a business-to-consumer model can implement an online shopping process in which a customer browses through an online bookstore catalog, selects an item, and makes a purchase. This process could be labeled buy book for the purposes of the test. Defining these objectives will provide a concise outline of the SLAs and mark the goals to be achieved with testing. 7

8 Second, you need to define the input data used for testing. The data can be created dynamically. For example, auctioning bids may change every time a customer sends in for a new request. Random browsing also may be used to obtain the data. Examples include any non-transactional process, such as browsing through a brochure or viewing online news. Emulating data input can avoid potential problems with inaccurate load test results. Third, you must determine the appropriate strategy for testing applications. You can select from three strategy models: load testing, stress testing, and capacity testing. Load testing is used to test an application against a requested number of users. The objective is to determine whether the site can sustain this requested number of users with acceptable response times. Stress testing, on the other hand, is load testing over extended periods of time to validate an application s stability and reliability. The last strategy is capacity testing. Capacity testing is used to determine the maximum number of concurrent users an application can manage. For example, businesses would use capacity testing to benchmark the maximum loads of concurrent users their sites can sustain before experiencing system failure. Fourth, testers need to cultivate a solid understanding of the system architecture, including: Defining the types of routers and network connectivity used in the network setup. Determining whether multiple servers are being used. Establishing whether load balancers are used as part of the IP networks to manage the servers. Finding out which servers are configured into the system (Web, application, database). Last, developers must determine which resources are available to run the virtual users. This requires deciding whether there is a sufficient number of load generators or test machines to run the appropriate number of virtual users. It also requires determining whether the testing tool has multithreading capabilities and can maximize the number of virtual users being run. Ultimately, the goal is to minimize system resource consumption while maximizing the virtual user count. Step 2: Creating Virtual User Scripts A script recorder is used to capture all the business processes into test scripts, often referred to as virtual user scripts or virtual users. A virtual user emulates the real user by driving the real application as a client. It is necessary to identify and record all the various business processes from start to finish. Defining these transactions will assist in the breakdown of all actions and the time it takes to measure the performance of a business process. 8

9 Step 3: Defining User Behavior Run-time settings define the way that the script runs in order to accurately emulate real users. Settings can configure think time, connection speed, and error handling. Think times can vary in accordance with different user actions and the user s level of experience with Web technology. For example, novice users require more time to execute a process because they have the least experience using the Web. Therefore, a tester will need to emulate more think time in the form of pauses. Advanced users, however, have much more experience and can execute processes at an accelerated level, often by using shortcuts. System response times also can vary because they are dependent on connection speed, and all users connect to the Web system at different speeds (e.g., modem, LAN/WAN). WAN emulation accurately simulates a variety of connections at varying network bandwidth and latencies (e.g., 28.8 Kbps, 56.6 Kbps, etc.). This is very useful for determining how the underlying network affects application response times. Error handling is another setting that requires configuration. Errors arise throughout the course of a scenario and can impede the test execution. You can configure virtual users to handle these errors so that the tests can run uninterrupted. Errors in network communications can also have a profound influence on application response times. You can also configure WAN emulation to introduce underlying network errors to understand their impact and measure the application s tolerance for them. Step 4: Creating a Load Test Scenario The load test scenario contains information about the groups of virtual users that will run the scripts and the load machines on which the groups are running. In order to run a successful scenario, you must first define individual groups based on common user transactions. Second, you need to define and distribute the total number of virtual users. A varying number of virtual users can be assigned to individual business processes to emulate user groups performing multiple transactions. Third, you must determine which load-generating machines the virtual users will run on. Load-generator machines can be added to the client side of the system architecture to run additional virtual users. Last, testers need to specify how the scenario will run. Virtual user groups can either run in staggered or parallel formation. Staggering the virtual users allows you to examine a gradual increase of the user load to a peak. Step 5: Creating Network Impact Tests The network test leverages information about where the groups of virtual users will be located relative to the server. The network scenario defines changes in certain network characteristics, such as bandwidth availability, contention, latency, errors, and jitter. The number of virtual users in this test is held constant; only network characteristics will vary. Staggering the decreases in network bandwidth or increases in latency, errors, and jitter enables you to understand their relative influence on application behavior. This data can be used to set network requirements for the application when it is deployed. This kind of testing 9

10 can be conducted directly over the network to remote locations, but for testing purposes it is generally more practical to emulate the network where a variety of conditions using WAN emulation can be easily established. This will allow the prediction of performance for remotely located users. Step 6: Running the Load Test Scenario and Monitoring the Performance Real-time monitoring allows testers to view the application s performance at any time during the test. Every component of the system requires monitoring: the clients, the network, the Web server, the application server, the database, and all server hardware. Real-time monitoring allows for early detection of performance bottlenecks during test execution. You then have the ability to view the performance of every single tier, server, and component of the system during testing. As a result, you can instantly identify performance bottlenecks during load testing. You can then accelerate the test process and achieve a more stable application. Step 7: Analyzing Results This is the most important step in collecting and processing the data to resolve performance bottlenecks. The analysis yields a series of graphs and reports that help summarize and present the end-to-end test results. For example, Figure 5 uses generic data to display a standard performance under load graph that shows the total number of virtual users against the response time. This can be used to determine the maximum number of concurrent users until response times become unacceptable. Figure 6 shows a transaction overview revealing the total number of transactions that passed in a scenario. Analysis of these types of graphs can help testers isolate bottlenecks and determine which changes are needed to improve system performance. After these changes are made, the tester must rerun the load test scenarios to verify the adjustments. Mercury LoadRunner Mercury LoadRunner, part of Mercury Performance Center, is a load testing tool that predicts system behavior and performance. It exercises an entire enterprise infrastructure by emulating thousands of users to identify and isolate problems. Able to support multiple Figure 5. This is a generic graph showing performance under load. This graph is useful in pinpointing bottlenecks. For example, if a tester wants to inquire about the user threshold at two seconds, the results above show a maximum of 7,500 concurrent users. Figure 6. This graph is a generic display of the number of transactions that passed or failed. In the above example, if the goal is to obtain a 90-percent passing rate for the number of transactions, then transaction 2 fails. Approximately 33 percent out of 100 transactions failed. 10

11 environments, LoadRunner can test an entire enterprise infrastructure, including e-business, ERP, CRM, and custom client/server applications, thereby enabling IT and Web groups to optimize application performance. By emulating the behavior of a real user, LoadRunner can test applications communicating with a wide range of protocols, such as HTTP(s), COM, CORBA, Oracle Applications, etc. LoadRunner also features a seamless integration with Mercury Business Availability Center. Therefore, the same tests created during testing can be reused to monitor the application once it is deployed. LoadRunner enhances every step of the load testing process to ensure that users reap the maximum return on their investment in the tool. The remainder of this paper discusses how LoadRunner offers support for each segment of the load testing process: Step 1: System Analysis LoadRunner advocates the same system analysis as mentioned previously in this paper. In emulating a test environment, it is necessary to identify all testing conditions, including system architecture components, the processes being tested, and the total number of virtual users with which to test. A good system analysis will enable customers to convert their goals and requirements into a successful, automated test script. Step 2: Creating Virtual User Scripts You begin by recording the business processes to create a test script. Script recording is done using LoadRunner s Virtual User Generator (VUGen). VUGen is a component that runs on a client desktop to capture the communication between the real client application and the server. VUGen can emulate the exact behavior of a real browser by sending various e-business protocol requests to the server. VUGen also can record against Netscape or Internet Explorer browsers or any user-defined client that provides the ability to specify a proxy address. After the recording process, a test script is generated. Figure 7. The Virtual User Generator allows testers to capture business processes to You can then add logic to the script to make it create virtual users. more realistic. Intelligence can be added to the scripts so that they emulate virtual user reasoning while executing a transaction. LoadRunner executes this stage using the transactions, as well as its verification and parameterization features. Transactions: Transactions represent a series of operations that are required to be measured under load conditions. A transaction can be a single URL request or a complete business process leading through several screens, such as the online purchase of a book. 11

12 Verification: VUGen allows insertion of verification checkpoints using ContentCheck. ContentCheck verifies the application functionality by analyzing the returned HTML Web page to ensure a successful transaction. If the verification fails, LoadRunner will log the error and highlight the reasons for the failure (e.g., broken link, missing images, erroneous text). Parameterization: To accurately emulate real user behavior, LoadRunner virtual users use varying sets of data during load testing, replacing constant values in the script with variables or parameters. The virtual user can substitute the parameters with values from a data source, such as flat files, random numbers, date/time, etc. This allows a common business process, such as searching for or ordering a book, to be performed many times by different users. Step 3: Defining User Behavior LoadRunner provides comprehensive run-time settings to configure scripts that emulate the behavior of real users. Examples of run-time settings include: Think Time: Controls the speed at which the virtual user interacts with the system by including pauses of think times during test execution. By varying think times for users, LoadRunner can emulate the behaviors of different users from novice to expert users. Dial-Up Speed: Emulates a user connected to the system using a modem and/or LAN/WAN connections. Modem speeds range from 14.4 Kbps to 56.6 Kbps. This is useful for controlling user behavior in order to accurately emulate response times for each request. Figure 8. The run-time settings are used to emulate the real user as closely as possible. In this example, think time is randomly generated to simulate the speed at which the user interacts with the system. Emulate Cache: Emulates a user browsing with a specific cache size. Caching can be turned off based on server requirements. Browser Emulation: Enables you to specify which browser the virtual user emulates. LoadRunner supports both Netscape and Internet Explorer, as well as any custom browser. Number of Connections: Allows the virtual user to control the number of connections to a server, like a real browser, for the download of Web-page content. IP Spoofing: Tests the performance impact of IP-dependent components by assigning virtual users their own IP addresses from the same physical machine. 12

13 Iterations: Commands repetition of virtual user scripts. Also paces virtual users, instructing how long to wait between intervals. Iterative testing defines the amount of work a user does based on the number of times a process is performed using varying data. Error Handling: Regulates how a virtual user handles errors during script execution. LoadRunner can enable the Continue on Error feature when the virtual user encounters an error during replay. Log Files: Stores information about virtual user server communication. Standard logging maps all transactions, rendezvous, and output messages. Extended logging also tracks warnings and other messages. Step 4: Creating a Load Test Scenario LoadRunner s Controller is used to create scenarios. As a single point of control, it provides complete visibility of the tests and the virtual users. The Controller facilitates the process of creating a load test scenario by allowing you to: Assign scripts to individual groups. Define the total number of virtual users needed to run the tests. Define the host machines on which virtual users are running. In addition, LoadRunner offers a Scenario Wizard, Figure 9. LoadRunner s Controller is an interactive environment for organizing, driving, a Scheduler, and TurboLoad to enhance the tester and managing the load test scenario. experience. Scenario Wizard. LoadRunner s Scenario Wizard is a feature that enables testers to quickly compose multi-user load test scenarios. Using five easy-to-follow screens, the Scenario Wizard steps you through a process of selecting the workstations that will host the virtual users as well as the test scripts to run. During this step-by-step process, you also create simulation groups of virtual users. Scheduler: LoadRunner Scheduler is used to ramp virtual user numbers up/down in order to position virtual users in both the ready state and the running state. For example, you may want to gradually increase the load of users logging into a site with a fixed batch size. This is referred to as the ready state. This method is useful for avoiding unnecessary strain on the system. The schedule also manages scheduling and features an automated process that allows the user to run the script without being present. In real time this would be analogous to running a script during offpeak hours of Internet traffic 6 p.m. to 6 a.m. To schedule a test, you simply click the Run Scenario button and enter the desired starting time. 13

14 TurboLoad: TurboLoad is a patent-pending technology that provides for maximum scalability. TurboLoad can minimize CPU consumption for each virtual user, thereby enabling more users to run on each loadgenerator machine. In recent customer benchmarks, using 10 Windows-based load servers (4 CPU, 500 MHz Xeon processors, 4 GB RAM), LoadRunner generated 3 billion Web hits per day against the Web system (or 3,700 hits/sec per machine). Moreover, the load generators were running at less than 40 percent CPU utilization. TurboLoad also can generate more hits/sec for a given machine. LoadRunner s replay speed can thereby generate more throughput against the server using a minimum amount of resources. Step 5: Creating Network Impact Tests With LoadRunner WAN emulation, the same virtual-user scripts used in the previous steps are leveraged for network impact tests. Network characteristics such as connection speed, latency, and error rates are modified for groups of virtual users that are simultaneously emulated during a single test run. The impact of the network on response time on the different groups, and the sensitivities of the application to the network, can then be accurately ascertained. Expected response time data can be recorded and network requirement set for use later when the application is deployed. Step 6: Running the Load Test Scenario and Monitoring the Performance Once the scenario is built, you are ready to run the test. LoadRunner s Controller provides a suite of performance monitors that can monitor each component of a multi-tier system during the load test. By capturing performance data over the entire system, you can then correlate this information with the end-user loads and response times in order to pinpoint bottlenecks. LoadRunner provides performance monitors for the network, network devices, and the most common Web servers, application servers, and database servers. The performance monitoring is done in a completely non-intrusive manner to minimize performance impact. Additionally all of these monitors are hardware and OS independent, as they do not require that agents be installed on the remotely monitored servers. Figure 10. LoadRunner online monitors help identify and isolate performance bottlenecks in real time. 14

15 LoadRunner supports a number of environments to provide for more accurate online monitoring, including: Runtime Graphs: Virtual User Status, User Defined Data Points Transaction Graphs: Response Time, Transactions (pass/fail) Web-Server Resource Graphs: Hits per Second, Throughput, Apache, MS IIS, Netscape System Resource Graphs: Server resource, SNMP Tuxedo Web-Application Server Graphs: BroadVision, ColdFusion, MS Active Server Pages, SilverStream, WebLogic Database Server Resource Graphs: SQL Server, Oracle Step 7: Analyzing Results Evaluating results is the most important step in the load testing process. Up until now, you have been able to record and play the actions of a real user with extreme precision, while conducting multiple processes on the Web. In addition, the performance-monitoring feature offers an accurate method for pinpointing bottlenecks while running the scripts. To fix these problems, you can follow several steps. First, a network specialist (DBA, consultants) can make the necessary adjustments to the system. As a next step, you need to rerun the scripts, to verify that the changes have taken place. Last, a comparison of the results from before and after enables the tester to measure the amount of improvement that the system has undergone. LoadRunner s Analysis component provides a single integration environment that collectively gathers all the data generated throughout the testing cycle. Because this tool is powerful and easy to use, you can create cross-scenario comparisons of the graphs and thereby enrich the data analysis process. For example, Figure 11a shows the results of a dot.com after testing the maximum number of concurrent users that its existing system can handle. Based on these results, the dot.com plans to improve its infrastructure to allow more user traffic. Figure 11b provides a comparison of a repeated test after adjustments had been made to the Web architecture to optimize server software. Figure11a. Figure11b. 15

16 LoadRunner Analysis provides advanced, highlevel drill-down capabilities that enable testers to locate bottlenecks in these scenarios. In addition, LoadRunner Analysis uses a series of sophisticated graphs and reports that answer such questions as: What was the Web server s CPU memory when the system was under a load of 5,000 concurrent users? How many total transactions passed or failed after the completion of the load test? How many hits per second did the Web server uphold? What were the average transactions times for each virtual user? Below are sample graphs that LoadRunner Analysis provides testers as it solves complex bottleneck issues. Running Virtual Users: Displays running virtual users during each second of a scenario. Rendezvous: Indicates when and how virtual users were released at each point. Figure 12. This activity graph displays the number of completed transactions (successful and unsuccessful) performed during each second of a load test. This graph helps testers determine the actual transaction load on their system at any given moment. The results show that after six minutes an application is under a load of 200 transactions per second. Transaction/Sec (Passed): Displays the number of completed, successful transactions performed per second. Transaction/Sec (Failed): Displays the number of incomplete, failed transactions performed per second. Figure 13. This performance graph displays the number of transactions that passed, failed, aborted, or ended with errors. For example, these results show the Submit_Search business process passed all its transactions at a rate of approximately 96 percent. Figure 14. This performance graph displays the minimum, average, and maximum response times for all the transactions in the load test. This graph is useful in comparing the individual transaction response times in order to pinpoint where most of the bottlenecks of a business process are occurring. For example, the results of this graph show that FAQ business process has an average transaction response time of seconds. This would be an acceptable statistic in comparison to the other processes. 16

17 LoadRunner provides a variety of performance graphs: Percentile: Analyzes percentage of transactions that were performed within a given time range. Performance Under Load: Indicates transaction times relative to the number of virtual users running at any given point during the scenario. Transaction Performance: Displays the average time taken to perform transactions during each second of the scenario run. Figure 15. This Web graph displays the number of hits made on the Web server by Vusers during each second of the load test. This graph helps testers evaluate the amount of load Vusers generate in terms of the number of hits. For instance, the results provided in this graph indicate an average of 2,200 hits per second against the Web server. Transaction Performance Summary: Displays the minimum, maximum, and average performance times for all the transactions in the scenario. Transaction Performance by Virtual User: Displays the time taken by an individual virtual user to perform transactions during the scenario. Transaction Distribution: Displays the distribution of the time taken to perform a transaction. Figure 16. This Web graph displays the amount of throughput (in bytes) on the Web server during the load test. This graph helps testers evaluate the amount of load Vusers generate in terms of server throughput. For example, this graph reveals a total throughput of more than 7 million bytes per second. LoadRunner offers two types of Web graphs: Connections Per Second: Shows the number of connections made to the Web server by virtual users during each second of the scenario run. Throughput: Shows the amount of throughput on the server during each second of the scenario run. Figure 17. This graph correlates the relationship of the system behavior to the number of users, using a compilation of results from other graphs. This enables the tester to view the CPU consumption based on the total number of users. 17

18 LoadRunner s Analysis includes a Correlation of Results feature to enhance the user analysis process of the data. Correlation enables the tester to custom design a graph beyond the basics, using any two metrics. As a result, the tester can pinpoint and troubleshoot performance problems more quickly. Summary In a short time, e-business has proven to be a viable business model for dot.coms and brick-andmortars alike. With the number of Internet users growing exponentially, it is therefore critical for these businesses to prepare themselves for high user volumes. Today s businesses can leverage load testing practices and tools to ensure that Web-application performance keeps pace with end-user demand. Moreover, by using automated load testing tools, businesses can quickly and cost-effectively assess the performance of applications before they go live, as well as analyze their performance after deployment. As a result, businesses can confidently stay one step ahead of performance issues and focus on initiatives to drive Web traffic and revenues. Mercury LoadRunner is the leading tool for predicting scalability, reliability, and performance issues of an e-business application; identifying system bottlenecks; and displaying results. LoadRunner emulates various types of transactions using a highly scalable number of users. This is essential for understanding an application s limitations while planning for growth and reducing business risk. LoadRunner also tests system behavior under real-time conditions and converts this data into easy-to-use, yet sophisticated graphs and reports. With this information, businesses can more quickly and efficiently resolve problems, thereby ensuring a positive end-user experience and providing the opportunity for increased revenue. Download a 10-day trial of Mercury LoadRunner and see how load testing your applications in pre-production will save you time and money when you're ready to go live

19 Mercury Interactive is the global leader in business technology optimization (BTO). We are committed to helping customers optimize the business value of IT Mercury Interactive Corporation. Patents pending. All rights reserved. Mercury Interactive, the Mercury Interactive logo, the Mercury logo, Mercury Business Availability Center, Mercury Performance Center, and Mercury LoadRunner are trademarks or registered trademarks of Mercury Interactive Corporation in the United States and/or other foreign countries. All other company, brand, and product names are marks of their respective holders. WP