Web Application s Performance Testing

Web Application s Performance Testing B. Election Reddy (07305054) Guided by N. L. Sarda April 13, 2008 1

Contents 1 Introduction 4 2 Objectives 4 3 Performance Indicators 5 4 Types of Performance Testing 5 5 Workload Creation 6 5.1 Steps involved in workload characterization........... 6 5.2 Developing Performance Test Cases............... 6 6 Performance Testing Activities 7 7 Tools 8 7.1 OpenSTA............................. 8 7.2 JMeter............................... 8 7.3 Comparison of LoadRunner and OpenSTA........... 9 7.4 GNU/Linux Commands for Performance Testing of a System. 10 8 Conclusion 10 2

Abstract Performance Testing is very important for web applications. Objectives of performance testing and Approaches used for it are discussed. Core activities involved in performance testing are discussed. Performance testing commands in GNU/Linux are explained. Tools used for performance testing like LoadRunner, OpenSTA and JMeter are analysed. 3

1 Introduction In Software Engineering Process Models, Testing is an important phase to guarantee the quality of the product. In most of the software projects the greatest part of the testing effort goes to functional testing and some projects don t do performance testing at all. For Web Applications, performance testing is crucial as the functioning of the application highly depends on the load conditions. The application may perform well at average load but may crash at high loads. So before actual deployment, performance testing will enable the developer to identify the bottlenecks in the system and can be rectified. 2 Objectives Assess production readiness: Performance testing enables to predict the performance characteristics of an application in production and evaluate whether or not to address performance concerns based on those predictions. These predictions are also valuable to the stakeholders who make decisions about whether an application is ready for release or capable of handling future growth, or whether it requires a performance improvement/hardware upgrade prior to release. Evaluate against performance criteria: Calculated Performance metrics like response time, throughput, resource utilization can be evaluated against desired values. In general, response time is a user concern, throughput is a business concern, and resource utilization is a system concern. Compare performance characteristics of multiple systems or system configurations: Performance metrics can be used to compare web applications. A web application s performance can be compared with different configurations so that best configuration can be identified. Find the source of performance problems: Bottlenecks in the web application can be identified by performing different types of testings. Bottlenecks for web applications can be code, database, network, peripheral devices. Support system tuning: By providing information related to scalability and stability of a product prior to the production release, Perfor- 4

mance testing enables to make informed decisions about whether and when to tune the system. 3 Performance Indicators Resource utilization: The percentage of time a resource(cpu, Memory, I/O, Peripheral, Network) is busy Throughput: The number of event responses that have been completed over a given interval of time Response time: The time elapsed between a request and its reply. It is a measure of how responsive an application or subsystem is to a client request. Database access rates: The number of times database is accessed by web application over a given interval of time Scalability: The ability of an application to handle additional workload, without adversely affecting performance, by adding resources such as processor, memory, and storage capacity 4 Types of Performance Testing Performance tests are broadly divided into following categories [6] Performance testing: This type of testing determines or validates the speed, scalability, and/or stability characteristics of the system or application under test. Performance is concerned with achieving response times, throughput, and resource-utilization levels that meet the performance objectives for the project or product. Load testing: It focuses on determining or validating performance characteristics of the system or application under test when subjected to workloads and load volumes anticipated during production operations. Stress testing: It focuses on determining or validating performance characteristics of the system or application under test when subjected to conditions beyond those anticipated during production operations. It also tests the performance of the system under stressful conditions, such as limited memory, insufficient disk space, or server failure. These 5

tests are designed to determine under what conditions an application will fail, how it will fail, and what indicators can be monitored to warn of an impending failure. Capacity testing: It is used to determine how many users and/or transactions a given web application will support and still meet performance goals. 5 Workload Creation Workload characterization consists of a description of the workload by means of quantitative parameters and functions. The objective is to derive a model able to show, capture, and reproduce the behaviour of the workload and its most important features. It is sometimes referred to Operation Profile in software engineering literature. Operation Profile is a probability distribution describing the frequency with which selected important operations are exercised. It described how the system has historically been used in the field and thus is likely to be used in the future[3]. 5.1 Steps involved in workload characterization 1. Model the software system 2. Collect the data, analyse it, identify the key parameters and probability distribution 5.2 Developing Performance Test Cases Identify the software processes that directly influence the overall performance of the system For each such process determine the input parameters that will most significantly influence the performance of the system Determine realistic values for these parameters by collecting and analysing existing usage data If there is no historical data available then estimate the reasonable data based on experience 6

If for a given parameter, the estimated values form a range, then select representative values from within this range that are likely to effect the system performance more. Each selected value will form a separate test case 6 Performance Testing Activities Performance testing is a complex activity that can not be shaped into onesize-fits-all approach. But most of the tests will include the following common activities[6]. 1. Identifying the Test Environment: Identify the physical test environment and the production environment as well as the tools and resources available to the test team. The physical environment includes hardware, software, and network configurations. Thorough understanding of the entire test environment at the outset enables more efficient test design and planning and helps to identify testing challenges early in the project. In some situations, this process must be revisited periodically throughout the project s life cycle. 2. Identifying Performance Acceptance Criteria: Identify the response time, throughput, and resource utilization goals and constraints. In general, response time is a user concern, throughput is a business concern, and resource utilization is a system concern. Additionally, identify project success criteria that may not be captured by those goals and constraints; for example, using performance tests to evaluate what combination of configuration settings will result in the most desirable performance characteristics. 3. Planning and Designing Tests: Identify key scenarios, determine variability among representative users and how to simulate that variability, define test data, and establish metrics to be collected. Consolidate this information into one or more models of system usage to be implemented, executed, and analyzed. 4. Configuring the Test Environment: Prepare the test environment, tools, and resources necessary to execute each strategy as features and components become available for test. Ensure that the test environment is instrumented for resource monitoring as necessary. 5. Implementing the Test Design: Develop the performance tests in accordance with the test design. 7

6. Executing the Test: Run and monitor your tests. Validate the tests, test data, and results collection. Execute validated tests for analysis while monitoring the test and the test environment. 7. Analysing Results, Reporting and Retesting: Analyze the data both individually and as a cross-functional team. Reprioritize the remaining tests and re-execute them as needed. 7 Tools 7.1 OpenSTA OpenSTA(Open System Testing Architecture) is a distributed software testing architecture designed around CORBA. It has the capability of performing scripted HTTP and HTTPS heavy load tests with performance measurements. It is available only for Windows OS.[4] 7.2 JMeter Apache JMeter is a pure Java desktop application designed to load test functional behaviour and measure performance. It was originally designed for testing Web Applications and further expanded to other test functions. It is Platform independent.[5] Used to 1. test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more) 2. simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types 3. test server/script /object behaviour under heavy concurrent load 4. make a graphical analysis of performance 8

7.3 Comparison of LoadRunner and OpenSTA Item Description LoadRunner OpenSTA Playback Replaying of the script Extended logging, supports Similar playback functions and script debugging view of parameter values facilities, but no integrated facilities and Server messages. Also comparison view and comparison with recorded version of web page view and client re- IP Spoofing Caching User connection speed emulation Cost System requirements Cookie Management Detection, recording and playback of HTTP cookies. Both tools need additional code to manage JavaScript The ability to emulate the behaviour of different IP addresses accessing a system. Particularly useful with Load balancing systems. The ability to emulate the caching of pages as performed by a web browser. The ability to emulate the different network speeds that can be used by real users Purchase costs of software and licenses, excluding upgrades or support. The operating systems required to host the tool. The hardware requirements to host the tool. sponse messages. Debugging facilities in script generator, step and breakpoints. HTTP header cookies are managed automatically and can be manipulated manually if required. Supports IP spoofing including automated router updates for IP forwarding. Can control browser cache emulation during playback and control setting for each individual vuser. Can emulate different network speeds during playback Controller costs $22000. Additional charge for each protocol and monitored resource and for virtual users. Linux, Windows, Unix, Solaris function. The debugging functions are in the controller, including set break points and single stepping. HTTP header cookies are managed automatically and can be manipulated manually if required. No inbuilt features to enable IP spoofing. No specific faculties, although can be emulated with simple script code. No inbuilt features to emulate userconnection speed emulation. Can be downloaded freely from net. Windows Hardware requirements RAM - Load generators: et 80MB RAM. Pref: Min: Pentium 350, 128MB Min : Pentium 200 Pentium 1GHz, 1MB per Pentium 500MHz+ VU. 128MB+ RAM. Table Source: Andy O Brien and Antony Marcano of etest associates [7]

7.4 GNU/Linux Commands for Performance Testing of a System ps - reports a snapshot of the current processes in the system top - provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel sar - displays performance data for CPU, run queue, disk I/O, paging (swap), memory, CPU interrupts, networking, and more uptime - gives the current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes vmstat - reports virtual memory statistics, information about processes, memory, paging, blockio, traps, and cpu activity iostat - reports CPU statistics and input/output statistics for devices and partitions free - displays the amount of free and used memory in the system slabtop - displays the kernel slab cache information in real time I have analysed the output of above commands on systems running Application Server and Database Server. Data is provided by my guide. The data and report on that data is available at[8]. 8 Conclusion Performance testing is crucial for web applications. Passing the functional testing does not guarantee the success of web application since its performance depends on the load. Functionally well tested application may also sometimes fail after deployment. So performance testing is important for the success of a web application. Performance testing enables the developer to identify bottlenecks in the web application before deployment so that he/she can improve its performance before release. Many tools are available to test the performance of web applications. These tools will automatically analyse and report the results of performance tests. Workload characterisation is the main issue in performance testing. The goal of performance testing is not finding bugs, but to remove the bottlenecks from the application and improve the efficiency. 10

References [1] Rukma Prabhu Verlekar, Varsha Apte A Methodology and Tool for Performance Analysis of Distributed Server Systems [2] Thomas Zwanzinger Testing and Improving Web Application Performance [3] Xiang Gan Software Performance Testing [4] OpenSTA web page visited on 06-03-2008 http://www.opensta.org/ [5] Apache web page visited on 06-03-2008 http://jakarta.apache.org/jmeter/ [6] Microsoft web page visited on 10-04-2008 http://msdn2.microsoft.com/en-us/library/bb924375.aspx [7] etest associates web page visited on 9-04-2008 http://www.etest-associates.com [8] My home page http://www.cse.iitb.ac.in/~electionreddy/seminar/performancetesting/ 11