Web Application Testing Web Performance Testing
Objectives of Performance Testing Evaluate runtime compliance to performance requirements Check different properties such as throughput (bits/sec, packets/sec) response time (mean time for getting a response after an input) memory utilization input/output rates etc
Types of Performance Testing Stress testing focus on system behavior at, near or beyond overloading conditions push system to failure emphasis near specified load, volume boundaries checks for graceful failures, non abrupt performance degradation Load Testing verifies handling of a particular load while maintaining acceptable response times
Web Performance Testing Performance is key to the success of a web application According to a survey by Zona Research (2001) Most users click away after 8 seconds Potential cost for poor web performance cost $25 billion US dollar revenue lost (in the USA)
Web Performance Testing Crash! Virtually Offline (> 30 sec.) Web Site Response Time Unacceptable (15 30 sec.) Borderline (9 15 sec.) The 8 Second Barrier Good (0-8 sec.) Web Site Load (Number of concurrent users) Source: Presentation by Lloyd Taylor, VP Technology & Operations, Keynote Systems
Web Performance Testing Test Objectives How many users can the system handle while maintaining a reasonable response time? Can the system serve x users without errors?
Web Performance Testing Help determine: The maximum capacity that can be handled while keeping an acceptable performance Whether hardware and/or software performance tuning is needed
Web Performance Testing Phases 1. Planning phase 2. Testing phase 3. Analysis phase
Performance Testing Process Planning phase Define objectives, deliverables, expectations Gather system and testing requirements environment and resources workload (peek, low) acceptable response time Select performance metrics to collect e.g. Transactions per second (TPS), Hits per second, Concurrent connections, Throughput, etc Identify tests to run and decide when to run them Scenarios/Transaction to run Decide on a tool/application service provider option to generate loads Write test plan, design user scenarios, create test scripts
Performance Testing Process Testing Phase Testing phase generate test data set up test bed SUT, test environment performance monitors run tests collect result data
Performance Testing Process Analysis Phase Analyze results to locate source of problems software problems hardware problems Change system to optimize performance software optimization hardware optimization Design additional tests (if test objective not meet)
Tools for performance testing of Web Applications Necessary to simulate loads Large number of concurrent users Record scenario/transaction scripts For regression testing Report results in a way that help analysis Example of tools: Apache JMeter IBM Rational Performance Tester HP Loadrunner...
JMeter GUI desktop application designed to load test functional behavior and measure performance command line execution also possible Originally designed for testing Web Applications but has since expanded to other test functions. Open Source Has a rich graphical interface Built in Java Part of Apache Jakarta project Can be distributed
JMeter Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. http://jakarta.apache.org/jmeter/ current version 2.3.4
Features of JMeter Graphical Analysis / Exporting Test Results Remote Distributed Execution If you want to generate load using multiple test servers. You can run multiple server components of JMeter remotely. And you can control it by a single JMeter GUI to gather results. http://jakarta.apache.org/jmeter/usermanual/remote-test.html Highly Extensible Custom Additions (Write your own samplers / listeners) PlugIns Test plans stored in XML and can be version controlled.
What Can You Do With It? JMeter lets you set up test plans that simulate logging into a web site, filling out forms, clicking buttons, links, etc. You can simulate the number of users doing this, the rate that they do it.
JMeter Control Panel
JMeter User Interface The Test Plan Login Sequence
Test Plan Describes a serie of steps to be executed by JMeter Consists of one or more Thread Groups, logic controllers, samplers, listeners, timers, assertions, and configuration elements. Represented as an Ordered Tree The whole tree or subtrees can be saved in XML format
Scoping Rules Some elements are primarily ordered (e.g. controllers and samplers). Other elements are hierarchical. An Assertion, for example, is hierarchical in the test tree. If its parent is a request, then it is applied to that request. If its parent is a Controller, then it affects all requests that are descendants of that Controller.
Elements of a Test Plan Thread Group controls the number of threads JMeter will use to execute the test used to simulate concurrent connections to the server application (number of concurrent users) a ramp-up period may be defined to specify how long it takes to start the full number of threads in the group It is also possible to define a number of times a test is to be executed
Elements of a Test Plan Controllers drive the processing of a test include Samplers and Logical Controllers Samplers tell JMeter to send requests to a server and wait for a response processed in the order they appear in the tree
Elements of a Test Plan Samplers Tell JMeter to send requests to a server and wait for a response By default, processing is done according the order of appearance in the test plan logical controllers used to alter default Typical samplers: FTP Request HTTP Request JDBC Request Java object request LDAP Request SOAP/XML-RPC Request WebService (SOAP) Request
Elements of a Test Plan Example: HTTP Request for loging
Elements of a Test Plan Logic controllers let customize the logic that JMeter uses to decide when to send requests. Can change the order of requests coming from their child elements. Can modify the requests themselves, cause JMeter to repeat requests, etc.
Elements of a Test Plan Examples of logic controllers Simple Logic Controller Loop Controller Only Once Controller Interleave Controller Random Controller If Controller While Controller Switch Controller ForEach Controller...
View results summary
View results as graph
Elements of a Test Plan Listeners provide access to the information gathered by JMeter while running a test plan can direct the data to a file for later use Sample listeners Graph Results listener Assertion Results visualizer View Results Tree View Results Table Aggregate report Summary report...
Elements of a Test Plan Timer introduces a delay between each request that a thread makes can be used to account for think time Assertion check responses obtained from server "test" that application is returning the expected results can be specified using Perl style regular expressions
Elements of a Test Plan Configuration Element can add to or modify requests (e.g. by providing variable definitions) Pre-Processor executes some action prior to a Sampler Request being made can be used to modify settings, update variables Post-Processor executes some action after a Sampler Request has been made can be used to process the response data, often to extract values from it
Execution order Timers Pre-Processors Sampler Post-Processors Assertions Listeners