Performance Testing of Java Enterprise Systems Katerina Antonova, Plamen Koychev Musala Soft Why Performance Testing? Recent studies by leading USA consultancy companies showed that over 80% of large corporations reported cases of performance problems in their systems. Small and medium enterprises lose an average 1% of their annual revenue due to unplanned downtimes. At the same time, user requirements become higher and higher - nearly 50% of them expect the response time of the web systems less than 3 seconds. All this makes the performance critical to the business. Companies are facing serious challenges short time for development, growing complexity of applications and pressure to rapidly deploy in production. Page 1
Today s software industry is characterized by a variety of solutions and technologies. The technologies have an influence in determining the test strategy. Each component is important and everything can cause performance bottleneck - connection between application tiers, actions and impact of external services and components, database structure, business logic implementation, etc. Performance Testing Process Performance testing is consistent and iterative process it can be performed as many times as necessary to confirm that the system meets efficiency, capacity and performance criteria. Performance Testing Tools The selection of appropriate tool is an important aspect of the performance test planning. The key criteria for selection of appropriate tool for today s complex Java Enterprise systems are: Protocols and standards support HTTP/HTTPS for Web applications, SOAP for Web Services, JMS, MQ Monitoring capabilities System Resources, JEE Application Servers, Databases Data correlation automated relation between test data for the dynamic parameters, for example the usage of session_id as unique parameter for all transactions in one business operation. Parameterization and data variation data pools, data tables or other capabilities to generate unique or random data, especially in case of large data volume. Page 2
IDE and scripting language useful IDE and support of industry-known scripting language. Performance Test Tools Architecture Most testing tools have similar architecture. Leading commercial tools: HP LoadRunner IBM Rational Performance Tester Others - RadView WebLoad, Borland SilkPerformer, SmartBear LoadComplete, NRG AppLoader Open source tools: Apache Jmeter Others SmartBear LoadUI, AppPerfect Load Test Test Infrastructure Building and configuration of the target testing system is important element of the test infrastructure preparation. The parameters of the test environment should be the closest possible to the production one. This includes specification and numbers of physical servers (CPU numbers, RAM), network infrastructure (bandwidth and connectivity), realistic volume of test data. There are also other elements, which should be specified and prepared: Page 3
Topology of application servers Product versions Internal or external systems and components emulation As part of the infrastructure there should be prepared enough hardware resources, needed to generate the expected load. The sharing of virtual resources and load balancing may need IP Spoofing/IP Aliasing implementation. There are also identified the required capacity and resources for scripting, test management and test analysis. Key Performance Indicators Effective performance testing requires well defined performance indicators (Service Level Agreements metrics), against which measurements, diagnostic and analysis are performed. Examples for key performance indicators are: Availability (uptime) the system availability to end users at expected load levels Concurrency number of active users at a given time Scalability system capacity at specific concurrency and load levels Тhroughput number of requests at a given time (per minute or per second hits per second, bytes per second) Response time response time of the system for specific operation/transaction from end user perspective Resource utilization CPU utilization, memory use, Disk I/O, Network I/O Page 4
Test Design There are three major aspects when building the performance test design. The first one is identification of the business scenarios business-critical, intensive and heavy transactions. Negative and incomplete transactions should also be considered. The next aspect is preparation of sufficient volume and variety of test data. The third one is definition of the load profile number of virtual users for transaction, expected iteration numbers for a given time, think time (delay/pauses between transactions), pacing time (delay/pauses between iterations), and also users injection profile - big-bang (all users start simultaneously), ramp-up (predefined number of users starting at specified intervals while meeting the target number of active users), ramp down (similar to ramp-up but in users stopping mode). Example test design: Test Scripts One of the practices for scripts development is by recording the user actions using the record features of the test tools. Another is manual scripting by using the supported programming language. Test scripts are customized for: Parameterization and data variation Data correlation Conditional logic Checkpoints The already completed test scripts should be executed with one or more users before being executed on test environment. Page 5
Test Schedules Completed scripts should be organized into test scenarios, which implement test design. This includes setup and configuration of test duration and number of virtual users, user groups, load generators, monitors and statistics. Test Execution and Monitoring During the performance test execution, load is generated and the test infrastructure behavior is monitored in real time key performance indicators and configured monitors and statistics. The most important monitoring parameters are: Transactions hits per second, response time, running users Resource utilization - CPU utilization, Memory use, Disk IO, Network IO Application server - JVM Heap, Thread pools, JDBC connection pools, JMS connection pools Database - SQL execution time, cache hit ration, locks timeouts, deadlocks, escalations Analysis In order to track what happened during the execution and to build complete picture for the system behavior is needed to gather all available information. The results should be correlated by analyzing how the indicators are changed when load is increased. This helps us to evaluate the system scalability. If a problem is identified, a deeper diagnostic should be performed, in order to identify the cause. The results are compared to those from the previous runs, then the changes resulting from the diagnostic and tuning are evaluated. Page 6
Performance Diagnostic There are many techniques for diagnostic. Example for diagnostic techniques for Java Enterprise systems are: - Thread Dump Analysis synchronization problems are detected - Heap Dump Analysis the code which may cause the memory leaks is isolated - DB Execution/Access Plan non-optimized SQL, missing indexes Reporting Leading test tools provide reach possibilities for automated report generation. Report preparation is the last step of the performance test process. Results are reported to and evaluated by all stakeholders. There are provided conclusions about the system performance in order to confirm its readiness for production. Page 7