CSE-427 Advanced Software Engineering Report on Project-2 Load Testing Using JMeter Prepared fordr. Shazzad Hosain Prepared bym Aminur Rahaman Mahmudur Rahman Group- 3
Performance Testing To understand performance testing, let s try to define some fundamental definitions of the word Performance. Performance- from the mechanical point of view performance is the manner or quality of functioning a machine's performance. A sustained phenomenon or one marked by gradual changes through a series of states. So we see performance as a combination of different functionalities responding as solely or together under certain situation. Performance in computer system concept- System performance is an interesting concept; everyone seems to define it differently. To some, it involves chewing through a complex spreadsheet. To others, it's how fast a 3D video sequence can be rendered, or how easily Web pages are served up. As much as I focused on megahertz and gigahertz for much of my computing life, the most important criteria for me these days are how fast the thing turns on, and how long it stays on before I have to recharge it. It goes against everything we grew up with, of course, as the PC industry has always been obsessed with the electronic era equivalent of brute force horsepower, rather than eco-friendly efficiency metrics. But when I'm sitting in a deserted airport terminal with an hour's worth of work to power through and only 45 minutes of remaining battery life, the last thing I'm thinking about is processor performance. Performance in Software Engineering concept- Performance is one of the hottest topics in the IT industry. Many people can write computer code; few can write software that's scalable. You can write a small application within a few days or weeks. But what happens to such software when the number of users grows from two to 10,000? In software engineering, performance testing is testing that is performed, from one perspective, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage. Performance testing is a subset of Performance engineering, an emerging computer science practice which strives to build performance into the design and architecture of a system, prior to the onset of actual coding effort. Performance is an indicator of how well a software system or component meets its requirements for timeliness. There are two important dimensions to software performance timeliness, responsiveness and scalability. Responsiveness is the ability of a system to meet its objectives for response time or throughput. The response time is the time required to respond to stimuli (events). The throughput of a system is the number of events processed in some interval of time Scalability is the ability of a system to continue to meet its response time or throughput objectives as the demand for the software function increases.
Purposes: Demonstrate that the systems meets performance criteria. Compare two systems to find which performs better. Measure what parts of the system or workload cause the system to perform badly. Goals of Performance Testing: Many performance tests are undertaken without due consideration to the setting of realistic performance goals. The first question from a business perspective should always be "why are we performance testing?". Performance failures can be roughly classified as the following three categories: l the lack of performance estimates, l the failure to have proposed plans for data collection, l the lack of a performance budget. Performance goals will differ depending on the application technology and purpose however they should always include some of the following:concurrency or throughput: If an application identifies end-users by some form of login procedure then a concurrency goal is highly desirable. By definition this is the largest number of concurrent application users that the application is expected to support at any given moment. The work-flow of your scripted transaction may impact true application concurrency especially if the iterative part contains the Login & Logout activity. Server Response Time: This refers to the time taken for one application node to respond to the request of another. A simple example would be a HTTP 'GET' request from browser client to web server. In terms of response time this is what all load testing tools actually measure. It may be relevant to set server response time goals between all nodes of the application landscape. Render response time: A difficult thing for load testing tools to deal with as they generally have no concept of what happens within a node apart from recognising a period of time where there is no activity 'on the wire'. To measure render response time it is generally necessary to include functional test scripts as part of the performance test scenario which is a feature not offered by many load testing tools.
Fig- Performance testing process Factors Affecting Application Performance Database design and code can affect performance greatly, so the next four articles in this series will teach you how to optimize this part of your application. However, performance tuning doesn't start or end at the database level. Application performance problems can be caused and mitigated at any of the following areas: Network topology and throughput Server hardware configuration (for database and web servers) Front-end application code developed in Active Server Pages, Visual Basic, C++, PowerBuilder, Cold Fusion, or other languages Middle-tier components, typically written in Visual Basic or C++ Database code, typically developed in T-SQL or PL-SQL SQL Server (or other RDBMS) configuration settings Logical and physical database design Operating system settings Client hardware Overall application architecture Any combination of the items above As you might guess, performance tuning is not an easy exercise and might take the efforts of a fairly large team.
Performance Testing Sub-genres PERFORMAN CE Load Stress Spike Configuration Isolation Soak Load Testing This is the simplest form of performance testing. A load test is usually conducted to understand the behavior of the application under a specific expected load. This load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration. This test will give out the response times of all the important business critical transactions. If the database, application server, etc are also monitored, then this simple test can itself point towards any bottlenecks in the application software. Load testing is the process of putting demand on a system or device and measuring its response. The term is often used synonymously with software performance testing, reliability testing, and volume testing.
A Load Test by definition Emulates real-life interaction between users and server systems Creates a model of what a typical user does within the application and measures the performance and reliability of the server Monitors and quantifies the performance of an internet application under expected and peak user conditions Measures the stability, responsiveness and throughput of the computer system in a realistic test environment Generates different types of traffic expected, in the volumes expected, using actual computer hardware and software Brings the entire functionality of the e-business web site under scrutiny in order to identify the general areas of possible bottleneck occurrences Measures the performance of the web servers and/or the firewalls Allows testers to draw useful conclusions from the test results for improving the site's components Finally, it is entirely different from functional and regression testing. Functional testing determines whether the system operates as expected in general - a single user mode and Regression testing determines whether an upgraded version operates as expected, but load testing determines whether system operates as expected when used by 100, 200, or a 1000 or more simultaneous users. Fig- Load testing scenario
Load testing Process 1. Determine load testing research question(s): Should we upgrade to new server or software? Where are bottlenecks in current system? How many concurrent users without failure? 2. Determine the system data to collect Response time, CPU usage, RAM usage, Page errors 3. Determine the testing tool to use Basic: JMeter, WebApp Stress Tool Enterprise-class: Mercury Interactive, etc. 4. Determine baseline data values What is the hardware/software configuration of web app? What are load test results under normal conditions? 5. Collect experimental data values Collect load test data with new load, hardware, or software 6. Determine changes (if any) required to system. A. Web app symptoms and typical underlying problems: Symptoms of web app problems: Long user response time Long server response time Memory leaks High CPU usage Too many open connections Length queues for requests Too many table scans of database Database deadlocks Erroneous data returned HTTP errors Pages not available B. Typical Database Problems: Insufficient indexing Tune database indexing to improve query processing Fragmented databases Place table records into adjacent pages. Out-of-date statistics Degrade query optimizer performance Faulty application design Excessive DB calls, excessive data requests
C. Typical Web Server Problems: Poor server design Inefficient data or page caching Use of Active Server Pages Memory problems Physical memory constraints High CPU usage Usage > 70% indicates problems D. Typical app server problems: Poor database tuning Application server sending too many DB requests Poor cache management Produces high CPU usage, disk access Poor session management Produces high CPU usage, disk access Time-outs Poor security design Excessive use of https protocol. E. Network Problems: Requires specialized testing to uncover (must stress network without causing application/db stress). Potential sources of network problems: Firewall throughput Internet access throughput Load balancers, gateways, routers Stress Testing This testing is normally used to understand the upper limits of capacity within the application landscape. This kind of test is done to determine the application's robustness in terms of extreme load and helps application administrators to determine if the application will perform sufficiently if the current load goes well above the expected maximum. System stress test" refers to tests that put a greater emphasis on robustness, availability, and error handling under a heavy load, rather than on what would be considered correct behaviour under normal circumstances. In particular, the goals of such tests may be to ensure the software does not crash in conditions of insufficient computational resources (such as memory or disk space), unusually high concurrency, or denial of service attacks.
Fig- Stress testing scenario Endurance Testing (Soak Testing) This test is usually done to determine if the application can sustain the continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation. That is, to ensure that the throughput and/or response times after some long period of sustained activity are as good or better than at the beginning of the test. Spike Testing Spike testing, as the name suggests is done by spiking the number of users and understanding the behavior of the application; whether performance will suffer, the application will fail, or it will be able to handle dramatic changes in load. Configuration Testing Configuration testing is another variation on traditional performance testing. Rather than testing for performance from the perspective of load you are testing the effects of configuration changes in the application landscape on application performance and behaviour. A common example would be experimenting with different methods of load-balancing.
Isolation Testing Not unique to performance testing but a term used to describe repeating a test execution that resulted in an application problem. Often used to isolate and confirm the fault domain. Performance vs. Load vs. Stress- A Comparison Many times people use these terms interchangeably, but they have in fact quite different meanings. The goal of performance testing is not to find bugs, but to eliminate bottlenecks and establish a baseline for future regression testing. To conduct performance testing is to engage in a carefully controlled process of measurement and analysis. Ideally, the software under test is already stable enough so that this process can proceed smoothly. In the testing literature, the term "load testing" is usually defined as the process of exercising the system under test by feeding it the largest tasks it can operate with. Load testing is sometimes called volume testing, or longevity/endurance testing. Stress testing tries to break the system under test by overwhelming its resources or by taking resources away from it (in which case it is sometimes called negative testing). The main purpose behind this madness is to make sure that the system fails and recovers gracefully -- this quality is known as recoverability. A web application scenario: Performance : 1000 logins per minute with 1000 users concurrent Load : How many logins when 2000, 3000, 4000 concurrent users. Stress : How many logins when 1000, 2000, 3000 concurrent users... under conditions like server memory very low, data transmission line poor, etc...
Benefits of Load Testing Minimize the risk of deploying systems that don't meet quality and performance requirements Find the root cause of performance bottlenecks and system failures with ease Optimize user experience by testing against service levels to ensure that Service Level Agreements are met in production Reduce the costs of defects by testing remote components early in the development cycle Minimize hardware and software costs by accurately predicting system capacity Provides performance and capacity statistics Tests scalability of the site Allows you to evaluate the bandwidth for the amount of data sent and received Allows you to verify predicted performance under varying conditions Verifies the tuning of your system through benchmarking Load Testing Strategy Poor software quality carries with it tremendous costs to an organization. Proper load testing can help to mitigate the costs of poor quality. We will explore the importance of load testing, when in the software development process to load test and how optimal performance can be achieved by proper load testing. A Jupiter Media Metrix survey of more than 2,000 users (shown in Figure 1 Jupiter Media Metrix Consumer Study, below) found that a full 46 percent of users permanently or temporarily abandoned their relationship with a preferred site because of technical difficulties or performance problems.
Typically, the load testing strategy is driven by available budget, rather than the business need or criticality of the application. This is not ideal because, as with anything, you get what you pay for. The main strategies employed for load testing are: Manual load testing In-house developed load testing applications Open-source load testing tools Testing frameworks integrated within IDEs Web-only load testing tools Hosted load testing services Enterprise-class load testing solutions PHASE Strategy and Planning Test Development Test Execution SCOPE Defining your performance objectives and how they will be measured and tested Developing test scripts which simulate your users' behaviour Executing your test scripts to simulate user load, interpreting you results and tuning your implementation In the context of load testing, we should emphasize the extreme importance of having large datasets available for testing. Many important bugs simply do not surface unless you deal with very large entities such thousands of users in repositories. So we need to use automated load testing tools. The high cost of commercial tools for performance testing has motivated the emergence of open-source tools for this purpose. These tools are reliable, fully featured, and applicable to most load testing projects. The tools simulate thousands of users interacting with Web, application and database servers; create realistic models simulating different business environments. Two ways to perform load testing Using web application (online) Using desktop application (offline)
Fig- Manual l Strategy for Success Load and performance testing is an indispensable activity for optimizing the quality of a software application. Companies that run mission critical applications cannot afford the high cost of application failures and unnecessary infrastructure maintenance overhead particularly if the applications are visible externally and used for revenue generating purposes. Additionally, failures experienced by external audiences leave an organization even more vulnerable to competitive pressures. Thus, to fully realize the benefits of load testing, it is important that an organization view quality holistically, across the entire software application lifecycle including the production environment. The earlier load testing starts in the application life cycle, the higher its return on investment.
Fig- Optimum Performance Above figure illustrates how time and money can be wasted once the tuning of an application extends beyond the point at which end-users are satisfied during peak loads Fig- Optimum Performance-to-Tuning Cost Ratio per Module Above figure illustrates how adhering to the 8-second-per-page rule doesn t automatically lead to success. It s necessary to analyze and optimize applications module-by-module and page-by-page. Frequently used pages with minimal end-user interaction (for example, the clicking of a single hyperlink) require better response time than do infrequently used pages that involve complex input. There are many types of load testing, each of which performs a given function. For example, some load tests might test peak stress levels for an entire application by simulating the maximum expected number of users of the application. Others might
maintain a steady state number of users for days, as a way to uncover memory leaks. Still other load tests might stress a single application component, e.g. a middleware component such as a Web service, in order to verify its performance singly, before testing performance of the overall composite application. Regardless of the specifics, all load tests aim to make an application more reliable by identifying where, when and under what circumstances the application breaks. Typical Load Testing Environment Load testing takes place in a client/server environment. The attributes of the client and server systems will impact your load test design. The basic components of a load testing environment are 1) Users are individuals who use an application 2) Client is a combination of hardware and software used by the user (browsers, network, connections, etc) 3) Server refers to system that provide service requests to clients application servers, database servers The below diagram shows a load-testing environment for a typical web application and a number of driver machines, connected together through a local and/or wide area network. Three conditions must be met before load testing can begin: 1) the load-testing environment the Web application to be tested, the network, and the load-driving machines needs to be adequately equipped 2) the virtual users must be set up to perform tasks in the same way as real users 3) the Web browser emulation used must be as accurate as possible
Basic Types of Load Testing To make an accurate diagnosis of a performance problem in an e-business application, it is usually necessary to run a sequence of tests that progressively focus in on the particular problem. 1. End-to-end Testing Is generally the first type of testing executed Consists of a global examination of the entire Web application Is designed to bring the entire functionality of the Web application under scrutiny in order to identify the general areas where bottlenecks are Can use a variety of load test types and workload types depending on the context most often, stress tests or stability tests, deploying constant or increasing numbers of virtual users are used 2. Module Testing Typically follows the end-to-end testing Separately tests specific modules of an application to accurately pinpoint any problems Generally uses isolation tests that are designed to bombard the suspect module with loads large enough to force repeated occurrences of a particular problem Generate very specific and detailed information, allowing the tester to accurately diagnose the problem
Load Testing Goals & Results The goal of any load test should be clearly understood and documented. A load test usually fits into one of the following categories: 1. Quantification of risk - Determine, through formal testing, the likelihood that system performance will meet the formal stated performance expectations of stakeholders, such as response time requirements under given levels of load. This is a traditional Quality Assurance (QA) type test. Note that load testing does not mitigate risk directly, but through identification and quantification of risk, presents tuning opportunities and an impetus for remediation that will mitigate risk. 2. Determination of minimum configuration - Determine, through formal testing, the minimum configuration that will allow the system to meet the formal stated performance expectations of stakeholders - so that extraneous hardware, software and the associated cost of ownership can be minimized. This is a Business Technology Optimization (BTO) type test To speak in load testing tool terms, load testing is to Determine each Transaction* Response time* under specific load Determine each user Session time* under specific load Determine Transactions/sec * within Load test simulation time Determine Errors/sec* within Load test simulation time Determine application overall Throughput* during Load test simulation time Finally, analyzing all the above Load test results under various Loads and determine how much load (maximum # of users) your Application can withstand without its performance getting degraded. Depending on these results, you can plan how your production environment should like and scale it up whenever needed. *Transaction a unique business query/event matching application requirements *Response time it is the time taken from a user initiating a request to the instant at which the first part of the response is received at by the application. *Session time it is the time taken by the application for each user to execute all transactions specified under load test scenario *Transactions/sec generally rate of no. of successful transactions per sec. Might differ for each load test tool, for few it is categorized as TransacationOK/sec which means successful transactions and Transactions/sec might mean successful and failed transaction rate. *Errors/sec No. of errors load test generated during simulation time. It need not be only transaction failures but can be actual errors application has generated (bugs), script errors mismatched with actual result, network issues etc. *Throughput it is the overall transaction time taken from a user initiating a request to the instant at which the last part of the response is received at by the application
Major Load Testing Tools (in the 93% of Market Share): HP s (earlier MERCURY INTERACTIVE) - LoadRunner [54%] IBM RATIONAL s Performance Tester [11%] COMPUWARE s QA load [9%] BORLAND s (earlier SEGUE) - Silk Performer - 7% EMPIRIX s e-test suite [6%] RADVIEW s WebLoad [3%] Open Source Load Testing Tools There are several load testing tools available in the open source industry that are free of charge. Only a few of them provide a script recorder that allow testers to easily record user actions into a script through a GUI and replay the scripts simultaneously as a load test. Many others offer a very limited scripting interface, which is also not extensible. While these tools may satisfy very basic needs for simple Web application testing, they lack many capabilities that are absolutely necessary for load testing mission critical applications. According to eweek [10], the main weaknesses in these tools are their lack of advanced scripting options for testing complex applications and their often-limited reporting options. Due to the lack of high-level scripting APIs, scripts also become very long. This, in turn, makes the scripts difficult to maintain. Many of the open source tools are also based on Java, a technology that carries significant performance overhead. This, in turn, leads to higher hardware/resource costs for maintaining multiple load testing driver machines. Open source tools are generally point solutions that focus only on stressing a Web site or application. They do not offer an integrated solution for functional/regression testing, application performance monitoring and management, and overall test management. Finally, open source tools also lack a supporting ecosystem to provide the necessary technical support and services to help organizations successfully implement load testing.
Load testing with Apache JMeter We have used JMeter to perform load test for a web application. Before moving on to the testing description, there are some important issues to be discussed. Apache JMeter is open source software, a 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions. Stefano Mazzocchi of the Apache Software Foundation was the original developer of JMeter. He wrote it primarily to test the performance of Apache JServ (a project that has since been replaced by the Apache Tomcat project). We redesigned JMeter to enhance the GUI and to add functional-testing capabilities. What can we do with it? Apache JMeter may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load. JMeter is not a browser. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc, but the timings are not included in any samples, and only one sample in one thread is ever viewed at a time). Excellent cost saving solution for small projects as it is an open source tool. Robust in handling complex test scenarios that demand n number of virtual users. Less scripting efforts as compared to other tools because of its user effective GUI. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load. Complete portability and 100% Java purity. Full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups. Moreover it s absolutely free and can be modified at ease as it s an open source.
JMeter Setup Requirements JMeter requires your computing environment meets some minimum requirements. A. Java: JMeter requires a fully compliant JVM 1.5 or higher. Because JMeter uses only standard Java APIs, please do not file bug reports if your JRE fails to run JMeter because of JRE implementation issues. B. OS: JMeter is a 100% Java application and should run correctly on any system that has a compliant Java implementation. JMeter has been tested and works under: Unix (Solaris, Linux, etc), Windows (98, NT, XP, etc), OpenVMS Alpha 7.3+ C. Optional: If you plan on doing JMeter development, then you will need one or more optional packages listed below. a. Java compiler b. SAX XML Parser c. Email Suppoort d. SSL Encryption e. JDBC Driver f. JMS Client g. Libraries for ActiveMQ JMS Using JMeter A. Building a test plan: A test plan describes a series of steps JMeter will execute when run. A complete test plan will consist of one or more Thread Groups, logic controllers, sample generating controllers, listeners, timers, assertions, and configuration elements. I. Adding and removing element: Adding and removing elements: Adding elements to a test plan can be done by right-clicking on an element in the tree, and choosing a new element from the "add" list. Alternatively, elements can be loaded from file and added by choosing the "merge" or "open" option. To remove an element, make sure the element is selected, right-click on the element, and choose the "remove" option. II. Loading and saving elements: Adding and removing elements: Adding elements to a test plan can be done by right-clicking on an element in the tree, and choosing a new element from the "add" list. Alternatively, elements can be loaded from file and added by choosing the "merge" or "open" option. To remove an element, make sure the element is selected, right-click on the element, and choose the "remove" option. III. Running a test plan: To run your test plan, choose "Start" (Control + r) from the "Run" menu item. When JMeter is running, it shows a small
green box at the right hand end of the section just under the menu bar. You can also check the "Run" menu. If "Start" is disabled, and "Stop" is enabled, then JMeter is running your test plan (or, at least, it thinks it is). The numbers to the left of the green box are the number of active threads / total number of threads. These only apply to a locally run test; they do not include any threads started on remote systems when using client-server mode. Error reporting: JMeter reports warnings and errors to the jmeter.log file, as well as some information on the test run itself. Just occasionally there may be some errors that JMeter is unable to trap and log; these will appear on the command console. If a test is not behaving as you expect, please check the log file in case any errors have been reported (e.g. perhaps a syntax error in a function call). Sampling errors (e.g. HTTP 404 - file not found) are not normally reported in the log file. Instead these are stored as attributes of the sample result. The status of a sample result can be seen in the various different Listeners. IV. B. Elements of a test plan: The Test Plan object has a checkbox called "Functional Testing". If selected, it will cause JMeter to record the data returned from the server for each sample. If you have selected a file in your test listeners, this data will be written to file. This can be useful if you are doing a small run to ensure that JMeter is configured correctly, and that your server is returning the expected results. The consequence is that the file will grow huge quickly, and JMeter's performance will suffer. This option should be off if you are doing stresstesting (it is off by default). If you are not recording the data to file, this option makes no difference. You can also use the Configuration button on a listener to decide what fields to save. I. Thread Group: Thread group elements are the beginning points of any test plan. All controllers and samplers must be under a thread group. Other elements, e.g. Listeners, may be placed directly under the test plan, in which case they will apply to all the thread groups. As the name implies, the thread group element controls the number of threads JMeter will use to execute your test. The controls for a thread group allow you to: Set the number of threads Set the ramp-up period Set the number of times to execute the test Each thread will execute the test plan in its entirety and completely independently of other test threads. Multiple threads are used to simulate concurrent connections to your server application. The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds
after the previous thread was begun. If there are 30 threads and a rampup period of 120 seconds, then each successive thread will be delayed by 4 seconds. Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen). Start with Ramp-up = number of threads and adjust up or down as needed. By default, the thread group is configured to loop once through its elements. Version 1.9 introduces a test run scheduler. Click the checkbox at the bottom of the Thread Group panel to reveal extra fields in which you can enter the start and end times of the run. When the test is started, JMeter will wait if necessary until the start-time has been reached. At the end of each cycle, JMeter checks if the end-time has been reached, and if so, the run is stopped, otherwise the test is allowed to continue until the iteration limit is reached. Alternatively, one can use the relative delay and duration fields. Note that delay overrides start-time, and duration over-rides end-time. II. Samplers: Samplers tell JMeter to send requests to a server and wait for a response. They are processed in the order they appear in the tree. Controllers can be used to modify the number of repetitions of a sampler. JMeter samplers include: FTP Request HTTP Request JDBC Request Java object request LDAP Request SOAP/XML-RPC Request WebService (SOAP) Request Each sampler has several properties you can set. You can further customize a sampler by adding one or more Configuration Elements to the Test Plan. If you are going to send multiple requests of the same type (for example, HTTP Request) to the same server, consider using a Defaults Configuration Element. Each controller has one or more Defaults elements (see below).
Remember to add a Listener to your test plan to view and/or store the results of your requests to disk. If you are interested in having JMeter perform basic validation on the response of your request, add an Assertion to the sampler. For example, in stress testing a web application, the server may return a successful "HTTP Response" code, but the page may have errors on it or may be missing sections. You could add assertions to check for certain HTML tags, common error strings, and so on. JMeter lets you create these assertions using regular expressions. III. Listeners: Listeners provide access to the information JMeter gathers about the test cases while JMeter runs. The Graph Results listener plots the response times on a graph. The "View Results Tree" Listener shows details of sampler requests and responses, and can display basic HTML and XML representations of the response. Other listeners provide summary or aggregation information. Additionally, listeners can direct the data to a file for later use. Every listener in JMeter provides a field to indicate the file to store data to. There is also a Configuration button which can be used to choose which fields to save, and whether to use CSV or XML format. Note that all Listeners save the same data; the only difference is in the way the data is presented on the screen. Listeners can be added anywhere in the test, including directly under the test plan. They will collect data only from elements at or below their level. There are several listeners that come with JMeter. JMeter Pros & Cons Pros: Open source Active development Free Cross-platform Extensible architecture support plug-ins Good project to help work on. Best free alternative I could find. Cons: Documentation marginal How to automate 100 s of separate v-users with separate session IDs? Unclear how to detect failure (I.e. 404s)--- maybe assertion?
Target Application Details for Load Testing Project We have created a website for an Online Blood Bank Management System. It has been developed using php script along with mysql database. The website is not hosted yet. So we have tested its load testing functionalities using local server. Some features of the website are provided below: i. Two types of user (member & admin). ii. Separate access rights for each user group. iii. Options for admin: a) New Blood Collection b) Supply Blood c) Blood Bank Information d) New Blood Request iv. Options for member: a) Searching for blood b) Donate blood for requested v. Unique login Id & Password This website is a demo version and a simple one. The main objective of this site is to test its performance under different kinds of load scenarios. Using this website, we may perform both GET & POST method against specific form of a page. We may also test the database performance.
Step by step load testing procedure Quick glance of our website: Homepage Member Sign In Form
Member Homepage Blood Bank Search Form
Search Result Donate Blood
Admin Homepage Blood Collection Form
Blood Bank Entry Info Request for Blood Form
Using JMetre for Load Test: Add Thread Group (virtual client) Adding HTTP Request Default (local server description)
Adding HTTP Request (webpage address) Using POST method to pass parameter with the requested page
Again Using POST method to pass parameter with the requested page Adding Listener for Viewing Results
Start Running Threads Analysis Results by Result Tree
View Results in Aggregate Graph Analysis Result in Aggregate Graph
Conclusion Load testing is the only way and plays an important role in analyzing the system performance for a specific load (no. of simultaneous users). Before an application is planned to go public - typically a client/server application where the no. of users accessing it are larger in number, unpredictable and random, there is a desperate need to perform load testing and define the maximum load/simultaneous users it can sustain without its performance getting degraded. Once this performance measured is defined as a requirement, we can definitely plan to scale up our production environment to meet whatever additional load of users that keep bombarding our application. References 1. Fundamentals of Load testing Search links from Google 2. SilkPerformer Modeling & Implementing Load tests student guide (Segue) 3. Apache JMeter Home Page, http://jakarta.apache.org/jmeter/ 4. JMeter Wik, http://wiki.apache.org/jakarta-jmeter/