Bus u i s n i e n s e s s s Cas a e s, e, S o S l o u l t u io i n o n & A pp p r p oa o c a h

Transcription

1 Work Load Modeling and Work Load Modeler in Performance Testing

2 Business Case, Solution & Approach Case An application is made ready to go-live in the next 2 months, but the application performance behavior with real-time and varying user load is not available. Solution Application performance behavior prior to its actual go-live with real-time and varying user loads need to be understood Approach A BETA version of the same application is made available to the end users and feedback by the end user pertaining to application s responsiveness is recorded. All the logs and other related data pertaining to the application are monitored and examined closely. Some varying patterns can be observed, due to application specific behavior at varying user loads. Work load modeling technique need to be used here, to arrive at an optimal throughput or user load with which the application can be subjected to a realistic user load. The performance tests carried out using this model will simulate the application performance behavior as in production.

3 Introduction What is Work Load Modeling? Work load modeling is a process that determines application usage pattern in production or which is expected after deployment in production. Performance testing scenarios will be executed as per the Work load model to identify the bottlenecks. Performance testing & Work Load Model (WLM) The work loads used during performance testing must represent the real world production scenario, so that the results obtained are proportional to understand the performance characteristics of an application in production. Business context for the use of the application, expected transaction volumes in various situations, expected user path(s) by volume and other usage factors must be understood thoroughly to create an accurate representation of real world application usage Contd..

4 Introduction Why, WLM is required? Work load modeling is mandatory to make sure that the calculated user load or throughput in the test environment represents an equivalent (scaled down) of production environment. Throughput in production environment can be calculated using the number of users using the application during peak usage, peak usage duration, response times of transactions in the peak usage duration and think time in production environment. Work load model also captures the transaction distribution, load distribution and load pattern.

5 Background Little s law can be used to derive the work load modeling statistics. According to Little s law - Actual Number of Users in the System (N) = (Transactions per second or Throughput (X))* (Response Time (R) + Think Time (Th)) N = X (R + Th) The average number of requests in a system is equal to the product of the throughput of that system and the average time spent in that system by a request. N - number of users should be simulated, to carry out performance testing of an application. Performance testing tools help in simulating the users. The system throughput and response time are obtained.

6 Work Load Modeling - Approach Identify the critical scenarios for the application from a performance perspective. This should be done as a part of the requirements analysis stage. Identify the set of operations/scenarios performed on the application by an average user. For each of the identified scenarios the following inputs should be captured: Number of Users using the application in peak hour: The total number of concurrent and simultaneous users accessing the application in a given time frame. Patterns of requests: A given load of concurrent users may be performing different tasks using the application. Patterns of requests identify the average load of users, and the rate of requests for a given functionality of an application. Think Time: The user views the information displayed on a page or enters details such as personal details, depending on the nature of action being performed during this time. It varies depending on the complexity of the data on a page. Response time SLA of transactions in that peak hour Contd..

7 Work Load Modeling - Approach Total number of servers which host the application. Work load model and the transaction load are calculated. Divide the throughput among all the production servers so that the throughput for one single server can be obtained. Scale down the throughput according to the test environment by multiplying the throughput of single server with the number of servers in test environment. Multiply the throughput with the percentage of load to arrive at the final throughput for a particular transaction. Calculate the user load in test environment, using throughput, response time and think time for that particular transaction. Similarly, calculate the user load for all the transactions and perform the load test. The results obtained will give the throughput almost similar to the production environment. NOTE: 5% of deviation will be considered with expected throughput.

8 Different cases of Work Load Modeling Work load modeling (WLM) technique can be applied for variety of cases to arrive at a work load model. The 5 most important and common cases depending upon the data available are identified and work load modeler is designed this to suit for those cases to arrive at the work load model. In all these cases, one needs to identify number of users and throughput in the test environment for a single transaction when the corresponding production data is available Case 1 Case 2 Case 3 Case 4 Case 5

9 Case Study of a Banking Application Below are the details shared by the client for one of the real time scenarios - Number of production Servers: 8 Number of Test Environment Servers: 2 Number of Concurrent users (Same is considered as peak load): 2000 Response time for every transaction: 2 sec The load distribution among different transactions is as follows, S. No Transaction Name % of load(100) 1 Transaction Transaction Transaction Transaction Transaction5 30 Table (1) Think time assumed as 5 sec Contd..

10 Case Study of a Banking Application Calculation of Throughput for both Production and Test Environments: Throughput of the application in Production Environment - According to little s law, N = X (R + Th) => 2000 = Z (2 + 5) Throughput (Z) = 2000/7 = trans/sec Throughput for each server = 285.7/8(No. of Prod servers) = 35.7 trans/sec Throughput of the application in Test Environment - Total throughput in Test Environment = Throughput for each server * 2 = 35.7*2 = 71.4 trans/sec The user load based on the throughput in test environment should be applied and the expected throughput of the application should be matched with production to create the real time scenario. 5% deviation in the throughput will be considered in this case. Contd..

11 Case Study of a Banking Application Calculation of Number of Users for each transaction in Test Environment: Throughput for individual transactions - Throughput (for a particular transaction) = % of load * Throughput of servers in Test Environment. For Example: Throughput of transaction1 = 20% of = Transactions Per Second (TPS) Number of Virtual Users for Transaction1 = Throughput of transaction1 * (Response time + Think time) N = *(2+5) = 100 Contd..

12 Case Study of a Banking Application Similarly, proceeding for other transactions Transaction % load Expected Response Time (sec) (R) Work Load Model Think Time (sec) (T) Throughput for each Transaction ( Z) (TPS) Number of Users N (N=Z(R+T)) Transaction Transaction Transaction Transaction Transaction Total 100 NA NA Table (2) Performance of the application in production can be known, by performing a load test in the Test Environment with above user load (Column: 6). The expected throughput will be approximately equal, when compared to the production volumes.

13 Work Load Modeler A work load modeler using excel macro is developed, which will calculate the work load (Number of Users or Throughput) for an application under test (AUT) or system. The work load modeler will save time and reduce manual work. Manual calculations in the above example takes lot of time to come up with final numbers. Manual effort is reduced using the work load modeler macro, which will automatically calculate all the Throughput or Number of Users. Please find below Work Load Modeler Microsoft Excel Macro-Enabled Worksheet Contd..

14 Work Load Modeler Working with Work Load Modeler Microsoft Excel 2003 or above version are required, to work with this work load modeler. Note: 1. In case of Microsoft Excel 2007, make sure that the macros are enabled with the help of macro settings or options. 2. In case of Microsoft Excel 2010, make sure to save this file with.xlsm (Microsoft Excel Macro-Enabled Workbook) extension. Please follow the below steps to run the work load modeler: 1. Open the file and make sure that macros are enabled 2. Enter the data as per the cases (data availability) above, in the Basic Data sheet Contd..

15 Work Load Modeler Example: Parameter - (Value) *Number of Servers in Production Environment - (8) **Throughput in Production Environment - ( ) **Number of business days considered for 'Throughput' in Prod - (2) **Number of business hours considered in a business day - (8) (And/or) **Peak Usage Percentage (Throughput) - (30) **Peak Usage Duration considered (Hours) - (2) (And/or) **Number of Concurrent Users in Production - (2000) (Or) Contd..

16 Work Load Modeler **Peak Time Production Throughput - (100000) **Peak Time Duration (Hours) - (2) *Number of Transactions under consideration - (10) *Number of Servers in Test Environment - (2) *Response time for every transaction (Sec) - (2) *Think time to be assumed for every transaction (Sec) - (3) Note: * denotes a mandatory field whereas ** denotes a case specific field 3. Click on Submit button 4. The work load model for the specified case will be obtained in the sheet WLM

17 Benefits of Work Load Modeling and Work Load Modeler Work load Modeling is Useful to identify business critical transactions and their usage pattern. Useful to replicate a scaled-down version of production. Useful to simulate realistic scenarios. Simple and efficient because it is based on simple operational laws. Platform and environment independent. Work load modeler Useful to calculate the user load or throughput. Saves time and reduces manual effort. Useful to handle all types of realistic scenarios.

18 References

19 Thank You Amith Kumar Arangi Suresh Suru

20 Case 1 When only the below parameters are available. Throughput in production Number of business days for which the throughput is observed Number of application servers in production environment Number of application servers in test environment Number of transactions under consideration Expected response time. Expected think time.

21 Case 2 When only the below parameters are available. Throughput in production Number of business days for which the throughput is observed Percentage distribution of throughput for each scenario. Number of hours for which the percentage distribution is considered. Number of application servers in production environment Number of application servers in test environment Number of transactions under consideration Expected response time. Expected think time.

22 Case 3 When only the below parameters are available. Percentage distribution of throughput for each scenario. Number of hours for which the percentage distribution is considered. Number of concurrent users in production. Number of application servers in production environment Number of application servers in test environment Number of transactions under consideration Expected response time. Expected think time.

23 Case 4 When only the below parameters are available. Number of concurrent users in production. Number of application servers in production environment Number of application servers in test environment Number of transactions under consideration Expected response time. Expected think time.

24 Case 5 When only the below parameters are available. Peak time production throughput Peak time duration considered for the peak time production throughput Number of application servers in production environment Number of application servers in test environment Number of transactions under consideration Expected response time. Expected think time.