SAP CRM WEB UI Performance Testing

Transcription

1 SAP CRM WEB UI Performance Testing Applies to: CRM WEB UI 7.0 version Performance Testing process. For more information, visit the Customer Relationship Management homepage. Summary This document describes about the performance testing of SAP CRM WEB UI using Vugen, Load runner and HP Performance Center (Controller and Load runner) tools. This document consists of Baseline, Load and Stress test procedure for SAP CRM WEB UI. Author: Jataprolu Krishna Prathyusha Company: Accenture Services Private Limited Created on: 12 June 2011 Author Bio Author is having above 5 years of experience in SAP CRM. Author has experience in Sales, Service, Marketing, Interaction Center modules in CRM and has precise expertise in understanding the CRM system SAP AG 1

2 Table of Contents Document Objective:... 3 Introduction:... 3 Types of Performance Testing Related to SAP-CRM Web - UI:... 3 Performance Testing Usage:... 4 Pre-requisites of Performance Testing:... 4 Installation details for Performance Testing:... 4 Test Script Preparation:... 6 Intall Vugen:... 6 Parameterization: Correlation: Uploading the script to Performance Center: Test Script Execution: Test Analysis and Final Report: Issues Faced while Executing PT: Related Content Disclaimer and Liability Notice SAP AG 2

3 Document Objective: This documents describes about the performance testing of CRM WEB UI 7.0 using Loadrunner and HP Performance Center tools. With the rigorous load-testing functionalities of the SAP LoadRunner application, organizations can better deliver high-performing business processes and can execute, upgrade,or modify existing processes on time and within budget. Introduction: Performance Testing is testing that is performed, to determine how fast some aspect or scenario of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage. Performance testing can refer to the assessment of the performance of the system. This is used to determine the speed or effectiveness of a system, network, software program or device. This process can involve quantitative tests like measuring the response time of the server or the Transactions Per Second (TPS) etc., Types of Performance Testing Related to SAP-CRM Web - UI: Types of Performance Testing for SAP CRM WEB UI: There are three types of performance testing can be done for SAP CRM UI. 1) Baseline Testing 2) Load Testing 3) Stress Testing 1) Baseline Testing: Single user executes each of the identified online activity and reports one at a time. The objective of Baseline testing is to determine baseline performance of the application and identify individual actions requires for fine tuning of the application. The Response Time of the server is key measure of this Testing. For Ex: Search accounts on search screen with default option with one user login 2) Load Testing: Load testing is the simplest form of performance testing. A load test is usually conducted to understand the behavior of the application under a specific expected load. This load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration. This test will give out the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software. For Ex: Search accounts on account search screen with default option with n number of users login at the same point of time. 3) Stress Testing: Stress testing is normally used to understand the upper limits of capacity within the application landscape. This kind of test is done to determine the application's robustness in terms of extreme load and helps application administrators to determine if the application will perform sufficiently if the current load goes well above the expected maximum. It will be useful to test, in order to see what happens when an acceptable load is exceeded does the system crash? How long does it take to recover if a large load is reduced? Does it fail in a way that causes collateral damage? Etc., For Ex: Search account on account screen with n number of user login initially and increase the user login till the system crashes 2011 SAP AG 3

4 Performance Testing Usage: Performance testing can serve different purposes. It can demonstrate that the system meets performance criteria. It can compare two systems to find which performs better. It can measure what parts of the system or workload causes the system to perform badly Pre-requisites of Performance Testing: The stable application which must resemble the Production environment is possible to do the performance testing. This is dangerous as if an UAT or Integration test or other tests are going on in the same environment, then the results obtained from the performance testing may not be reliable. As a best practice it is always advisable to have a separate performance testing environment resembling the production environment as much as possible. Test conditions: In Performance Testing, it is often crucial or difficult to create the test conditions similar to the expected actual use. It is not entirely possible in actual practice to exactly replicate production system workload variability in the performance testing application. Timing: The timing of the test conditions also crucial in the performance testing. It is critical to the cost performance of a new system which performance test efforts begin at the inception of the development project and extend through to deployment. The later a performance defect is detected, the higher the cost of remediation to fix the defect. Installation details for Performance Testing: Tools for Performance Testing: HP load runner software is an automated performance and load testing product. It is used to examining the system behavior and performance, while generating actual load. HP Load Runner can emulate hundreds or thousands of concurrent users to put the application through the rigors of real-life user loads, while collecting information from key infrastructure components. The results can then be analyzed in detail, to explore the reasons for particular behavior. HP Load Runner consists of several different tools: Virtual User Generator (VuGen), Controller, load generators, Analysis and Launches. In this SAP CRM WEB UI Performance Testing Virtual User Generator and HP Performance Center have been used to prepare test scripts and execute the scripts accordingly. HP Performance Center login details should be created by the admin team of HP PC. 1. Virtual User Generator: This tool is used to prepare the test scripts for the performance testing. The Virtual User Generator (VuGen) is used to emulate the steps of real human users. VuGen can also run scripts for debugging. VuGen lets the user record and/or script a test to be performed against an application under test, and play back and make modifications to the script as needed, such as defining Parameterization (selecting data for keyword-driven testing). Vugen can be installed on the system from HPQC Performance Center. Vugen is used to record the scenario in WEB UI. After the recording, it will create a code in C language automatically. We can test the scenario from the Vugen itself for one user and can correct it or modify it if any errors occurred at the time of creation. We can even update the code at Vugen according to our scenario. Once the script is running fine in Vugen, we can execute the script from performance center to get the summary of the scenario execution SAP AG 4

5 For Ex: Create a vugen script by executing the scenario like search accounts from account search screen with one user login to WEB UI. HP Load Runner supports more than 51 protocols including Web HTTP/HTTPS, Remote Terminal Emulator, Oracle and Web Services. A protocol acts as a communication medium between a client and a server. For example an AS400 or mainframe-based application can use a terminal emulator to talk to a server, and an on-line banking application can use HTTP/HTTPS with some Java and Web services. Load Runner can record scripts in both single and multi-protocol modes. The protocol used for CRM WEB UI is: SAP-WEB. During recording, VuGen records a tester's actions by routing data through a proxy. The type of proxy depends upon the protocol being used and affects the resulting script. For some protocols, various recording modes are available to further refine the resulting script. When recording web activity, HP Load Runner allows a user to choose one of three types of recording modes: GUI based, URL based and HTML based. The recording mode for CRM WEB UI is HTML based. Parameterization: Load Runner allows users editing a script with VuGen to replace recorded values in a script with parameters. This is called parameterization. Parameterization is often used: 1. When the application needs unique data (such as user name) 2. Data dependency (such as passwords) 3. Data cache 4. Date constraints Correlation: HP Load Runner uses Correlation to handle dynamic content. Dynamic content refers to page components that are dynamically created during the execution of a business process, and the value may differ from the value generated in a previous run. For Examples of dynamic content include the multiple users login to WEB UI and most importantly the unique session ID that is created each time a user logs in. Such dynamic values must be correlated in order to run the recorded script. The script while running will return an error if such a dynamic value has not been correlated, since the value changes each time the user logs in. If the dynamic content is a part of a previous server response, Load Runner can correlate that value with future instances in which it must be sent back to the server. HP Load Runner saves the changing values into parameters, which are used during emulation. For Ex: Create a test script from Vugen for account search scenario for different levels (For Ex: 8 levels) of users in Organization hierarchy. The script will be created to check the timelines for each and every level user to search the accounts created by them at CRM WEB UI. For this scenario one script will be created with account search scenario by passing 8 level users as parameters. 2. Performance Center: HP Performance Center is the de-facto standard for enterprise performance testing. Built on HP Load Runner, it includes the management framework for a web-based, globally accessible platform that facilitates enterprise-wide testing. The login details for HP Performance Center. Using Performance Center, you can standardize and centralize your testing resources and build a testing CoE that supports distributed teams and enables them to collaborate across the application lifecycle. And you can give everyone from executives to LOB managers to testers visibility into project status and application quality SAP AG 5

6 HP Performance Center features and benefits: 1) Tests a range of applications and technology, from rich internet applications (RIA) with Web 2.0 technologies, to legacy applications and technology 2) Pinpoints root-cause application performance problems 3) Helps improve application performance while reducing hardware and software costs 4) Reduces the risk of deploying systems that fail performance requirements 5) Supports single and multi-project-based testing The Test scripts prepared in Vugen can be used to test the Baseline, Load and Stress tests from performance Center to get the timelines, CPU usage and Transactions per Second for that scenario. For Ex: The script created for Account search on account screen for 8 levels of users can be uploaded to Performance Center. The scenario can be executed for specific timelines to check the time taken for each and every level user to search the accounts created by them on CRM UI. Test Script Preparation: Intall Vugen: 2011 SAP AG 6

7 Click on Download to download Vugen software 2011 SAP AG 7

8 After the installation of Vugen software, system will show the mandatory softwares to be installed to use the vugen from the system. Need to install the softwares which do not exist in the system. Then only Vugen will open in the system SAP AG 8

9 Vugen has been installed successfully on the system. Go to Start program and click on Virtual User Generator to access Vugen. Run the Vugen and create a test script for searching accounts on account search screen with created by Me option (login user) SAP AG 9

10 Go to File -> Click on new to create a new script. Select SAP Web protocol to create the test script of CRM WEB UI. If WEB protocol is not available on the screen, click on New Multiple Protocol Script -> Select SAP Web and click on right arrow to get it on the list. Check the recording settings and General settings under Tool menu option before recording the script 2011 SAP AG 10

11 Click on SAP Web to create the script Click on OK to record the script SAP AG 11

12 Vuser_init: This is used to record the login function of the system Action: Action part is used to record the particular action for the scenario. For Ex: here it is used to record the search account screen search functionality action. User_end: This is used to record the logoff function of the system According to the client requirement, the script can be recorded by splitting the login, action and logoff details separately or within the action part itself. In this scenario everything is recorded under Action part. Comments session is used to give the description of each and every action performed while recording. This is useful to understand the process sequence of the application on the coding page. Start Transaction: This option is used to indicate the starting of every transaction End Transaction: This option is used to indicate the ending of the every transaction. Login page is already opening. Click on End Transaction and enter Account Search_login which indicates that the login page action has been completed SAP AG 12

13 Click on Start Transaction to store the next happening transaction after the login process. Click on Start Transaction Enter Account Search_Homepage as Homepage will display after the login. Enter the username and password details and click on Logon button. The events will be recorded on the screen SAP AG 13

14 Homepage will display on the screen. Once the recording events count is stopped, the transaction Account Search_Homepage can be ended by clicking on End Transaction. Next action will be account search screen display. Click on Start Transaction and enter Account Search_Search Screen. Click on Account Search option on the screen which will open the account search screen SAP AG 14

15 Account Search screen has opened. Click on End Transaction and select Account Search_Search Screen as the action has been completed. Next action is to display the Result screen. Click on Start Transaction and enter Account Search_Result Screen. Click on search button to see the results. Click on End transaction to end the Account Search_Result Screen action SAP AG 15

16 Next action is to logoff the system. Click on Start Transaction and enter Account Search_Logoff before proceeding with the logoff action. Click on Logoff to logoff from the application. Click on End Transaction and select Account Search_Logoff to end the transaction. The script recording is completed. Click on Stop to stop the script recording. Coding for the above script will be created on Vugen automatically. Add lr_start_transaction ( Account Search_Login ) to complete the loop for login page display SAP AG 16

17 Parameterization: Click on the parameter value -> right click -> select replace with parameter to pass the values automatically. In this example: same scenario can be executed with different logins (8 different levels of users from organization model). The login details user Id and password can be parameterized to pass 8 level of user logon to the system automatically at runtime. Enter the parameter ID and click on Properties. Select the options mentioned in the following screen shot. Dat file will be created with the user ID login details SAP AG 17

18 Click on Password to pass the password details also as parameters along with user IDs. Select Filepath dropdown as User_ID.dat to create password along with the user ID details. Select the option Same line as user_id to pass the correct passwords with the correct user ids. Click on Edit with Notepad to add more user ID and passwords to the input parameters file for allowing multiple logons to the system. Unique: Read only once Sequential: parameters access sequentially at run time Random: parameters selected randomly at run time. To check the parameter selections for each run click on simulate parameter and pass the required values and simulate it. It will display the parameter selection for iteration of the test script. The script can be executed from vugen and check whether it is picking the correct parameters values or not before proceeding with the performance center. Before executing the script in Vugen, settings need to be updated in the Vugen SAP AG 18

19 Number of iteration: The script will be executed for 1 time. The script will execute one time with one parameter value entered in the parameter file. While executing the script, parameters values will be displayed on the screen to check whether the correct parameters are passing or not SAP AG 19

20 Think time: The idle time taken by the user while executing the scenario. This value has to be taken from the client. For Ex: If the think time is 10 sec - While recording the scenario, if the user spends 25 seconds idle time it will be corrected to 10 while executing the scenario. In this scenario the think time has been considered As Recorded whatever may be the think time while recording the script, same time only used while executing the test script SAP AG 20

21 Correlation: Correlation has to be done for this scenario. A unique system ID will be generated whenever user login to the system. This is to be handled dynamically using correlation function module to capture the SIDs for user login. For Ex: using the following correlation FM the SID has been handled dynamically in this scenario. The correlation FM should be entered before the server response code. It will pass the dynamic value to the server to get the correct response. /*This Correlation function is to handle the Session ID of the application*/ web_reg_save_param( "SID", "LB=(", "RB=)", "Ord=1","Search=Body",LAST ); Correlation Process: Check correlation options before correlation process SAP AG 21

22 Manual Correlation: Correlation for single user Create two test scripts for same scenario with single user. Go to Tools -> Compare with script. Compare one script with the other in Vugen. It will show the differences between the scripts with yellow lines. The different values between the scripts should be correlated using correlation function module. Correlation for multiple users - Create two test scripts for same scenario with two different users. Go to Tools -> Compare with script. Compare one script with the other in Vugen. It will show the differences between the scripts with yellow lines. The different values between the scripts should be correlated using correlation function module. In this scenario correlation should be done to handle multiple user logins to the system. Session ID value is different in both the scripts. Session ID value should be handled using correlation. This will show in uif_callback 2 block in the code SAP AG 22

23 Click on View Tree to check the server coding. Here it needs to handle the Session ID before the server response. In the above screen shot the SID difference is showing yellow in uif_callback2. This needs to be checked in before block uif_callback itself to handle the Session ID. Click on the Server Response tab and click on uif_callback on the right navigation bar. Find the Session ID in the coding and capture the left boundary and right boundary of the SID. These values need to be passed to the correlation function module to handle this SID dynamically SAP AG 23

24 Test Run of the script from Vugen for one user ID: Check the settings mentioned above before running the script from Vugen. Click on Run option on the screen as shown below. The script will be executed from Vugen and results will be displayed as shown below Script passed successfully. Click on the required steps on the right side navigation pane to check the screens display in each step. Uploading the script to Performance Center: Click on Tools and select Performance Center Connection to connect to PC and upload the script to Performance Center SAP AG 24

25 Enter the Performance Center user ID and password to connect to PC from Vugen. Select the script from the list and click on OK the script will be uploaded to Performance Center SAP AG 25

26 Script has been uploaded to performance center successfully. Test Script Execution: Login to Performance Center. Click on Vuser Scripts to check the script uploaded in PC. Click on Manage to create the baseline testing for account search scenario. Click on new Load Test to create a new baseline testing SAP AG 26

27 Enter the basic details on the General tab Click on Workload tab to assign a load generator to the script. Load generator has to be attached to the script to execute the scenario. Select the workload type 2011 SAP AG 27

28 Select LG Distribution Option: Adding LG Manually: 2011 SAP AG 28

29 Click on Select Virtual Load Generators link and select and LG from the list by checking the check box. Click on OK. Enter LG field value as 1 to assign the default load generator 2011 SAP AG 29

30 Select the script as Account Search_Me from the dropdown under script field. Assigned the number of Vusers 8 (i.e., the number of users assigned for Baseline Test scenario), Load Generator (Which generates Load), Running VuGen scenarios (i.e., Simultaneously or Gradually) and Duration of Run (15 minutes, For: Time Specific running scenarios, Indefinitely: No Time Bound) for Load Test. Then the scenario Duration of Vusers to run Select Duration for 15 minutes to run the baseline testing for all 8 users simultaneously. End vusers also simultaneously at a time once the duration of the time is over SAP AG 30

31 Set the total duration of the scenario for 30 minutes (15 minutes for script execution and 15 minutes for report generation). Note: If the load generator connection failed at the time of executing the scenario. Go to Project in the left side options and click on Hosts to check the load generator server is active or not. Check with the PC admin team if the load generator server is showing red status at hosts screen. Hosts Screen shot: 2011 SAP AG 31

32 Click on Proceed button. Note: To cancel the run of the scenario click on Cancel. Timeslot booked for this scenario have to be deleted from Project -> Timeslots. Select the timeslot booked on the screen and click on Delete. Then only the timeslot assigned for this scenario will be deleted and can be used for next run. Timeslot Screenshot: 2011 SAP AG 32

33 Script Execution from PC: After the Test Run, click on Test Runs Before creating Analysis Data to view the Results Baseline testing has been done the above scenario successfully. Load testing also can be done in the same way by executing the combination of scenario test scripts from the Performance Center according to the client requirement. Stress also can be executed in the same like load test. While execution, users need to be passed for the stress test scenarios till the system crashes. Click on + to add multiple script at the time of load testing SAP AG 33

34 2011 SAP AG 34

35 View Script on PC using the following option: Run time settings can be edited at PC using the following option: 2011 SAP AG 35

36 Test Analysis and Final Report: Download the Analysis software from the Performance Center to display the analysis report results. Run it and install analysis software to analyze the results of the scenario SAP AG 36

37 Click on Test run on the HPQC and click on result URL which will display the following Rawresult page Click on Raw Results and save the results zip file on the desktop. Open the LRA file to check the summary report. Select the required graphs and add it to the report to show the client according to their requirement SAP AG 37

38 The following summary will explain the minimum, maximum, average times of each and every transaction for the account search scenario. Standard deviation value will be used if there is any specific time given by the client to execute the transaction. SD value is used to find the deviation from the standard value specified by the client. 90 Percentile: This time is calculated after the 90% execution of the script Result Analysis Screen of Base Line Test: In Result screen it will display the result for all 8 levels of users. The Minimum, Average and Maximum timelines taken by the server will display in the summary report for base line testing. In the following graph the Average Response time for the 8 levels of users to execute the scenario ( Account Search Party involve Me ) has been shown clearly to present it to the client. Graph: (Average Response Time Graph for Account Default Search) 2011 SAP AG 38

39 Issues Faced while Executing PT: The ICM (Internet Communication Manager) parameters should be noted initially before the test is conducted. Supposing the threads at ICM is maintained as 50 and request sent by the user is more than 50, server will start giving error as Connection got shutdown prematurely. Hence maintain the no of requests sent less than the thread parameter maintain at ICM (T-code: RZ10 and SMICM) 1. Maintain CPIC_MAX_CONV parameter in ITS Internet Transaction Server) at OS level, as after CRM 5.0 version onwards the ITS is a part of SAP CRM package. 2. For executing BI report, maintained co-relation parameters for Drill down scripts. 3. Co-relation parameters should be maintain for F4 help and New button in CRM web UI 4. Need to work with the Basis team closely while executing the scripts at on peak and off peak levels. Network settings should be changed according the number of users used in baseline\load testing SAP AG 39

40 5. Most of the issues we faced while running the test script. We need to check the problem areas by keeping debugging points in the test script. We faced the issues with Correlation function modules syntax and coding errors. 6. According to the client requirement 1 2% errors in PT can be acceptable. 7. If the test run is giving more than the above mentioned errors, then runtime settings should be adjusted like Network Simulation timings etc., We changed it from 129 to 999 to stop the erros. 8. According to the scenario test scripts can be executed by keeping all the coding in Action part of by dividing the test scripts. 9. As we created one test scripts with different users we faced most of the issues at the time or correlating the scripts only. 10. Need to work closely with the performance testing developers while creating the test scripts for the scenarios SAP AG 40

41 Related Content For more information, visit the Customer Relationship Management homepage SAP AG 41

42 Disclaimer and Liability Notice This document may discuss sample coding or other information that does not include SAP official interfaces and therefore is not supported by SAP. Changes made based on this information are not supported and can be overwritten during an upgrade. SAP will not be held liable for any damages caused by using or misusing the information, code or methods suggested in this document, and anyone using these methods does so at his/her own risk. SAP offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this technical article or code sample, including any liability resulting from incompatibility between the content within this document and the materials and services offered by SAP. You agree that you will not hold, or seek to hold, SAP responsible or liable with respect to the content of this document SAP AG 42