Performance Testing from User Perspective through Front End Software Testing Conference, 2013 Authors: Himanshu Dhingra - Overall 9+ years IT extensive experience years on Automation testing - Expertise on Automation Testing tools QTP, SilkTest, FitNesse, Selenium - Name & Address of Organization: Deloitte Consulting, Salarpuria Supreme, Near Innovative Multiplex, Marathalli Tushar Roy - Overall 7+ years IT extensive experience years on Automation testing - Expertise on Automation Testing tools QTP, TestPartner, Selenium - Name & Address of Organization: Deloitte Consulting, Salarpuria Supreme, Near Innovative Multiplex, Marathalli
Abstract Performance testing so far has been known as capturing response times and metrics at an application s protocol level. The metrics we generally captured here are of the data at the protocol level. But in real world, when the application would be used by real users, the transactions will happen via the user-interface (UI) and it will also include those extra time needed by the UI or the browsers for rendering the data on to the UI. Thus to capture the actual and quantitative performance test metrics, it becomes crucial to capture these extra times as well. Our paper gives an alternation option on how to capture quantitative performance test metrics. Here, we are suggesting the Performance Testing to be done through and inclusive of the UI. Tracking the UI s turnaround time using a functional test tools and the multi-user simulation through performance test tool side. Another benefit of this approach is, you would not need to again write scripts for performance testing tool. The script created for functional testing can be integrated and used in performance testing tool. For this, we used and integrated two open source tools, Selenium and JMeter. Since, both the tools are open-source tools, we could also save valuable money in procuring the licenses. The net time and metrics captured by both the tools can be used in establishing an actual and quantitative performance test metrics. Performance Testing from User Perspective can help in capturing the actual and quantitative performance test metrics. Instead of just capturing the protocol level times and metrics, it would be beneficial to also capture the extra times needed by the UI and browser. This is necessary because in real world, the actual users would not be acting at the protocol level but through the user interface. Integrating the functional test tool with performance test tool would save from recreating the scripts. With the use of Selenium and JMeter for this integration, both are open-source tools, valuable money can be saved. Performance Testing from User Perspective through Front-End Why Needed? And How to Implement? Functional automation tools, like Selenium and QTP are not created to do performance testing, and check how much time an application takes for a single user or multi-user operation to complete. The functional testing tools are created to test the functionality of the application from a single user perspective. These tools independently can find out how much time a particular user operation takes to complete. For example, if a user logs in to a web application, it is possible to find out how much time login functionality took to complete for this user. But the functional automation tools cannot be used to find out the application turnaround time for multiple users at one time. For example, time take for the login functionality to complete if 1000 users are accessing the application at same time.
The performance testing tools, such as JMETER and Load Runner, on the other hand, can be used to measure the performance of an application in single user or multi-user environment. These tools record the user actions directly through protocol or proxy. But these tools cannot be used to simulate the user interaction with a web application through a browser. Also, JMETER is not able to handle javascript and cannot be used to test application containing javascript or AJAX. This solution discusses how to integrate Selenium and Jmeter to test performance of an application from a user perspective. Selenium is a functional testing tool for web applications and can be used to create scripts simulating user-interaction with a web application. Selenium can then be integrated with Jmeter and the script created using Selenium can be used to do multi-user performance testing. This solution leverages the ability of Jmeter to create multiple thread groups and create reports with the parameters, such as throughput time to find the performance of the application. The benefit of integrating Selenium and Jmeter is that the same functional scripts created in Selenium for functional testing can be used for performance testing. Also, Jmeter and Selenium are open source tools, and this makes the integration easy and saves the money. Following are the steps to create a test in Selenium and execute it using Jmeter. For this example, WebDriver, eclipse and Junit is used to create Selenium test case. 1. Create a project in eclipse, add a package, say JmeterTest, and add a class in this package. As an example, this test logs into ebay.com. package jmetertest; import java.util.regex.pattern; import java.util.concurrent.timeunit; import org.junit.*; import static org.junit.assert.*; import static org.hamcrest.corematchers.*; import org.openqa.selenium.*; import org.openqa.selenium.firefox.firefoxdriver;
import org.openqa.selenium.interactions.actions; import org.openqa.selenium.support.ui.select; public class EbayTest { private WebDriver driver; private String baseurl; private boolean acceptnextalert = true; private StringBuffer verificationerrors = new StringBuffer(); @Before public void setup() throws Exception { driver = new FirefoxDriver(); baseurl = "http://www.ebay.in/"; driver.manage().timeouts().implicitlywait(30, TimeUnit.SECONDS); @Test public void testebay() throws Exception { driver.get(baseurl + "/"); driver.findelement(by.linktext("my ebay")).click(); driver.findelement(by.id("userid")).clear(); driver.findelement(by.id("userid")).sendkeys("seleniumtesting"); driver.findelement(by.id("pass")).clear(); driver.findelement(by.id("pass")).sendkeys("welcome1"); driver.findelement(by.id("signed_in")).click(); driver.findelement(by.id("sgnbt")).click(); driver.findelement(by.id("gh-uga")).click(); Thread.sleep(4000); driver.findelement(by.id("gh-uga")).click(); assertequals("selenium selenium", driver.findelement(by.id("gh-un")).gettext()); catch (Error e) { verificationerrors.append(e.tostring()); driver.findelement(by.linktext("sign out")).click(); @After public void teardown() throws Exception { driver.quit(); String verificationerrorstring = verificationerrors.tostring(); if (!"".equals(verificationerrorstring)) { fail(verificationerrorstring); private boolean iselementpresent(by by) {
driver.findelement(by); return true; catch (NoSuchElementException e) { return false; private boolean isalertpresent() { driver.switchto().alert(); return true; catch (NoAlertPresentException e) { return false; private String closealertandgetitstext() { Alert alert = driver.switchto().alert(); String alerttext = alert.gettext(); if (acceptnextalert) { alert.accept(); else { alert.dismiss(); return alerttext; finally { acceptnextalert = true; 2. Export the project as a jar file. To export the eclipse project as a jar file, go to File > Export.
3. On the JAR Export, save the exported jar to the lib\junit folder in the jmeter directory. For example, C:\jakarta-jmeter-2.5.1\lib\junit\JMeterSelenium.jar 4. Copy the Selenium webdriver library files in the lib folder in jmeter directory. If you are using Selenium-Server-Standalone jar file (for example, selenium-server-standalone-2.33.0.jar), you can copy this file to lib folder. 5. Start jmeter and create a test plan. In the test plan add a thread group. In the thread group, add the Logic Controller, Once Only Controller. 6. In the Once Only Controller, add a JUnit Request Sampler. After these configurations, the Jmeter test plan should like:
7. Select JUnit Request, and you should be able to see the class name in Classname drop down list. Select the test method from the Test Method drop down list if it is not already selected. 8. Run the test case and you should be able to see the result. You can configure the ThreadGroup if you want to run the test for multiple users. You can also iterate this test multiple times by creating and configuring the Loop Controller. If you don t want to see the multiple windows opening, you can use the htmlunit driver. This will execute the tests in background and you will not see multiple browser windows opening. Conclusion Performance Testing from User Perspective can help in capturing the actual and quantitative performance test metrics. Instead of just capturing the protocol level times and metrics, it would be beneficial to also capture the extra times needed by the UI and browser. This is necessary because in real world, the actual users would not be acting at the protocol level but through the user interface. Integrating the functional test tool with performance test tool would save from recreating the scripts. With the use of Selenium and JMeter for this integration, both are open-source tools, valuable money can be saved.