Load Testing Ajax Apps using head-less browser tools. NoVaTAIG April 13, 2011 Gopal Addada and Frank Hurley Cigital Inc.

Transcription

1 Load Testing Ajax Apps using head-less browser tools NoVaTAIG April 13, 2011 Gopal Addada and Frank Hurley Cigital Inc. 1

2 Agenda About Cigital Background : AJAX and Load Test requirements Tools research HTMLUnit Virtual User (VU) Generation Results and reporting Pros and Cons Limitations and Future work

3 Software Assurance services Software Quality Agile testing Test automation Continuous integration Test process improvement Software Security Secure design Secure coding Security testing Continuous integration Dec10 3

4 Background: AJAX Web 2.0! Interactive Several AJAX frameworks are being used GWT, jquery, MochiKit, Adobe Flex, Dojo, YUI COTS tools (claim to) support Ajax load tests: LoadTester, Neotys, LoadRunner(HP) Very expensive (per license per Virtual User) Open source solutions Jmeter, Grinder and opensta

5 Background: AJAX Taken from Aztecsoft Whitepaper [1]

6 Please refer to Descriptive Picture at URL [2]: s/essays/ajax-fig2_small.png

7 Classic Web applications Background: AJAX AJAXy web applications Requests sent as direct response to user actions Requests are normally <form> that use GET/POST methods Browser - server involves requests for complete pages The response from server involves entire new page which is rendered at client browser Requests can happen due to user actions or asynchronously at intervals determined by AJAX engine Can be of any form supported by remote application server (e.g. XMLHttpRequest) Typical requests involve updating parts of already loaded page The response is intended for small portion of page which is interpreted by AJAX engine on client side and updates DOM of current page Partially from Aztecsoft Whitepaper [1]

8 Load Test Tool Requirements for Web Need to simulate User behavior using Virtual Browsers Real browsers don t work: Do not scale well as running multiple browser tests from test machines is not feasible Intent is to test the Server performance JMeter and Grinder are prominent open-source tools Use HTTPClient implementations and are multi-threaded GUI to create load tests Do not support Java Script and hence can t simulate AJAX engine behavior

9 Load Test Tool Requirements for Web Classic Web applications Need any virtual (headless) browser that simulate user actions using HTTP client Involves testing mostly server, client behavior may not have to be replicated (e.g. cross-browser does not matter). Simulating concurrent HTTP GETs/POSTs is enough (e.g. Jmeter, Grinder) Response has to be asserted and response times may need to be captured. Server has to be monitored for CPU performance, memory usage and DB performance AJAXy Web applications Need any virtual (headless) browser that simulate user actions using HTTP client with JavaScript Support Browser runs AJAX Engine (JS calls) -> need to simulate client behavior to maximum extent possible Response has to be asserted and response times may need to be captured. Server has to be monitored for CPU performance, memory usage and DB performance

10 Tool research (open-source) Jmeter One of the best load test tools, but does not support JS calls HttpUnit Good Headless browser tool and No JS support Grinder Good tool, but no known JS support HtmlUnit 100% Java-based headless browser Supports HTTP Requests, HTML Parsing, JS execution, CSS Easy to script tests (similar to Selenium) WebTest Built on HtmlUnit and tests written in Groovy or XML Can potentially be used for load test design Easier and non-programmer friendly

11 public void testgoogle() throws Exception { WebClient webclient = new WebClient(BrowserVersion.FIREFOX_3); HtmlPage gpage = webclient.getpage( ); assertequals( Google", gpage.gettitletext()); HtmlUnit qtxtbox = (HtmlInput)gPage.getElementByName( q ).get(0); qtxtbox.setvalueattribute( NoVaTAIG ); HtmlElement btn = gpage.getfirstbyxpath( //input[@value=\ I'm Feeling Lucky\ ] ); HtmlPage gpage2 = btn.click(); } } asserequals( Northern Virginia Test Automation Interest Group,gPage2.getTitleText());

12 HtmlUnit (contd.) SHOW ARCHITECTURE DIAGRAM from MARC s PRESENTATION

13 HtmlUnit (contd.) CurvyCorners Dojo ExtJS GWT JQuery MochiKit Prototype Sarissa Yahoo UI... AJAX Libraries Supported

14 Approach for our load tests Complete Black-box approach simulating a browser (Vuser) and perform user actions Write functional tests for pages to be load tested Generate desired number of threads Each thread calls functional test(s) Each thread can use different user To simulate a close-to-reality scenario Each thread may have to simulate behavior of different browser (IE, Firefox, Chrome)

15 Approach ThreadPoolMain T Test Status Data reported back to main Virtual users (VU) Thread 1 T Thread 2 T Thread 3 T Thread n T HtmlUnit CodeT HtmlUnit CodeT HtmlUnit CodeT HtmlUnit CodeT

16 HtmlUnit code demo //Every thread calls this function which has sequence of tests public void rungtest() throws Exception{ HtmlPage currentpage; webclient.setuseinsecuressl(true); final HtmlPage loginhomepage = login(url,username,passwd); WebAssert.assertTextPresent(loginHomePage, "Change security question"); currentpage = loginhomepage; gotogmail(loginhomepage); /** INVOLVES AJAX CALLS**/ logout(currentpage); }

17 HtmlUnit code demo (Contd..) //Sample test involving Ajax calls private HtmlPage gotogmail(htmlpage dashboardpage) throws Exception{ webclient.setajaxcontroller(new NicelyResynchronizingAjaxController()); final HtmlAnchor gmaillink = dashboardpage.getfirstanchorbytext("gmail"); final Page gmailpage = gmaillink.click(); // AJAX Engine call (JS) WebAssert.assertElementPresent((HtmlPage)gmailPage,"loading"); WebAssert.assertLinkPresentWithText((HtmlPage)gmailPage,"Load basic ); WebAssert.assertTitleContains((HtmlPage)gmailPage, "Gmail"); return dashboardpage; }

18 Thread class public class GmailThread implements Runnable { private int userindex = -1; public GmailThread(int userindex) { this.userindex = userindex; } public void run() { String origname = Thread.currentThread().getName(); Thread.currentThread().setName("user_" + userindex); GAccountAccess gtest= new GAccountAccess(); gtest.rungtest(); } }

19 Test Thread Details class (New Object Passed from main to all the way to HtmlTest) public class TestStatusData { private long threadstarttime; private long threadendtime; private String tusername; private String tpasswd; // Don t save password just place holder Private threadpage1responsetime; Private threadpage2responsetime; Private threadpage3responsetime; private long threadlogintime; private long threadlogouttime; String threadcurrentstatus; String tcomment; //sample setter and getter public void setsftplogintime(long sftplogintime){ threadsftplogintime = sftplogintime; } } public long getsftplogintime(){ return threadsftplogintime; }

20 Thread Pool (Main class) *** Can be used as black box for most load tests. Details not really important. Partial code (Will be posted in blog) private ExecutorService userthreadpool = null; private HashMap<Future<GmailThread>, GmailThread> taskmap = new HashMap<Future<GmailThread>, GmailThread>(); userthreadpool = Executors.newFixedThreadPool(concurrentThreadCount, new NamedThreadFactory("UserPool")); for (int i = 0; i < totalruns; i++) { testdata[i] = new TestStatusData(); // Detailed in previous slide int userindex = i % usernames.length; //randomize username/paswords to be used testdata[i].settusername(usernames[userindex]); TestFlexorAuthThread handler = new TestFlexorAuthThread(testData[i],usernames[userIndex],passwds[userIndex]); Future<TestFlexorAuthThread> servertask = userthreadpool.submit(handler, handler); taskmap.put(servertask, handler); } awaittermination(); userthreadpool.shutdown();

21 Good Serves the purpose without COTS tools Complete control and Minimal Java needed Can be cronned for run overnight builds Underlying HtmlUnit continuously updated and supports new frameworks Can simulate multiple Browser in single Load test HtmlUnit Simulates IE7, IE8, Firefox 2, Firefox3 Can configure each thread for different browser Generic load generation Load generation code can be re-used for other web apps Used for client/server applications (SFTP, FTP, SMTP)

22 Bad Does not have reporting UI Working on incorporating into JMeter Investigating incorporating into Grinder HtmlUnit does not support all AJAX libraries (RIAs) Adobe Flash, Silverlight, WPF Scalability: Limited by JVM and CPU availability Adding additional hardware helps, or making tests distributed Hard to maintain as UI/ object properties change It s an issue with every load/functional test tool Not very friendly for non-programmer Very basic programming needed Use Canoo WebTest instead of HtmlUnit We are currently working on this

23 Lessons Learned JVM tuning sponse hause as powerful machine as possible (multi core) JVM assigned to max possible (32-bit, 64-bit) All the HTMLUnit tests have to be thread-safe To be incorporated into load generation framework Fine-grained Logging needed for trouble shooting E.g. To know what response server is sending back? Every possible exception better be handled in HtmlUnit code and Thread code for failure analysis

24 Other ways to perform load tests Using TestNG ThreadPool framework for HtmlUnit tests WebTest + Groovy multi-threading Jmeter Groovy Sampler+ WebTest (needs extra effort)

25 Future Work Use Groovy with WebTest Canoo to make tests nonprogrammer friendly Work on potential integration with Jmeter Isolate Load generation and load execution (LoadRunner/Grinder approach of Controller + VUgen) For large-scale tests Additional Virtual Machine support Cloud based support Develop distributed version of tool for CLOUD IN Progress! Later : Develop WebUI for configuration purposes

26 Questions, Comments and Suggestions?

27 Gopal s Load testing Blog: Please post comments and suggestions Updates on load test framework(s) will be posted 27

28 References [1] ormance_testing_ajax-based_applications.pdf [2] 28