Comparative Analysis of Various Automated Test Tools for Flex Application

Transcription

1 Comparative Analysis of Various Automated Test Tools for Flex Application Nisha Kaushal, Rupinder Kaur National Institute of Technical Teachers Training & Research, Punjab University, Chandigarh, Panjab Abstract This paper is aimed to carry a comparison study of automated testing frameworks for Rich Internet Applications (RIA) made using Flex framework.. Test cases will be written in different languages like Ruby, ActionScript etc. using identified frameworks on a reference application. Frameworks ( and ) are evaluated on the basis of Execution time, CPU Utilization, Memory used, Number of lines of code, ease of automation, extensibility options, maintainability of automated test scripts, options to run test suites, options to generates test reports etc. Results of this study will be helpful to software development professionals to select better test automation framework for their application. Keywords:Flex Application, Automated testing,, 1. Introduction Software testing is an integral part of Quality Assurance in Software development. Software testing is aimed at determine how well a software system meets the required results. Test Automation is an emerging field to meet these challenges and increase efficiency, report and catch bugs and software defects as early as possible in SDLC. Test automation overcomes few of the manual testing limitations of being slow and costly for the repetitive functional and regression testing. Manual testing is also not consistent or deterministically repeatable. Automating test cases leaves manual testers to focus more on writing test cases and analysis of defects and quality issues in the project. There was lack of automated test tools so Researches and professionals in test automation industry have produced many tools to automate testing of web [1]. These tools vary in terms of their capabilities and commercially fall in two either paid or open source tools. With so many tools in the market, in the past comparison studies have been done to evaluate features and usability of different tools. Watir and Selenium has come up top in the list of Open source test automation frameworks available and Rational robot, Quality Test Professional are most preferred in the list of commercial test tools/framework. Flex framework developed by Adobe in around 2004 is an answer to problems like browser compatibility, an adaptive framework to develop both web and desktop applications. Flex provides additional capabilities like adding charts and graphs to applications. Using Flex it is claimed to be easy to put a fresh new face to legacy applications. Flex framework has picked up in the industry and thus there has been a demand to have test tools to automate testing of flex based applications. As the Flex based applications are becoming popular, recently enhancements are being made in exiting tools to support automated testing of flex applications. Few new test tools and frameworks are being written to provide automation of flex-based applications. 2. Flex Applications Adobe Flex is a collection of technologies for the development and set up of cross platform Rich Internet Application. Flex enables to rapidly build applications that are installed to Flash player which means that Flash Player runs all Flex Applications. Features of flex are- Flex is capable of providing integrated solutions that utilizes data services, interactive UI design, media elements such as audio and video and even real time communications. It utilizes metaphors from desktop computing such as drag and drop and double click. It has small download time. It is a cross-platform and cross-browser compatible. It runs on Window, OS X and Linux and all the major browsers including Firefox, Internet Explorer, Safari and Opera. 3. Flex Applications And Rich Internet Applications (RIA) Flex is a framework developed by Adobe Systems Incorporated for developing RIA for the Flash platform. It was a result of the increased demand for more desktoplike applications for the Web.The initial release in March 2004 by Macromedia included a software development kit, an IDE, and a J2EE integration application known as Flex Data Services [2].Rich internet application (RIA) was introduced by Macromedia (in Macromedia flash MX-A next generation rich client) in RIA are web applications that have the features and functions of traditional desktop applications [3]. 4. Automated Test Tools Because of the more advantages of the automation testing over manual testing, various companies are

2 engaged in developing various automated test tools for various applications. There are two types of test tools. Open source test tools Commercial test tools Open Source Test tools- These test tools are free for the users to use. It can be downloaded from the internet or can be obtained by the vendor without any charges e.g. Watir, Selenium, FunFx.Commercial Test Tools- These test tools are not free and user has to pay for it e.g. TestComplete, Quality Test Professional, Rational robot. [4] is developed by Peter Motzfeldt as a result of his master thesis at the Norwegian University of Science and Technology in June, is created as a framework that will enable to drive a Flex application through a web browser and it was the only solution of testing flex application. is a framework for functional testing of Adobe Flex applications. The framework encourages test-driven development, and provides a Ruby API for interacting with Flex application or it is a Ruby tool for functional testing of Adobe Flex applications. It is the only free alternative to achieve functional testing of Adobe Flex applications. is open-source and uses Ruby to write tests. framework: The framework supports any Flex application compiled with the library file. The framework must be able to drive a Flex application programmatically. The framework must be able to support test-driven development. The framework must support writing tests in Ruby. The framework must be able to run multiple test suites, and provide useful results from the tests. is the only Flex record/playback tool that is entirely Flex-based and was created by adapting Adobe's sample application, AutoQuick.. It requires no browser plug-ins (other than the Flash player of course) or special scripting languages. If you know Flex, you already know 99% of what you need to know to use, and you can run anywhere you can run Flex (ie, virtually any platform) [5]. is an open source Flex application and library that records and plays back user interface interactions and generates replayable test cases that can be run from a continuous integration framework such as Cruise Control. It is a unit testing framework for Flex apps that provides for automating the testing of Flex UI functionality. can record and play back Flex UI interactions, and generates editable ActionScript-based testing scripts that can easily be included within a continuous integration process. It uses the Flex Automation API and was created by extending Adobe's sample automation adapter, AutoQuick. is a new tool to perform automated testing on Flex applications. With it, a set of interactions can be recorded, save those commands into a TestCase class, add the TestCase to the TestSuite, and include it as part of a FlexUnit testing workflow. Features: Records and plays back Flex UI interactions. UI Interactions can be edited and replayed. Generates FlexUnit TestCases, and can also be used with non-flexunit-based testing frameworks. Tests can be run from build systems such as Ant. Handles all Flex UI events. Uses Flex Automation API to provide native control over your flex app requires no javascript or browser plug-ins to use. Unit tests are written entirely in ActionScript. No other programming or special purpose scripting languages are needed to develop comprehensive UI test suites. Non-invasive. Requires no modifications to your application source. 5. Methodology Figure5 shows Reference flex application will be taken from internet for automated testing. Various automated test tools will be searched and test cases will be written using identified frameworks on a reference application. Test cases will be written in JavaScript, Ruby, ActionScript etc, depends upon the automated tools through which flex application being tested. Frameworks are evaluated on the basis of Execution time, CPU utilization, Memory used, Number of lines of code, Ease of automation, Extensibility options, Maintainability of automated test scripts, options to run test suites, options to generate test reports etc. Execution Time This parameter measures the time taken by the tools for testing the applications in seconds. CPU Utilization This parameter identifies the percentage of CPU usage during testing of applications by test tools. Memory usage This parameter identifies the percentage of RAM or memory usage during testing of an application. Number of lines of code This parameter identifies the number of line of code required to write the test cases for each tool. This will be calculated through a Line Counter tool.

3 Flex application Download Automated test tools along with their compatible files Write test cases in ruby, ActionScript for Automated test tools Link test cases of an application to Automated test tools Execution of test cases Figure 6.1 Home page with Figure 5. Method of Automated Testing 6. Implementation Automated test tool takes seconds during testing of library application and takes seconds. Various test cases like Test_verify_addition_contents, Test_add_success,Test_add_failure,Test_delete_success, Test_delete_failure are written in and tool to do the automated functional testing of a library application. As these test cases provides constraints like the Test_add_success testcase provide constraint that the book details can only be added if the addbook button be clicked. All other testcases provide different constraints to do validation. will open Internet Explorer browser having url of home page. Home page consists of two buttons(add book and delete book) and one datagrid. Whereas opens application in Adobe Flash player for automated functional testing. In figure 6.1 through Test_Verify_addition_contents testcase do functional testing of library application. As it checks that after filling various textboxes both the buttons work properly or not.during execution of Test_verify_addition_contents test case,the will test the application automatically by opening the application in internet explorer and then automatically fills the textboxes(book title, author etc ) and then click on adddbook button,after clicking the button book details will be added into a datagrid. Hence application works properly.in this way funfx can do automated functional testing and it takes seconds during testing. Test_Verify_addition_contents Figure 6.2 Functional Testing with Test_verify_addition_contents testcase in Figure 6.3 Execution Time of Test_verify_addition_contents testcase in Test_add_success

4 To use Framework your application needs to be compiled using Adapter.swc, automation.swc and automation_agent.swc library files. Your application need to include AutomationGenericEnv.xml file to use. Test cases are written in ruby language.testers have to be proficient in ruby to write test cases. Figure 6.4 Execution Time of Test_add_success test case in est_delete_success Figure 6.5 Execution Time of Test_delete_success test case in Ease of Usage Flex Monkey Installation is easy. Tool is distributed as a swf file which comes with a record and play interface. Tool does not require any special changes to your application. You need to copy swf file in same location where you application swf file is located and then launch the application through tool. Framework uses action script based language to write test cases. Testers have to write assertions using this language in the test cases. 7.2 Extensibility Support for new type of components can be added through making entries in AutomationGenericEnv. Xml file. There are no instructions or user manual on how to add such a support. In absence of this it is not very easy to extended capabilities of tool. Extensibility Framework supports advance flex components. Framework comes with a Flex spy tool which is very helpful in identify Flex components used in application. Figure 6.6 Functional testing of library application with 7. Qualitative Comparison between FunfX and 7.1 Ease of Usage Installation of framework is tricky.framework is distributed as a ruby gem file. framework has to be compiled with specific same version of flex sdk as your application before you can use framework. 7.3 Maintainability of automated test cases Test cases written in ruby. Different versions of test cases can be maintained in source control. Existing test scripts has to be changed every time during structure of application changes. Maintainability of automated test cases A swf file of your test cases can be made which can be maintained in source control. Existing test scripts has to be changed every time during structure of application changes. 7.4 Options to run test suites

5 Both the frameworks provide ability to run individual test cases in a sequence to run as test suites. Test_add_failure Qualitative Comparison between FunfX and 8.1 Execution time Table 8.1 Test_verify_additon_contents execution time Table 8.3 Test_add_failure_Execution _Time Test_add_failure Execution Time Table 8.4 Test_delete_successs Execution time Test_delete_success Test_Verify_addition_ contents Test_Verify_addition_contents Test_delete_success Test_delete_success Table 8.2 Test_add_success Execution Time Test_add_success Execution Time Test_add_success Table Test_add_failure execution time 2 Table Test_add_failure execution time Test_Verify_addition_cont ents Execution Time Test_add_success Table 8.5 Test_delete_failure Execution time Test_delete_failure Execution Time Table 8.6 Total Execution Time Total Execution Time CPU Utilization Table 8.7 CPU Utilization CPU Utilization (%) 50 20

6 Memory space CPU Utilization (%) CPU Utilization (%) Table 8.8 Memory space Memory Space (K) 25,192 34,880 40,000 35,000 30,000 25,000 20,000 15,000 10,000 5, Lines of code Memory Space (K) Memory Space (K) Table 8.9 Lines of code Lines of Code Lines of Code Lines of Code 9. Conclusion Automate test cases is one of the most important aspect of Flex Application and Web Application testing. There is limited research in effectively testing of Flex Application using automated test tools. This paper focuses on Flex Application testing through Automated Test Tools. In this and open source automation tools are compared on the basis of various Quantitative and Qualitative parameters. In this paper library Flex application is being implemented in FlexBuilder tool and its validation is performed with and automated test tool. and are compared on the basis of various Quantitative (Execution time, CPU Utilization, Memory used, lines of code) and Qualitative parameters(ease of Usage, Extensibility, Maintainability of automated test cases, Options to run test suites).this paper concluded that automated test tool is a better functional testing tool as compared to as it takes less Execution time, require few lines of code for writing test cases and require less CPU Utilization during functional testing of Library Flex Application. Although requires more memory space during execution but still it is better than. This paper also concluded that from Qualitative parameters point of view is better than because it is easy to install, does not require any modifications to flex application, components of flex application can easily be identified etc. Results of this study will be helpful to software development professionals to select better test automation framework for their application. In future a new tool can be implemented for testing Flex application and then it can be compared with the existing automated test tools so that the companies can select the best tool for their applications. References [1] Jason I. Hong, Jeffrey Heer, Sarah Waterson, and James A. Landay WebQuilt: A Proxy-based Approach to Remote Web Usability Testing ACM Transactions on Information Systems, vol. 19, no. 3, [2] AdobeFlex Retreived January 22nd 2009 [3] RichInternetApplication, /Rich_internet_application, Retrieved March 9, [4] Peter Nicolai Motzfeldt, A Framework for Functional Testing of Flex Applications, Norwegian University of Science and Technology Department of Computer and Information Science, [5] Jon Rose, Flex Testing with, InfoQ Tracking change and innovation in the enterprise software development community, [6] AutomatedQA, TestComplete 5 FAQ,