MarathonITE. GUI Testing for Java/Swing Applications

Transcription

1 MarathonITE GUI Testing for Java/Swing Applications

2 Overview Test automation is not a sprint... it is a marathon Test Automation As the applications in today s environment grow more complex, the testing of user interface becomes more and more important. Though the task of automating tests for user interface is challenging, it is also rewarding. One of the mistakes committed by early adopters of test automation is in assuming that it is a one-time task. However, more experienced teams realize that test automation and specifically GUI test automation is an ongoing task. A single change in the UI of the application might necessitate changes to a lot of test scripts. Regular refactoring and maintenance of test scripts might alleviate this pain. Test teams need the right tools for maintaining and continuously updating the test scripts. Marathon and MarathonITE Marathon is the premier open source test automation tool for Java/Swing applications. Jalian Systems adopted Marathon in 2006 and was providing enhancements and support for the open source project. Based on the same open platform of Marathon, Jalian Systems provides MarathonITE - Marathon Integrated Testing Environment, a commercial, enhanced version of Marathon. MarathonITE provides tight integration with the Java platform. The test engineers can record the scripts in either Ruby or Python - alleviating the need for learning a tool specific language. By providing access to the GUI elements from the Java runtime, MarathonITE provides unprecedented flexibility in controlling the application under test. Besides supporting fully automated tests, MarathonITE supports exploratory tests and semi-automatic tests. Exploratory tests are useful when a test engineer needs to record his findings while exercising the application. Semi automatic tests are used for checking an application for confirming to a set of guidelines. 2

3 Features and Benefits Intelligent Recording and Replay Records test scripts while the application is being executed. Records semantic actions (for e.g: entering text into a JTextBox) rather than individual key strokes and mouse clicks. Recognizes and records actions on all standard Java/Swing controls. Monitor resolution independent script recording. Modules for reusing script-lets and organizing test project. Built for Flexibility Vendor neutral scripting language. Supports Ruby/Python. Simple script elements for accessing components from the application runtime. Inspect and act on almost any aspect of application. Use any constructs supported by the scripting platform for automating any and all actions of the application. Use any third party modules provided by the script runtime. Simple Design Built by a team that has expertise in Agile processes - Marathon embodies the value of simplicity. Adds only a handful of well defined script elements to the script runtime. Provides a test runner view for running all or selected tests and generate report. Cross Platform Tested and certified for Mac, Linux (Ubuntu) and Windows. Supports any platform for which Java 1.5+ is available. Ability to choose the command code ( for Mac and control for others). 3

4 Integrated Testing Environment Dockable views for configuring the workspace in whichever way one prefers. MarathonITE even saves the workspace environment for each project and restores the open files and environment exactly the way it was earlier. Syntax aware editor provides eye pleasing formatting to the script code. Integrated debugger for stepping through the test scripts. A script console, that can be opened while recording and playing, for inspecting the application state. Color coded output pane - shows different styles for standard output and standard error from both scripts and the applications. A test runner view for running tests as a batch. A report is generated for each run and is available from the toolbar. Integrating into External Systems It is trivial to integrate MarathonITE into any build system. A command line tool for running test scripts is provided to integrate into *nix shell scripts. Example ant build.xml is also provided. Reports can be generated in text, XML and HTML formats. The XML report can be converted into a standard JUnitReport format and integrated into other continuous integration systems like hudson, cruise control etc. Our team is actively studying test management systems like XStudio and testlink. MarathonITE, at a future date, shall provide integration into test management systems. 4

5 Smart Test Scripts MarathonITE test scripts can be modularized into script-lets. The modularization can be based on any of the application specific criteria, like screen based or functionality based. When recording a test script, the test engineer can just select a pre-recorded module, and it will be inserted into the test script. Verification points (or assertions) can be inserted into a test script while recording a test case. No need for updating the scripts offline. Assertions can be inserted for all properties of the controls. On test failures, (optionally) screenshots can be recorded. MarathonITE takes the idea of Fixtures from JUnit family and integrates into the framework. When a project is created a default fixture is created. Custom fixtures can be created to setup the application for different group of tests. Java Applications MarathonITE supports testing different types of java applications - Applets, runnable Jars, Java!om command line or JNLP (Webstart). Test Types MarathonITE supports exploratory, semi-automatic and automatic test script generation. Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design and test execution. MarathonITE provides an exploratory test option to capture the test execution results while exploring the application. Semi automatic tests allows the test engineer to insert a checklist at a specific point in the application flow. When the semi automatic test scripts are executed, MarathonITE pause the application and displays the checklist which should be filled up by the engineer. These tests are especially useful for ensuring that all the UI elements follows organizational guidelines. 5

6 Refactoring and Data Driven Tests Test scripts can accept data from CSV files. MarathonITE provides facilities for organizing test data and use it in test scripts through provided script methods. Supports creation of Data Driven tests by refactoring a test script. The test engineer chooses the fields that need to be extracted into the data file and MarathonITE does the rest. A test data file is created and the script is modified to use the data from the test data file. MarathonITE also provides facilities for extracting a part of a test script into a module script-let. 6

7 F.A.Q What is MarathonITE and how does it differ from Marathon? Marathon is the premier open source Java/Swing test automation tool. You can consider MarathonITE as a commercial edition of Marathon with enhancements for enterprise test script authoring and maintenance. Specifically MarathonITE includes the following functionalities that are not available for Marathon. Data driven test cases Ability for converting recorded scripts to data driven tests. Refactoring support: currently MarathonITE support create module!om selection refactoring. Over a period of time Jalian Systems will be adding more refactorings to MarathonITE. A configurable naming strategy. Are MarathonITE scripts compatible with Marathon Scripts? Yes and No. If enhanced functionalities like data driven tests or the naming strategy are used, MarathonITE scripts will not be compatible with Marathon scripts. Otherwise, the scripts are interchangeable. However, Marathon projects are compatible with MarathonITE and you can use MarathonITE test runner for executing Marathon test scripts. What GUI technologies are supported by MarathonITE? MarathonITE, at this time, supports only applications developed using Java/Swing framework. We are working on adding web testing to the technologies supported and that is still some time away. Can MarathonITE be used for load and performance testing? MarathonITE can be used for load and performance testing with some effort. The steps involved are as follows: 1. Create a test script using the scenario that is used for load/performance testing. If it is for performance testing, you might include some logging messages providing the resource utilization like memory or time. 2. Run multiple copies of MarathonITE test runner. Note that, some operations like menu selection etc. expect an active window while the tests are executed. You can use, Xv& on Linux and PSTools on WindowsXP to provide each instance its own GUI desktop - without actually displaying UI on the monitor. 3. Collect the statistics from all the logs and collate. Sometime in future, MarathonITE might support some or all of the above actions itself. However, our goal is to create best of the breed GUI functional Testing tools and this is low on our priority. 7

8 For which platforms MarathonITE is available? Any platform for which Java 1.5+ is available can be used with MarathonITE. Specifically, we certify OSX, Windows and Linux (Ubuntu) for working with MarathonITE. If you face problems with other platforms, please get back to us and we see what we can do. Who uses MarathonITE? MarathonITE is an enhanced version of Marathon. As of date there have been more than 60,000 downloads of Marathon. Since Jalian Systems adopted Marathon and release of version 1.0 there have been more than 30,000 downloads. As with any open source project, it is difficult to predict the total number of real users for Marathon. We run unit tests. Is that not sufficient? The goal of unit tests differ from functional tests. The former is geared towards ensuring the code does what the developer wants it to do, where as the later is for ensuring that the end user requirements are met. GUI based tests (atleast a few) are required to ensure that the general flow of the application works. What is the use of GUI test automation? Automated software testing helps in reducing the cost of quality assurance by letting a machine do the rote testing. Test automation is indispensable when the application complexity increases, because the time spent on regression testing exponentially increases. Automated software testing reduces the time to market of a product by catching regression defects in an early phase. What are MarathonITE s features? Easy to use Intuitive integrated test environment with syntax aware editor. Tests can be created in quick time by using the recording facility. Flexible and Powerful MarathonITE is a flexible tool. You can use all the programming constructs and modules available in Python/Ruby for creating robust, reliable test scripts. Refactoring and Modularization MarathonITE supports convert to module script refactoring which is handy for creating reusable script-lets. When recording just select the module function to use and provide the parameters (if any) and MarathonITE executes all the operations available in that function. Support for data driven tests Use in built support for converting test scripts to data driven tests and test multiple scenarios with a single test script. 8

9 Support for Exploratory and Semi manual test cases MarathonITE supports exploratory and semi manual tests. Use semi manual tests and create a test suite for checking application compliance to enterprise standards. Use exploratory tests and share the created test scripts for communicating the found defects with the development team in a standard and easy manner. Standard programming languages MarathonITE uses Python and Ruby as scripting languages. No need to learn a vendor specific language for effective use of MarathonITE. How much programming expertise needed to use MarathonITE? Not much. In fact, the knowledge of Ruby/Python for effectively using Marathon can be gained in half-a-day by going through language tutorial. MarathonITE distribution has Python/Ruby Concepts Tutorials that provides all the knowledge that is required to use MarathonITE effectively. When needed, you can use the full power of the languages and their runtime and that time a full tutorial of the languages are recommended. How much does MarathonITE cost? The price for one license starts at US$ 1230 including one year maintenance and upgrades. Bulk discounts are available. Please refer to the attached supplement Prices and Licensing. I want to run the scripts from multiple machines. How many licenses do I need? We do not charge license fees for test runners - use as many instances of MarathonITE as you wish. You need to purchase licenses only for MarathonITE instances used for recording and creation of test suites. Help? Tutorial and User s guide Technical F.A.Q You can try Marathon google group Free support from support@marathontesting.com Available screen casts at for general usage instructions. Individual support and training: contact support@marathontesting.com Complete support packages are available. See attached supplement Prices and Licensing. 9

10 Prices and Licensing Licensing Who requires license? Everyone who uses MarathonITE needs a license. Every user who creates, records, edit or modify tests requires a MarathonITE license. MarathonITE test runners licenses are free once a MarathonITE license is acquired. License Types All MarathonITE licenses are floating and entitle you to run one instance of MarathonITE at any given time, independent of the machine it is running on or the person operating it. Multiple licenses allow multiple instances of MarathonITE to be run concurrently. Integrated Test Environment License A ITE license allows unrestricted usage of all MarathonITE functionality to create, execute, modify test scripts and suites. Test Runner License MarathonITE test runner licenses are free of charge once you acquire at-least one ITE license. You can use any number of MarathonITE instances for batch running, unattended execution of tests etc. once you acquire required number of ITE licenses. Pricing MarathonITE License Pricing Number of Licenses Unit Price (US$) (includes first years Maintenance) Maintenance (Annual) 1 $ 1, $ $ 1, $ $ 1, $ Above 10 licenses Site licenses available. Contact sales@jaliansystems.com. * All prices are in USD and exclusive of taxes (if any) that are applicable * Contact sales@jaliansystems.com if you need clarifications * Annual Maintenance is inclusive of major minor upgrades released during the period 10

11 About Us About Jalian Systems Founded in 2005, Jalian Systems is a privately held company based in Bangalore, India. Jalian adopted Marathon in 2006 and provides enhancements and support for the open source product. Jalian s flagship product is MarathonITE - the commercial edition of Marathon, the premier Java/Swing GUI test automation tool. Dakshinamurthy K (KD), Managing Director of Jalian Systems, is a founding member of Linux-India (pan-indian Linux User Group) and ASCI (Agile Software Community of India). He is active in open source communities and contributed to various open source projects like Eclipse, JEdit, Cygwin etc. Prior to starting Jalian Systems, KD was the Chief Technology Officer at Subex Ltd. KD is the main contributor and maintainer of Marathon project. For more information, contact: Jalian Systems Pvt. Ltd. #44, Maruthi Complex, C.M.R Road 2nd Block, Kalyan Nagar H.R.B.R Layout, Bengaluru , INDIA E: info@jaliansystems.com T: M: W: W: 11