White Paper Why OpKey is advantageous solution with Open Source Test Tools and Comparison with Robot Framework Functional Test Automation with Open Source Tools is always a cause of anxiety for testing teams. Organizations across globe are exploring the space of free test tools so to ensure quality of their deliverable, ability to customize the open source free test tools as per their project needs and all this without any cost of expensive license solution as with commercial tools. Author Pallavi Sharma pallavi.sharma@crestechglobal.com Visit us at: www.crestechglobal.com
Why OpKey is advantageous solution with Open Source Test Tools and Comparison with Robot Framework Functional Test Automation with Open Source Tools is always a cause of anxiety for testing teams. Organizations across globe are exploring the space of free test tools so to ensure quality of their deliverable, ability to customize the open source free test tools as per their project needs and all this without any cost of expensive license solution as with commercial tools. Tools like Selenium RC, Web Driver, Robotium, Watir have now developed serious client list of companies from small, to mid level to fortune 500. This shift in trend towards open source tools has lead to a set of all new challenges faced by now a days test teams. Some of the challenges are 1. Requirement from test teams to be programming language savvy, a forte held usually by the development team 2. Not being able to use effectively manual testers ability and participation of Business Analyst also remains a challenge 3. High maintenance effort required to sustain the open source test tool adaptation within the team 4. Heavy Dependency on skilled and costly resources. 5. Maintaining external test data management solution 6. Maintaining external result management solutions 7. Leveraging of existing artifacts with other test tools like defect management tool, test management tool is a challenge 8. Ability to ensure robustness of the open source test tool implementation to the ever changing business requirement is pain area many are familiar with. 9. To add to all this now a days business applications are complex in nature with various interfaces of web, mobile, web service, desktop. To test all such interfaces and their integrations using different open source technology is a mammoth task. All the above mentioned challenges leads to such high cost incurred per test that many times the ROI of adapting open source test tools is not justifiable versus the commercial counterpart. OpKey, as a solution was conceived by CresTech to make available a platform which provides a seamless experience to test teams for open source test tools adaptation. The many useful features in OpKey helps QA Teams overcome challenges faced during open source test automation adaptation. In here laid out are a few features which will help explain why using OpKey with open source test tools like selenium is much better than using them stand alone. ZeroCode Automation Enterprises now days want their manual testers and business analyst to also participate in automation. They are looking for test automation solution that allows non technical users to work efficiently.
Open source test automation tool adaptation requires significant programming knowledge, which only a skilled resource can have. Automation using open source tools makes it difficult to find a common language which all three participants, automation tester, business analysts and manual tester can communicate with. Such a situation is of concern and causes many failure to seamless adopt open source technology which otherwise has many advantages successfully for test automation. OpKey rich Gen 5 keyword driven interface provides a common platform to everyone and enables one to author test cases without any requirement for prior programming knowledge. With more than 400 keywords available that covers almost all actions to be done on objects of application under test. Along with there are keywords to handle database, xml operations, string, file handling date time etc. Fig 1 It is actually possible with OpKey to do test automation without even writing a single piece of code with open source test tools like Selenium, Watir etc. Tool Agnostic Engine You have prepared your scripts using Selenium as a tool and down the line after two years you have realize that you might have to shift to another automation tool. So are your months of effort wasted? Because it s a challenge in itself to migrate scripts prepared using one automation tool to
another. Test teams are looking for solution that provides this flexibility and freedom to not be bound by any specific test tool for automation. OpKey patent pending core-engine provides a tool independent solution; where scripts prepared using one automation tool can be migrated seamlessly to another. So you can actually author test scripts using one automation tool and down the line migrate it to other test tool without much glitches. Fig 2 Multi Tool Support Application now a days are complex in nature, where interfaces ranging from web, mobile, service to desktop. To test such complex scenario you require more than one open source test tool technology to cover all interfaces. Creating such a solution is both challenging as it requires different skill sets and its maintenance comes with a lot of over head. No one open source test tool makes available a solution which can cover varied interfaces and provides one solution for automation of complex business applications. OpKey patent pending core engine, provides the support to call more than one open source test tool in the same script. So you can easily automate the web button click with selenium, verify the data of web service using Soap UI, check the text received in the mobile application using robotium and the backend database using a sql query. All this is feasible in one single test script. No automation
solution so far provides such flexibility which OpKey does. Also testing of rich web application with custom controls is feasible with OpKey image based test solution and can be easily used in collaboration with test tools like Selenium, Watir etc. Fig 3 Parallelize Automation with Cloud Now a days the cross browser testing of a web application is a mammoth task due to various combinations of OS and browsers available. With each release cross browser testing on different test environments with various combinations of OS and browsers requires heavy infrastructure in form of a test lab management. Open source test tool although comes with no license fee, means its available for free but ensuring that your test lab is up-to date with their patch releases, and it is more of an overhead on the test team to manage this open source test lab than on the open source community. Most of organizations IT team don t have such skilled resource, so in the end that one skilled resource is burdened by creation of automation suites, sharing of results, maintenance of the test lab impacting his productivity and also impacting actual testing of application with efficiency. This can be easily avoided by exploiting the cloud test automation arena. OpKey cloud provides more than 100 different combinations of OS and browsers, allows scheduling run suites in parallel. Also your tester is free from the over head of maintaining the complex test lab with open source solutions. They can simply concentrate of preparing better and effective test scripts with OpKey and then with ease chose configurations from the rich set of 100 combinations of OS browsers available. Once the runs are scheduled on the OpKey cloud, the machine is also free to the user for other activities. Save time and cost with the OpKey cloud.
Fig 4 In Built Test Artifact Management Open source testing adaptation comes with a huge baggage to build and maintain your own solution to manage test artifacts like object information, test data, managing results, reports etc. Applications are usually dynamic in nature and it becomes quite difficult to ensure your test scripts are up to date with the ever changing object definition. So the end result is that by the time you have stabilized your test automation suite with version 1 of your application, a new version comes with changes in objects definition and your automation suite fails in the regression. So you become trapped into a vicious cycle of ensuring your automation scripts are upto date with your application changes, rather than performing actual test automation to verify quality of application under test. You also have to rely on external data management solutions like excel, csv files or open source database solutions like mysql and if your application is of data intensive nature the more difficult this external management becomes. There is almost no out of box feature available with open source tool for result management. Some do provides libraries but their implementation is left to the test teams who use them. So in the end to ensure a smooth process you end up building your own framework with open source test solutions. OpKey provides in built test artifact management solution. It provides in built object management solution. The objects are easily captured using OpKey IntelliChoose technology. One can if required go and change the physical description of the captured objects. You can also add your own virtual objects if they don t exist in the application.
Fig 5 With OpKey you do not require any external test data management solution. It provides excel like interface in form of data repository. It also allows you to import data from various sources like excel, xml, csv, databases etc. Fig 6 It also provides in built result management solution which allows results to also be exported in various formats to be shared with stake holders. So why reinvent the wheel with every open source test tool adoption, use OpKey and have freedom to concentrate on actual test automation with open source test tools.
Fig 7 Built in Support for Exceptional Handling You were finally able to over-come all challenges of open source test tool and have built in your framework, prepared your test automation suite. You finally start using it in production for nightly regression suite runs. And then when no manual intervention was available a pop up appears sayingyour OS requires update. You come in the morning to find your test automation suite paused and the screen showing that pop up. Open source tools do not come with feature of handling exception situations which are commonly available in their commercial counterparts. So you now have to built your own exception handling solution as well, or you can use the inbuilt one available with OpKey. Fig 8 For hassle free test automation experience, in OpKey there exists support for handling of exceptional situations which can be encountered during execution. This is done with the help of advance recovery manager. Many events are recognized with recovery manager, which can be captured and a proper path can be defined to come out from it in a clean manner. This feature helps saves time and effort during test runs which are scheduled nightly. Fig 9
Built in Scheduler Once the automation test suite is prepared, it is actively used for regression tests. No open source tools provides an in built scheduler feature which is a must to start your test automation run without any manual intervention required. With OpKey you can define a given date and time when you require the regression suite to execute and also on a defined machine which requires only an OpKey agent to be installed on it. So utilize your resources efficiently with OpKey scheduler feature. Fig 10 Integration with CI Now a days the test automation solution is required to have integrations available with the development release build out of box. With open source tools you can surely build your own CI integrations and support is also available. The task though challenging but is achievable. OpKey provides out of box integrations with Continuous integrations tools like Jenkins, Hudson. Once your application batch is available from Jenkins, it sets into trigger the regression test automation suite in OpKey. This feature is extremely helpful and provides an end to end test automation solution to the end users without any manual intervention required. Fig 11
Integration with ALM Automation is no longer a stand-alone process. Many commercial application provides a complete application life cycle management solution. Such enterprise level solution is missing from the open source test automation tools. OpKey provides integrations of ALM tools with open source thus allowing test teams to leverage on their existing artifacts. Fig 12 Collaborative Automation Support Test teams do not work in seclusion. A collaborative approach to test automation is a necessity for project success and in time finish. With open source test tools you will require to develop using third party tools your own methodology to share test artifacts within teams. This can easily lead to loss or duplicity of data as visibility and transparency are usually missing. OpKey provides a platform to testers in a team to collaborate and work together simultaneously on the same project at the same time with open source tools. It provides features like identity based login, locking of files features which allows users to collaborate together for an automation project.
Fig 13 Business Process Test Support Enterprises now a day s require the SME s and BA s to actively participate with test automation. The test automation zone itself sounding technical and with open source adaptation difficulties, organization find it difficult to bring together a platform where the business analyst can participate actively in test automation with the automation tester. Commercial solutions like HP provide solution for it. But such a feature is missing from the open source test tool community. OpKey provides a platform which allows one to follow business process test approach with open source tools. One can easily create business components, which are then combined to form business flows. Fig 14
The BA can fill in the appropriate business data require during test run. And with one click your business suite is all ready to be executed using open source test tool technology. Fig 15 Advance Reports and Analytics Most of the open source tools requires integrations with different libraries to show case results for sharing with management to facilitate their decision process. This is a time consuming activity, but a necessary activity. OpKey provides in built advance reports and analytics feature which makes available more than 15 KPI s to help people take those managerial decisions faster and efficiently. Fig 16 There is no requirement to build your own reporting interface with OpKey. One can also get customize reports with OpKey as per their requirements.
Fig 17 Available below is a comparison table of OpKey with another widely used test automation framework knows as Robot Framework. Feature Robot OpKey Comment Introduction It is a Framework for Acceptance Test Driven Development It is a Functional Test Automation Tool Agnostic Platform, which provides end to end test automation solution In Built Object Not Available Available Management In Built Data Not Available Available Management Keyword Driven Available Available Interface GUI Not Available Available OpKey provides a rich Gen 5 GUI for a world class experience to its end users Programming Yes No knowledge required for implementation Integration with External Test Tools Available in form of many libraries, one for each tool Available in form of many plugin libraries for each test tool Both solution provides libraries with external test tools and interfaces with databases, xml operations etc. Scheduler Not Available Available A scheduler is a crucial
feature required to schedule test runs at a given time. OpKey provides an in built feature for it. Reporting Not Available Available Recovery Manager Not Available Available This feature is required to ensure that the exceptions encountered during the test suite run are handled gracefully and the execution is not hampered. OpKey provides this as an inbuilt feature Cloud Enabled No YES With OpKey cloud available now run your suite on more than 100 browser OS configurations available. Out of box Integration with CI Tools Out of Box Integration with Defect Management Tools Support for BPT Mode of Testing Collaborative Automation Support Yes, it has available Jenkins plugin. No Yes, support with CI tools like Jenkins, Hudson available. Yes No Yes OpKey BPT Edition enables business analyst to participate actively in test automation No Yes With feature like identify based login, locking, unlocking of component, shared database, OpKey allows many people of a team to work on the same project
Mobile Test Automation Support Multi Test Tool Support in one Test Case Out of box support not available Out of box support available. One can import objects from mobile apk, author test cases and easily execute them using pre built keyword set with OpKey. simultaneously. No Yes OpKey Tool agnostic engine allows integration of different test tools in one test case script. Recorder No Yes Active Support Dependent on response from users of community Active support, round the clock available 24/7 Edition One available, for FREE Many editions, for diverse needs. Personal Available for FREE Team- For Testing Team size >2 BPT- For Business Process testing. Link http://robotframework.org/ http://opkey.crestechglobal.com So where as Robot is just a test automation framework which provides a keyword interface over many external tools with its rich library set, OpKey is much more than just a keyword driven test framework. It is a complete end to end functional test automation solution. OpKey is essentially a tool agnostic automation platform that allows users (Dev, Tests, BAs/SMES, Everyone) to create automation test script without needing to know any automation tool or programming language. It allows team to leverage on their existing artifacts by providing out of box integration with CI tools, ALM tools. It also provides support for collaborative test automation and business process test out of box unlike the Robot Framework. The Scripts prepared in OpKey are Tool agnostic, which means test case created with selenium can easily be executed with QTP as well. OpKey also allows calling of multi tools in the same test case something which no automation solution provides as of now. OpKey also provides 24/7 round the clock active support to its end users. We ensure that in time patches and updates are provided as we always remain up to date with changes in the open source test automation world. OpKey unlike robot framework understands that one automation solution cannot fit everyone needs. For the same reason OpKey makes available different editions in form of Personal; Team; BPT and execution on cloud for test teams to chose from for their requirements. It allows customization of features as per need of end users. Many MNC s, banks have already established OpKey as their de-facto test automation solution over open source test tools. So if you are thinking or are into open source testing, you should definitely think OpKey. There is no other test automation solution available which provides a complete end to end solution for test automation with open source test tools.