Codeless Test Automation

Transcription

1 Codeless Test Automation Business Assurance & Testing AuthOr: Ravi Kant Sharma and Naveen Saxena Automation CoE,HCL Technologies Ltd. Bangalore, India WHITEPAPER January 2015

2 TABLE OF CONTENTS ABSTRACT 3 OVERVIEW 3 NEW BUSINESS DYNAMICS AND TEST AUTOMATION 5 CORE DRIVERS FOR CODELESS AUTOMATION 6 HOW CODELESS TEST AUTOMATION WORKS 7 TOOLS EXPECTATIONS 7 PROSPECTS 8 BENEFITS 8 SUCCESS FACTORS 9 DATA POINTS 10 CONCLUSION 10 ABOUT HCL , HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 2

3 ABSTRACT There are heaps of testing tools available in the market today, and many of them are marketed as being simple to use and maintain, having multiple features. As automation SMEs we come across plenty of real-time instances where we need to leverage coding/ scripting to bring better coverage for automation and to avoid lines of code from accumulating. The more code - the more complexity, and the higher the maintenance costs for scripts as well as frameworks. Thus, as practitioners, we need to be able to make this journey with as little coding involved and achieve higher quality and maintainability, setting the foundation for Codeless Automation. This whitepaper highlights the need for a Codeless Test Automation Framework that allows us to effectively design automated tests without any programming language and make test design, execution, the tracking of results, and script maintenance, easier. This paper is for business managers, test leads and software test engineers and the key focus areas of application include system and regression testing. OVERVIEW In the traditional Test Automation approach, sophisticated test automation tools were used with the understanding that the tools would do the job. We relied up on the Record and Playback feature available in thetool with small augmentations and code changes to perform validations on Application under Test (AUT). Cheaper Automation with Record and Playback Simpler Automation with Reusable Functions Faster Automation with Frameworks Better Codeless Automation Fig 1: Evolution of Test Automation Test Automation revolves around the concept that advanced with Microsoft and revolutionized the entire IT industry. Post this, automation tools flooded the market and each vendor boasted of a better tool. The Recordand Playback functionality in each of these tools became a selling point, making it easy for testers to use these features for repetitive tasks with parameterization of the test data. 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 3

4 This eventually led to an era of sophisticated code development and modularization where many automation frameworks came into being, as listed below: y Keyword Driven Framework y Modular Framework y Hybrid Framework y This table highlights the key differences between the frameworks mentioned above: Keyword Driven Framework Keyword-driven /excel driven are interchangeable terms that refer to an application-independent automation framework. This framework requires the development of data tables and keywords/functions. Modular Framework Hybrid Framework Here, the test script code drives the application-undertest and the data. Keyword-driven tests look very similar to manual test cases. The modular framework requires the creation of small, independent scripts and functions that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests. It involves coding/scripts for each of the test cases, and hence is required to be the simplest to grasp for automation engineers. The hybrid framework is the most commonly implemented framework. It is a combination of both modular and keyword-driven frameworks and leverages the merits of both while mitigating their weaknesses to a certain extent. Table 1: Key automation frameworks types These automation frameworks resolved multiple issues in relation to scripts standardization, scripts uniformity, faster development, etc. Most of the time, this would lead to munificent test maintenance efforts that arose due to the complexity involved in these automation frameworks. In some instances, maintenance efforts exceeded half the development efforts due to complexity arising out of debugging the script according to the automation framework that was employed. 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 4

5 Setting up Automation F/W Structure Developing the base function libraries for available technological stack Developing the Result Reporting Mechanism Framework Integration with HP-ALM Framework Development Test Automation Implementation Analyze Manual Test Cases for Automation Feasibility and Reusability Manual Test Case Management Developing Automated Script for feasible Test Integrating Test Script with HP-ALM Reviews and Fixes Setting up Automation F/W Structure Developing the base function libraries for available technological stack Developing the Result Reporting Mechanism Framework Integration with HP-ALM Test Automation Maintenance The figure above illustrates the different stages of test automation commencing from the development of the automation framework, to test execution and maintenance. Here we spend most of the time in test script creation and maintenance arising due to application changes or data changes. We also expend considerable effort in developing the framework that drives the need for codeless automation where we could create an automated script without any specific automation framework. NEW BUSINESS DYNAMICS AND TEST AUTOMATION With new business dynamics in the software development life cycle and the introduction of the Agile methodology, test automation became more and more challenging. In the traditional Waterfall development approach, the product is built stage-by-stage and testers plan first and then enforce the plans through development. Here, the test automation team predefines a test automation plan within anticipated timelines for a given project scope. On the other hand, Agile is a non-linear approach where the entire software product is split into small fortnightly or monthly releases when the project scope is unstable. Unlike Waterfall, it has multiple iterations with less time to perform test automation. It is characterized by more time being spent in test automation development and maintenance, creating tests and updating those based on new scope changes and executing them. This leads to the foundation of unconventional script-less test automation that could simplify test automation development and maintenance. 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 5

6 Background Challenges Approach Agile Projects present their own challenges to the Automation team; unclear project scope (changing requirements), multiple Iterations, minimal documentation availability, speedy deliverables for frequent releases, active involvement of stakeholders and unavailability of application. Inconsistency in Test Automation Scope Minimal documentation Changing Environments i.e. URL, Login Credentials, Workflow Changes, Application look and feel changes Speedy deliverables for frequent releases Active stakeholder involvement Object Maps changes from one sprint to another Over commitment with limited sprint durations Selection of What to Automate and What Not Selection of Right Framework Selection of Approach for speedy deliverables and goal success Selection of Right Resources and Tools Resource Management (to have skill set both in manual and automation testing) Developing right and quality automated test scripts for better reusability To run test suites in regression testing smoothly Adaptable for changing requirements To have less maintenance overhead Selection of Right Test Automation Tool To meet Test Environment Requirements, Application Requirements To meet Test Automation Requirement User friendly To give quick results and reporting abilities Keep your tests small and lean To help in dealing with complex scenarios To help in debugging To enable easy integration with other tests Selection of Right Approach and Framework A new era commenced with Codeless Test Automation, where we moved away from conventional, time consuming, and code-based processes of writing code. This helped both the test automation team and the manual team create and update tests easily. Codeless Test Automation made it easy to use, with automated assets in English and in a format that is easy to create and maintain. CORE DRIVERS FOR CODELESS AUTOMATION With increasing cost pressures, IT companies are focusing more on the cost benefits and on improving work efficiency. The core drivers aggressively encouraging the evolution for Codeless Test Automation are: y Speed (quick time to market i.e. in Agile) y Maintainability (less maintenance overhead cost) y Configurability (make test automation assets easy to confirm for different requirements with minimal changes) y Robustness (easy plug and play anywhere) y Work efficiency (with codeless test automation that reduces up to 80% of the test automation efforts, saving time for test engineers that can be spent on core business areas for results analysis, finding defects, and executing more tests) HOW CODELESS TEST AUTOMATION WORKS Codeless Test Automation is a concept/methodology supported by a test automation tool. It has a user-friendly GUI with provisions that enable test engineers to create test cases by simply choosing objects and adding operations to them. This is done by selecting the actions from a drop-down lookup list along with the provisioning of test data passed locally or externally, while making use of the codeless test automation 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 6

7 tool. The end users do not have to program scripts for each test case and test steps in any scripting language. Hence, Codeless Test Automation becomes language agnostic. TOOLS EXPECTATIONS Codeless Automation is expected to bridge all gaps that code-based test automation carries. It is expected to offer coverage to the following environments: y Multiple browser support: for cross browser testing y Multiple technology support: for supporting all existing technological platforms y Multiple database support y Multiple file system support y Data-driven testing support y Mobility support y Good and flexible reporting mechanisms y Plug and play assets: offering the flexibility to plug and play test automation assets and calling already automated assets/components y notifications y API access for integrating with tools such as Test Management, Bug Tracking, etc. y Provisioning of custom objects/controls in the application, such as Virtual Object Handling y 12. Provisioning of Batch Runs and Scheduling The industry currently does not have any tool that covers all the expectations listed above. 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 7

8 Mobility Multiple Browser Data Driven Testing Automation Solution Multiple Technology Multiple File System Multiple Database PROSPECTS As described above, Codeless Automation is a futuristic idea to mitigate most of the challenges of Test Automation, but it also raises a concern on whether code-based test automation will eventually vanish. While this is quite subjective, with the present findings and the pace in technological development, it seems evident that script-based test automation may eventually disappear and the void befilled with Codeless Test Automation. BENEFITS Codeless Test Automation offers several benefits. Some of them include: y Automating test cases becomes easier and more comprehensive; we can see what is automated and what data has been used y Time for automating tests/scenarios is reduced considerably through pre-defined sets of methods to be operated on test objects; less time is spent in developing functions or procedures y Test maintenance becomes easier and less time consuming as code level debugging presence is negligible y Training non-technical test engineers or business analysts becomes easier, enabling them to contribute to test automation or review automated test scripts y Creating and managing complex tests becomes easier y Testers can spend more time analyzing test results, detecting defects, and focusing on business areas/ functional aspects, etc. y Both automated and manual tests can reside in the same location and provide 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 8

9 transparency about what is automated and what is not. It also helps eliminate the cost of having another tool to maintain manual tests. y Promotes greater stakeholder participation and increases testing process productivity SUCCESS FACTORS The overall success of Test Automation does not depend on how many tests have been automated or how much better we have coded the tests but on what we have automated, though the numbers may bring up a value and not necessarily the desired success. We see that the decision to opt for automation, the candidates for automation, and the tool used to carry out the automation - all plays a very vital role in the success. Here the candidates (automation requirements i.e. manual tests/scenarios) are the preliminary factors that determine automation success. We therefore emphasize on the following factors: Approach Selection of Right Candidates Test Strategy Quality Considerations Tool Selection Faster Simpler Cheaper Better Reusability and Repeatability Complexity of Tests Inter Test Dependencies Data creation Tests that needs to run on different environments Business Indispensable tests Quality Clean up of Test Cases (Redundancy Removal) Improve overall quality of test cases Optimize regression test cases Focus on End to End Testing that mimics business use Emphasize on testing Needs Better Test Coverage Consistency Reliability Robustness Coverage to Application and Environment Ease of creating and updating tests Ease of debugging test scripts Result reporting Skill availability License cost and affordability Assimilation with other Tools DATA POINTS Based on our findings across various projects, we have seen that a considerable amount of saving is achieved using Codeless Automation. Let us consider a typical scenario with minimal functional and technical complexity and the availability of a Test Automation Framework with all processes in place. We find that there is a significant reduction in overall test automation efforts with the use of Codeless Automation, and we save around 56% in automation efforts per script. 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 9

10 Traditional Vs Codeless Automation Efforts Vs Cycles Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Traditional Automation (in Mins) Codeless Automation Traditional Automation (in Hours) Codeless Automation (in Hours) CONCLUSION Based on the above findings,we can conclude that Codeless Test Automation has a number of advantages and is also scalable for Agile projects with limited timeframes and unstable scopes. This, however, does not mean that Codeless Test Automation is only suitable for Agile; it can also be plugged in to Waterfall projects. The bigger challenge for these projects include the migration of existing test scripts from existing tools to newer ones supporting Codeless Test Automation. While critics may argue that Codeless Automation poses a threat for the learning curve as well as for test engineers, we believe that it makes the job simple for them and for the business. 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 10

11 ABOUT HCL About HCL Technologies HCL Technologies is a leading global IT services company working with clients in the areas that impact and redefine the core of their businesses. Since its emergence on the global landscape, and after its IPO in 1999, HCL has focused on transformational outsourcing, underlined by innovation and value creation, offering an integrated portfolio of services including software-led IT solutions, remote infrastructure management, engineering and R&D services and business services. HCL leverages its extensive global offshore infrastructure and network of offices in 31 countries to provide holistic, multi-service delivery in key industry verticals including Financial Services, Manufacturing, Consumer Services, Public Services and Healthcare & Life sciences. HCL takes pride in its philosophy of Employees First, Customers Second which empowers its 95,000 transformers to create real value for customers. HCL Technologies, along with its subsidiaries, had consolidated revenues of US$ 5.5 billion, for the Financial Year ended as on 30th September 2014 (on LTM basis). For more information, please visit About HCL Enterprise HCL is a $6.7 billion leading global technology and IT enterprise comprising two companies listed in India HCL Technologies and HCL Infosystems. Founded in 1976, HCL is one of India s original IT garage start-ups. A pioneer of modern computing, HCL is a global transformational enterprise today. Its range of offerings includes product engineering, custom & package applications, BPO, IT infrastructure services, IT hardware, systems integration, and distribution of information and communications technology (ICT) products across a wide range of focused industry verticals. The HCL team consists of over 100,000 professionals of diverse nationalities, who operate from 31 countries including over 505 points of presence in India. HCL has partnerships with several leading global 1000 firms, including leading IT and technology firms. For more information, please visit Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration, applied innovation and new generation partnership models that put your interest above everything else. Right now 100,000 Ideapreneurs are in a Relationship Beyond the Contract with 500 customers in 31 countries. How can I help you?