Automation 2.0 Taking test automation beyond regression testing d Global Delivery Centre: 401-408, A-Wing, Pride Silicon Plaza, S.B. Road, Shivaji Nagar, Pune -411006, Maharashtra, INDIA Email: info@nitorinfotech.com Tel: +91-20-41020202
Introduction Whenever Test Automation is a point of discussion, it has always been linked with regression testing. Traditionally, it has been believed that test automation can only be used to support regression testing by taking the existing manual tests & converting them into automated test suites which are then rerun as part of regression testing to assess the quality of the software. Although there is no denying in the fact that test automation is best suited for regression testing, however there are few more facts attached to this. Regression tests of this sort miserably fail to detect new defects. Such automated regression suites are the simplest form of test automation which is highly expensive to maintain. It is fragile and breaks easily and hardly utilizes the full potential of test automation. These regression suites merely automate what the manual testers have been doing, in fact many a times, losing many of the advantages which manual testing has for finding defects. Through this document, Nitor intends to go much beyond the regular regression testing and try to throw light on what all can be achieved by leveraging the strengths and possibilities of test automation. Although there are limitless possibilities which one can achieve through test automation, this documents restricts to only few of those key possibilities, which would help in effectively extending the reach of test automation much beyond traditional regression testing Automation beyond regression Below mentioned are some of the key possible areas, wherein test automation can be utilized to leverage and fully utilize the immense potential of test automation Building a BVT (Build Validation Test) suite Creating test data for heavy data dependent scenarios Continuous Integration, thereby facilitating end to end (Build-Deploy-Test) testing Automated Unit Testing Technical testing such as Web Service, SOA and API testing
BVT (Build Validation Test) Suite With extensive adaption and implementation of agile methodology, there has been a drastic shift when it comes to test execution cycles. Unlike the traditional Waterfall methodology, wherein the QA team gets a prolonged test execution cycle, in Agile, the test execution cycles have become much shorter & a repetitive effort. Within a sprint cycle (of 2 or 3 weeks), there are multiple test execution cycles which are carried out to test the intended functionality being developed and released. The test execution cycles usually begin with a Smoke test cycle, wherein the basic functionalities of the application are checked, just to ensure whether the build is stable enough & testable enough so as to proceed further with a comprehensive functional testing cycle. This smoke testing suite usually consists of a fixed set of test scenarios/test cases designed using a shallow & wide approach covering the basic aspects & key functionalities of the entire application. These tests are subset of tests cases that verify main functionalities. As this smoke test suite is executed for every build cycle (almost on a daily basis), it does make sense to have it automated. This automated smoke test suite is nothing but the Build Verification Test suite which is a set of test cases run on each new build to verify that the build is testable. Benefits of automating the BVT suite Saves the manual testing effort in executing the same set of test cases manually for every build cycle Helps to surface out critical defects early Helps to save the efforts of QA team in setup and testing of the build, when major functionality is broken Test Data Generation One of the most important aspects in any testing activity is the usage of effective test data. Test data plays a very important role in success of testing cycles. The test data drives the test cases. However, generating the test data becomes a humungous task, especially for data driven/menu driven applications which are heavily data dependent. In such cases, generating the test data takes away valuable time of the QA resources. At times, when the customer is ready to share his live/production data the life becomes less miserable, however using customer s data is never a wise approach and at times, if such live data is exposed to unauthorized people, it not only puts the credibility at stake, but eventually leads to loss of business.
To overcome such situations and to generate effective and quality test data without spending too much of manual effort, generating the test data through automation scripts is the most viable and feasible solution. There are several test automation and automated tools available which can be utilized for creating a comprehensive and effective set of test data. Continuous Integration Continuous Integration (CI) is a practice/process in which the developer integrates the code into a shared repository frequently, which can even be several times a day. This arises the need to test the quality of the build for each commit or each check-ins. Carrying out such repetitive tests manually consumes a lot of manual efforts. Moreover, usually as it is the same test of scripts which needs to be tested every time, it makes more sense to have these tests automated rather than testing them manually. This is where executing automated test scripts in a CI environment is beneficial. In CI, each of the integrations or check-in builds are verified by an automated build, allowing teams to detect integration and other errors/problems early. CI allows the team to run automates test scripts after each commit & thus convey the results to the development team quickly and help to locate the issues more easily. The below diagram depicts the CI process:
All the above mentioned phases/activities of CI can be automated, right from compiling the build to publishing/deploying the build to triggering the unit testing suite, automated BVT suite, functional testing suite to reporting the results to the relevant stakeholder, everything can be made possible through a single test automation framework Using test automation in CI process effectively can yield multiple benefits. Some of these have been mentioned below: Helps in unearthing the issues/defects early and also helps in identifying the location of those defects Debugging time is reduced, hence developer can spend more time on their core activities Better confidence on the quality of the build Risks are reduced Automated Unit Testing There is no denying in the fact that, it s virtually impossible to write a bug free code. We humans are bound to make mistakes. Hence, testing of the code been written by the developer is so very imperative in the overall development cycle to ensure a thorough quality check. This is where Unit testing comes into picture. A unit test is a piece of code that tests another piece of code. This unit testing can be automated by using several unit testing tools. There are several commercial and open source tools available such as N-Unit, J-Unit, Struts, TestNG etc: Based on the platform and the technology for which the code needs to be tested, the appropriate automation tool and framework can be used. An automated unit test suite has a number of important and tangible advantages: Unit tests find problems early in the development cycle. An automated unit test suite test your code in two dimensions:
o Time dimension because once you ve written a unit test, it ensured that the code works well for now and in the future o Space dimension because unit tests written for other features ensures that the new code did not break them The development process becomes more flexible Reduces the need and effort for manual testing All of the above leads to making software more predictable, robust, repeatable and better quality. Technical testing API testing, SOA testing and Web service testing Although technical testing can be carried out manually, however there are several limitations and challenges in doing so. Complexities and dependencies needs to be addressed Lack of required technical skills Human errors leading to erroneous results These limitations can be easily overcome by implementing test automation for carrying out these technical testing activities. There are several tools (both commercial & open source) currently available in the market which can be used & by leveraging the inherent benefits of automated testing tools; the technical testing assignments can be made much more effective, reliable, faster & error-free. Conclusion Test Automation can do much more than traditional regression testing. Hence, limiting test automation for performing functional testing or regression testing would not only do injustice to test automation, but would also be like utilizing just 10% of its overall capabilities. Leveraging the benefits of test automation and utilizing it effectively in tasks which are complex, tedious, time & energy consuming and difficult to carry out manually is the what the current time & situation demands.
About Nitor Test Services Nitor Test Center of Excellence (TCoE) is an independent test practice with well-defined testing processes and profound knowledge of tools and techniques to cater to the testing needs of varied products/applications. Our professional team of test analysts has worked on products of varying complexity in terms of technology and functionality, projects and solutions across focus areas such as Business Intelligence, Portals & Collaboration and Enterprise Mobility. Nitor offers end to end testing services at aid our partners in continuous business growth through our niche technology focus and diverse domain expertise. Our TCoE offering include Highlights of Test Center of Excellence (TCoE) ISTQB Gold Certified test partner SSAE Soc- II Type-1 accredited organization Technology and platform agnostic. NitorTCoE is technology /software / tool independent 70 + strong team of certified test engineers Seven years of profound QA experience Strong knowledge base with expertise in diverse methodologies and tools
Contact Us NitorInfotech 401-408, A-Wing, Pride Silicon Plaza S.B. Road Pune -411006; Maharashtra, INDIA Email: info@nitorinfotech.com