AGILE SOFTWARE TESTING Business environments continue to rapidly evolve, leaving many IT organizations struggling to keep up. This need for speed has led to an increased interest in the Agile software development methodology. Introduced in 2001, Agile s focus on customer service, quick delivery and continuous feedback has made it one of today s go-to development methodologies. However, adopting Agile in today s multi-vendor, multi-location project environment can be challenging since the methodology requires new ways of managing SLAs and holding vendors responsible for their deliverables. IT organizations that attempt to bring outsourcing vendors into their Agile environments or transition their Agile-novice vendors are often met with surprises when these vendors fail to keep pace. TEKsystems.com 1
The TEKsystems Approach to Agile Testing TEKsystems provides testing services using our proprietary quality management approach, which is based on standard Agile principles and augmented with our own expertise. Our services are customized to meet each client s unique business needs. We utilize proven testing approaches to effectively support the client s technical environment. Unlike many quality assurance (QA) organizations that struggle to maintain quality in fast-paced Agile environments, TEKsystems embraces an approach to quality assurance and testing that includes: 1. Adopting a team approach to identifying and addressing quality issues 2. Doing things right the first time 3. Reviewing quality throughout all stages of the development process 4. Automating tests early and often The chart below shows how Development, Business and QA work together to produce a quality application in an Agile project. QA Business Development Assigned Review Intent Clarified Walkthrough Developed TDD Coding Pair Programming / Code Review Set Criteria Test Creation Provide Automation Support Provide Feedback Desk Checks Execute Unit & Tests Final Desk Check Intent Confirmed Accept Deploy to QA Smoke Test Tested Sprint Demo Delivered Deploy to UAT Test Case Creation Automation Creation Exploratory Testing Regression Testing QA Signoff The Importance of Automation In order to meet ever-increasing regression testing needs, Agile project teams incorporate automation into their testing strategy. This strategy addresses all of the automated testing responsibilities of the entire team, including any unit testing performed by developers and incorporating the input provided by the business units. While customizable for every project, the automation testing strategy utilizes best practices based on previous experiences and real-world lessons learned. Each project s automation testing strategy includes the following components: Strategy development Maintainable test automation framework Level-specific tools and approach for each level of testing TEKsystems.com 2
High-impact automation, including automated unit testing Collaboration with Quality Assurance (QA) team for knowledge share and automated user story testing Tools aligned with the client environment, budget criteria, technologies and available skill sets An effective test automation strategy requires automating tests at three different levels, as shown in the diagram (at right). In the traditional waterfall test automation approach, unit tests constitute the smallest percentage of test automation the majority of automation is end-to-end regression. Automation begins after a long period of manual testing to find, resolve and close defects. Conversely, the Agile test automation pyramid calls for more automation at the unit and integration levels. In an Agile environment, this streamlined approach for test automation can be executed quickly and often. These low-level automated tests also provide precise information to developers when failures occur. Although end-to-end regression tests comprise the smallest percentage of test automation assets, their importance is vital. This type of automation is used in end-to-end test cases for regression testing and can test multiple stories as one business transaction scenario. These test cases are designed while new stories are developed and re-factored. Once the new stories are integrated into the application, test cases are executed manually to find and resolve issues and are only automated once all defects are corrected. Instead of creating a full suite of regression tests, end-to-end automation scripts are constructed in at least two suites: smoke tests and full regression tests. High-risk test cases for positive conditions are automated first, followed by automation to cover more negative conditions. A successful end-to-end test automation is dependent upon usage of the right framework and tool. The project team works together to select a tool that is compatible with the technologies chosen for the application and with a framework that is robust, scalable and easy to maintain. In-depth discussions on test automation allow developers to design code that is easier to automate (e.g., using unique object IDs or creating test hooks). The test team should also have familiarity and expertise with the chosen tool and framework in order to grow and maintain test automation independently. Traditional Waterfall Test Pyramid Unit tests constitute the smallest percentage of test automation the majority of automation is end-to-end regression. Automation begins after a long period of manual testing to find, resolve and close defects. Agile Test Pyramid Conversely, the Agile test automation pyramid calls for more automation at the unit and integration levels. In an Agile environment, this streamlined approach for test automation can be executed quickly and often. Key Characteristics of Successful Agile Testers Agile testers understand design discussions, help the business refine user stories, help developers test the backend logic, perform manual testing, provide feedback to the team, automate test cases and report problems as they arise. A successful agile tester should possess the following skills: TEKsystems.com 3
Demonstated QA expertise Ability to manage software quality Robust Agile knowledge and test management experience Practical knowledge of software design and a rchitecture Keen understanding of an integrated development environment Familiarity with the programming language being used Exceptional test automation skills Business domain knowledge Intuitive understanding of the end users usability needs Strong communication skills Since it is often difficult to find testers who possess all of these skills, IT organizations may want to pair an automation engineer and a manual tester. Another alternative is to have your test team initiate two streams: one with a functioning testing track and the other with a test automation track. Keep in mind that you will also need to institute a sound knowledge transfer process to facilitate communication between the two streams. Benefits of Partnering with an Agile-Experienced Vendor An Agile-experienced partner can successfully leverage the agile methodology, processes and associated human capital skills to enable businesses to achieve the following benefits: Synergy between Business, Development and QA Open communication and trust are hallmarks of successful partnerships. Collaboration enables constant testing and consistent feedback beyond the confines of typical vendor-client relationships. An experienced QA partner can present options rather than constraints. A Proven Agile Testing Methodology Agile teams follow development best practices such as unit testing, test-driven development, code reviews, continuous integration, source code management and automatic build deployment. An effective QA partner must also follow a proven test methodology including test planning, test case design, test automation, defect tracking, test environment and data management. There must be mutually agreed-upon processes to ensure that the Agile QA process runs smoothly from beginning to end. Project Governance Your partner should have dedicated project oversight to ensure your project runs as expected. A formal Project Management Office (PMO) utilizes sound methodologies, best practices and lessons learned from other similar projects to complete QA initiatives on time and within expectations. Further, it is highly advisable to have an independent governance office that provides oversight to your PMO and holds it accountable. This will help you gather and leverage lessons learned, enhance processes and focus on continuous improvements that evolve over time. Ensure That the Right Skills Are in Place A knowledgeable QA partner will help build a project team with the appropriate skills, comprised of internal resources or contract technical professionals. A Collaborative Approach QA is no longer confined to a phase guarded by rigid gating rules, and its effectiveness is no longer solely measured by the number of defects it finds. In an Agile environment, quality is a team responsibility. An Agile partner must facilitate a proactive, creative and open project environment that fosters new ideas and collaboration between business stakeholders and the QA team, so QA s ultimate value defect prevention is witnessed in action. An Agile partner must be willing to take on any task to help the team deliver the best possible product to clients. In order to work effectively with business managers and software developers through all iterations, Agile testers need to understand design principles, help the business refine user stories, assist developers in testing the backend logic, perform manual exploratory testing, participate in desk checks and provide feedback to the team, automate test cases and, most importantly, speak up when problems occur. TEKsystems.com 4
Proven Quality Outcomes: Real World Examples HMSA needed a partner to assist with identification of future QA projects, requirements and resources for initiatives. It sought a partner that could support its largest effort, a HIPAA 5010 project that required substantial changes to the claims process, as well as the associated data and applications. TEKsystems presented a 30-day work analysis, identifying upcoming projects and the QA and testing requirements needed for each initiative. Our QA & Testing practice provided a managed service solution to assist HMSA with prioritizing testing needs, optimizing testing tools and templates, and continually improving the process. Apollo Group s QA department was tasked with reducing testing cycle times and maintenance costs while improving test coverage. The goal was to establish sustainable automation architecture to easily advance its automation initiative and eventually automate regression testing on more than 30 mission-critical applications. Via an assessment, TEKsystems determined that our BPT Accelerator Solution would provide Apollo with a smooth and efficient implementation of its existing BPT product suite, resulting in a faster return on investment. The BPT Accelerator was successfully installed, configured and implemented into Apollo s environment. This solution has facilitated a 75% reduction in implementation time due to our proprietary code libraries and a 30% cost savings from remote sourcing options provided during the planning and development phases. To improve its PeopleSoft capabilities, ETS embarked on a major initiative to add four new modules, including General Ledger, Projects, Contracts and Time and Labor, and upgrade its current development environment, PeopleTools, across the entire platform of 11 modules. TEKsystems testing efforts were successful through the implementation of a risk-based QA and testing approach and methodology to ensure adequate testing coverage, the development and implementation of an automation strategy and improved documentation to complement the functional testing efforts and the cross-training of the QA resources to extend their QA and testing capabilities beyond their initially targeted modules. TEKsystems QA and testing support enabled the client to accomplish its production release goal encompassing the new modules and business processes within the forecasted budget. Conclusion IT organizations are under ever-increasing pressure to deliver results in the form of software products that drive improvements to the bottom line, all while facing reductions in IT budgets. As IT leaders continue to search for new ways to automate and streamline processes, Agile will become the rule, not the exception, to managing QA projects. The Agile methodology represents a departure from traditional approaches and will require a shift in commitment and responsibilities among team members. Working with an experienced Agile partner can help clients adopt Agile principles, navigate challenges and realize the quality and flexibility associated with Agile practices. ABOUT US People are at the heart of every successful business initiative. At TEKsystems, we understand people. Every year we deploy over 80,000 IT professionals at 6,000 client sites across North America, Europe and Asia. Our deep insights into IT human capital management enable us to help our clients achieve their business goals while optimizing their IT workforce strategies. We provide IT staffing solutions, IT talent management expertise and IT services to help our clients plan, build and run their critical business initiatives. Through our range of quality-focused delivery models, we meet our clients where they are, and take them where they want to go, the way they want to get there. Written by: Akemi Micallef, Manager of Solutions Delivery TEKsystems.com 5