test automation for quality enablement

Transcription

1 whitepaper test automation for quality enablement improving software test automation for visual studio projects March 2013

2 TABLE OF CONTENTS Executive Summary...1 Background...2 Test Organization...3 Testing Methods...3 Improved Testing in Visual Studio...4 Summary...5 About LogiGear...5

3 Executive Summary IT organizations are under constant pressure to deliver high quality software at an accelerated cadence. Rapid development processes that achieve shortened development cycles carry both challenges and risks that must be managed throughout the application lifecycle. Testing is essential for determining software quality, managing risk, and providing the information necessary to make informed decisions about whether or not software is ready to be placed into operation. The application lifecycle management (ALM) functionality of Visual Studio 2012 supports rapid development environments, but a higher level of test automation than the native coded UI testing tool can support is necessary to keep up with testing demands. Adopting modern software testing methods, and tools that support those methods, will enable teams using Visual Studio s integrated development environment (IDE) to provide the level of testing necessary for the delivery of high quality software. 1

4 Background Development and project management tools have a long history of use in IDEs, however, test automation tools were slow to be incorporated into those environments. Visual Studio 2010 included a native coded UI test tool to automate manual UI tests. The tool proved useful for developer unit testing and adequate for functional testing in traditional development environments where test-afterdevelopment was the norm. Current practices for coded UI tests rely primarily on generating tests by recording the step-through sequence of the application under test.¹ The record and playback method requires that the application under test is well developed before test creation begins. This test-after-development approach introduces multiple business challenges and risks primarily in the areas of: Compromised Test Coverage Test-after-development can lead to compromised functional quality in order to achieve good enough performance. This is rarely a viable option, as users are intolerant of software applications that are perceived as buggy or poorly performing. Increasing Testing Resources Increasing testing resources to keep up with shortened development cycles is a stop-gap solution that adds development costs and cannot scale to keep up with improved development practices. Business Challenge Solution Business Risk Balancing velocity and quality Shipping minimally-viable products Customer dissatisfaction Changing requirements and priorities Acceptance test-driven development Long test cycles, missed release dates Meeting user requirements and value expectations Developing and delivering fully functional software applications Lowering operation cycle times and costs Continuous testing and feedback Defining and continuously test operations acceptance criteria Increasing functional test automation Increased testing cost Increased mean-time-to-repair (MTTR) Wrong approach; unable to scale Rapid development and deployment environments are driving the adoption of Agile development philosophies. Companies see Agile as a way to improve productivity and better manage changing customer priorities.² Market and competitive pressure to constantly reduce the number of iteration cycles creates challenges that can be addressed by improving the approach to testing. ¹ Best Practices for Coded UI Tests, ² The Sixth Annual State of Agile Development Survey, VersionOne

5 test organization The common approach has been to do test-after-development, with the production team handing code off to test teams to write and conduct tests. However, rapid development and deployment environments demand that test design and development begin well before software production is complete. This requires a higher level of collaboration than can be achieved with segmented teams. Traditional silo structures must be replaced with cross-functional teams made up of business sponsors, developers, and individuals with testing acumen who all have access to the same collaborative tools and reports. Knowledge sharing across teams allows test creation to start much earlier in the development process and enables more tests to be automated in a shorter period of time. Increasing automation gives teams more time to focus on gray-box testing and increases the frequency of test execution even in the same development sprint. Figure 1. Quality enablement emphasizes full lifecycle practices³ testing methods Test design and test organization are among the greatest enabling factors for test automation success much more than the actual automation technology. The Action Based Testing (ABT) methodology is modeled on object-oriented development, with proven value in both waterfall and Agile development environments.⁴ ³ Quality Enablement with Microsoft Visual Studio 2012", Microsoft Corporation, 2012, p.6 ⁴ Action Based Testing: The Solution for Agile Test Automation, Hans Buwalda, /1020-action-based-testing-the-solution-for-agile-test-automation.html 3

6 ABT recognizes three product lifecycles that are planned and managed within a project framework: System Development - follows the traditional or Agile software development lifecycle model Test Development - includes test design, test execution, test result follow-up, and test maintenance Automation - focuses solely on the action keywords, interpreting action definitions, and matching application interfaces The ABT approach enables test automation development to begin as soon as the functional requirements of the application have been identified. Test design and organization are facilitated through the creation of test modules. Each test module contains a clear set of test objectives and inter-related test cases. Test cases are created using keywords to describe the functional actions as opposed to scripting or coding. Keyword tests result in a readable sequence of actions that make the tests self-documenting and provide an auditable record of test execution. Most applications use common functions, for example a login screen that requires entering a user name and password. The actions for common functions can be assembled in a library and used repeatedly to create new test cases. The UI and programming details that are subject to frequent changes during iterative development are contained in the actions. When a change in the system under test occurs, only the impacted actions need to be updated as opposed to rewriting or creating new tests cases. As a result, there are fewer actions than test cases, making the automation effort faster and highly maintainable. Improved testing in visual studio The increased functionality introduced in Visual Studio 2012 calls for a high level of test automation. TestArchitect for Visual Studio is a test automation extension designed specifically to augment quality enablement in the Visual Studio 2012 development environment. TestArchitect employs the ABT methodology and a keyword-based test authoring platform with more than 280 reusable, preprogrammed actions that make it possible to create tests in parallel with software development and easily update tests as software application requirements change. The TestArchitect solution resides in the same IDE environment as Visual Studio and other solution-oriented ALM components, such as Team Foundation Server (TFS) and Microsoft Test Manager (MTM). This provides developers with the flexibility to create manual unit tests with the familiar coded UI solution and reuse those tests in TestArchitect for functional and regression test automation. 4

7 Summary There is constant pressure to shorten software delivery cycles. Testing must keep pace with development across the entire application lifecycle in order to assure software quality. Adopting modern software practices, test methods and tools make it possible to synchronize development and testing in order to deliver high quality software. Organizations invested in the Visual Studio IDE will find that TestArchitect for Visual Studio addresses the limitations of the native test environment and makes test automation and quality enablement long-term maintainable. about logigear LogiGear Corporation provides global solutions for software testing and offers software testing training programs worldwide. We are a leader in the integration of test automation, offshore testing resources and US-based project management for fast, cost effective results. LogiGear Headquarters 2015 Pioneer Ct. San Mateo, CA Phone: (800) Fax: (650) sales@ testarchitect.com Founded in 1994 by top thought-leaders in software quality, LogiGear now serves clients around the world, ranging from the Fortune 100 to early-stage startups in a wide range of industries. Be it consulting, training, high-volume and cross-platform test automation, outsourced testing or automation technology, we partner with organizations to create customized approaches that precisely meet their needs. For more information about TestArchitect for Visual Studio and the Action Based Testing methodology, visit 5