Shift Left: Test Automation in Agile Environments Unlock the Secrets WWW.GOTOSCA.COM Copyright by TRICENTIS, all rights reserved 1
Shift Left - Test Automation Agile Environments AGENDA Today we will Cover Challenges around Agile Testing Shift-Left Specification by Example The Agile test-pyramide What are the benefits of Shift-Left & early automation? Agile customers - case studies How do developers need to contribute? Summary Q&A Copyright by TRICENTIS, all rights reserved 2
Agile Methods are on the Rise More than 2/3 of new software developments are now being set up with an agile method. here they want to get to Agile here is where they are today Waterfall Copyright by TRICENTIS, all rights reserved 3
Costs and experienced value conventional Costs increase constantly (often linearly) over the duration of the project. The customer will not see the expected value until much later on. Copyright by TRICENTIS, all rights reserved 4
Costs and experienced value conventional (II) And what now? Project aborted!! Project aborted due to shortage of resources or the project scope needs to be changed significantly Copyright by TRICENTIS, all rights reserved 5
Costs and Experienced Value of Agile (Scrum) 120% 100% 80% Costs Kosten Value Wert 75% 85% 90% 70% 95% 80% 98% 90% 100% 60% 55% 50% 60% 40% 40% 30% 20% 0% 20% 15% 10% 5% 1 2 3 4 5 6 7 8 9 10 Short sprints generate deliverable versions early on. The most important thing first strict value/risk-oriented procedure Copyright by TRICENTIS, all rights reserved 6
TOSCA Waterfall-View Copyright by TRICENTIS, all rights reserved 7
Agile Development Shift from the should to the must Shift to the left Specification by Example Early test automation Copyright by TRICENTIS, all rights reserved 8
TOSCA Agile Testing BDD Behaviour Copyright by TRICENTIS, all rights reserved 9
Shift to the Left! Specification by Example Copyright by TRICENTIS, all rights reserved 10
When we were children we drew Copyright by TRICENTIS, all rights reserved 11
V-Model Requirement definition Acceptance testing (UAT) Specification (System) integration testing Techn. design Unit/Module testing Implementation Copyright by TRICENTIS, all rights reserved 12
From the waterfall to the agile method The size of the birds changes! Birds remain birds! Many small birds are considerably more maneuverable (more flexible) than large ones. Each of them can fly on its own! Copyright by TRICENTIS, all rights reserved 13
Specification by Example - Key to Success Requirement definition Specification by Example Acceptance testing (UAT) Specification (System) integration testing Techn. design Unit/Module testing Implementation t Copyright by TRICENTIS, all rights reserved 14
Specification by Example with Test Case Design Requirements substantiated checked Test Case Design specified Test Case Design is sufficiently substantiated flexible if changes occur lean and very cost-effective (compared with the specification of test cases) Copyright by TRICENTIS, all rights reserved 15
Shift to the Left! Early Test Automation Copyright by TRICENTIS, all rights reserved 16
Test Pyramid conventional testing procedure Practice: main focus of efforts is on the E2E test! Poor cost efficiency Limited test coverage Availability of all systems in the test? Consistent test data in all systems? Copyright by TRICENTIS, all rights reserved 17
Objective: reducing the number of E2E tests Slow and Brittle: Replacing End-to-End Testing Our goal for this workshop is to generate serious, practical alternatives to end-to-end testing Copyright by TRICENTIS, all rights reserved 18
Test Pyramid Target State Powerful, automated Unit und Acceptance Tests (system tests) Test access: non-ui Tests via GUI only for: Testing the display on the screen Testing the connection to the underlying layers User Acceptance Testing Copyright by TRICENTIS, all rights reserved 19
E2E System internal und Crossover Tests System 1 System n Systeminternal Tests E2E Tests Copyright by TRICENTIS, all rights reserved 20
Testing up to the Interfaces System 1 System n Systeminternal Tests E2E Tests System-internal tests extended right to the interfaces Coupled systems are stubbed or mocked Share of functionality that can be tested without coupled systems (= system-internally) is sufficiently extended Copyright by TRICENTIS, all rights reserved 21
Testsequence UI and nonui - Example Dialog/Data SRec/Data Example: Create Customer, Customer Project TRICENTIS, via WebService 2 3 # of test steps: Save 75%! Maintenance: Save 75%! Test duration: Save 90%! Example: Create Customer, Customer Project TRICENTIS, via Browser-UI 13 3 Copyright by TRICENTIS, all rights reserved 22
Test streams & Test automation traditional Functionality to be tested Releasetest Regressiontest R R R 1 2 3 One-time (manually) Test of new functionality with regressive error potential (manually) Regressiontest (automated) Releases t Copyright by TRICENTIS, all rights reserved 23
Test streams and Automation agile Functionality to be tested Regressiontest automated: UI High automation degree from the start Dominated by nonui access nonui S 1 S 2 S 3 Releases/Sprints t One-time (manually) Test of new functionality with regressive error potential (manually) Regressiontest (automated) Copyright by TRICENTIS, all rights reserved 24
Testing Challenges: o Move towards Agile software development o Cost and scheduling pressure o All previous attempts at automation have failed Solution: Model-based test automation. Results: o Core regression suite implemented in less than 3 months 1,009 test cases 33,797 test actions GBST Shares Settles close to 50% of all trades on the Australian Stock Market, ~ AU$ 2.5 billion every day 35 end-to-end business process scenarios, multiple day scenarios 2,511 verification points o Execution effort down by 80% (from 535 h to less than 100 h). o Annual cost reduction exceeds 50%. Copyright by TRICENTIS, all rights reserved 25
German Payback System vendor, hosting payback systems for Lufthansa, Deutsche Bahn, etc. Testing Challenges: o Agile development with continuous integration o Very short development cycles o Multiple interfaces (Browser, Smartphone, rich client) Solution: Development provides test-accessible nonui interface (WebServices, REST) for every single use-case! enabling model-based test automation, early nonui test automation Copyright by TRICENTIS, all rights reserved 26
How Developers need to support Agile testing requires early automated nonui testing. Early automation won t work via UIs! Copyright by TRICENTIS, all rights reserved 27
How Developers need to support Agile developers need to provide appropriate hooks: Comprehensive, public interface for each use-case allowing to fully test business logic avoiding noisy communication... being maintained! Copyright by TRICENTIS, all rights reserved 28
Summary Agile Testing: Volatile Backlog, Short Sprints, Value Management and Autonomous Teams Quality remains the Number 1 Concern Quality results from testing the right things and improving test coverage and is held back by the lack of testing efficiency Shift Left: establish specification by example as mandatory part of your user stories Shift Left: apply testing literally from Day 1 of the project and nonui test automation is the KEY Agile developers need to provide and maintain nonui interfaces allowing for gapless testing of business logic. Copyright by TRICENTIS, all rights reserved 29
WWW.GOTOSCA.COM Thank You! Copyright by TRICENTIS, all rights reserved 30