Shift Left: Test Automation in Agile Environments Unlock the Secrets

Size: px

Start display at page:

Download "Shift Left: Test Automation in Agile Environments Unlock the Secrets"

April Atkins
9 years ago
Views:

2 Shift Left - Test Automation Agile Environments AGENDA Today we will Cover Challenges around Agile Testing Shift-Left Specification by Example The Agile test-pyramide What are the benefits of Shift-Left & early automation? Agile customers - case studies How do developers need to contribute? Summary Q&A Copyright by TRICENTIS, all rights reserved 2

are the benefits of Shift-Left & early automation?

3 Agile Methods are on the Rise More than 2/3 of new software developments are now being set up with an agile method. here they want to get to Agile here is where they are today Waterfall Copyright by TRICENTIS, all rights reserved 3

4 Costs and experienced value conventional Costs increase constantly (often linearly) over the duration of the project. The customer will not see the expected value until much later on. Copyright by TRICENTIS, all rights reserved 4

5 Costs and experienced value conventional (II) And what now? Project aborted!! Project aborted due to shortage of resources or the project scope needs to be changed significantly Copyright by TRICENTIS, all rights reserved 5

6 Costs and Experienced Value of Agile (Scrum) 120% 100% 80% Costs Kosten Value Wert 75% 85% 90% 70% 95% 80% 98% 90% 100% 60% 55% 50% 60% 40% 40% 30% 20% 0% 20% 15% 10% 5% Short sprints generate deliverable versions early on. The most important thing first strict value/risk-oriented procedure Copyright by TRICENTIS, all rights reserved 6

4 5 6 7 8 9 10 Short sprints generate deliverable versions early on.

13 From the waterfall to the agile method The size of the birds changes! Birds remain birds! Many small birds are considerably more maneuverable (more flexible) than large ones. Each of them can fly on its own! Copyright by TRICENTIS, all rights reserved 13

Many small birds are considerably more maneuverable (more

14 Specification by Example - Key to Success Requirement definition Specification by Example Acceptance testing (UAT) Specification (System) integration testing Techn. design Unit/Module testing Implementation t Copyright by TRICENTIS, all rights reserved 14

Specification (System) integration testing Techn.

15 Specification by Example with Test Case Design Requirements substantiated checked Test Case Design specified Test Case Design is sufficiently substantiated flexible if changes occur lean and very cost-effective (compared with the specification of test cases) Copyright by TRICENTIS, all rights reserved 15

substantiated flexible if changes occur lean and very cost-effective

17 Test Pyramid conventional testing procedure Practice: main focus of efforts is on the E2E test! Poor cost efficiency Limited test coverage Availability of all systems in the test? Consistent test data in all systems? Copyright by TRICENTIS, all rights reserved 17

Poor cost efficiency Limited test coverage Availability of all

18 Objective: reducing the number of E2E tests Slow and Brittle: Replacing End-to-End Testing Our goal for this workshop is to generate serious, practical alternatives to end-to-end testing Copyright by TRICENTIS, all rights reserved 18

workshop is to generate serious, practical alternatives to

19 Test Pyramid Target State Powerful, automated Unit und Acceptance Tests (system tests) Test access: non-ui Tests via GUI only for: Testing the display on the screen Testing the connection to the underlying layers User Acceptance Testing Copyright by TRICENTIS, all rights reserved 19

Testing the display on the screen Testing the connection to the

21 Testing up to the Interfaces System 1 System n Systeminternal Tests E2E Tests System-internal tests extended right to the interfaces Coupled systems are stubbed or mocked Share of functionality that can be tested without coupled systems (= system-internally) is sufficiently extended Copyright by TRICENTIS, all rights reserved 21

22 Testsequence UI and nonui - Example Dialog/Data SRec/Data Example: Create Customer, Customer Project TRICENTIS, via WebService 2 3 # of test steps: Save 75%! Maintenance: Save 75%! Test duration: Save 90%! Example: Create Customer, Customer Project TRICENTIS, via Browser-UI 13 3 Copyright by TRICENTIS, all rights reserved 22

23 Test streams & Test automation traditional Functionality to be tested Releasetest Regressiontest R R R One-time (manually) Test of new functionality with regressive error potential (manually) Regressiontest (automated) Releases t Copyright by TRICENTIS, all rights reserved 23

24 Test streams and Automation agile Functionality to be tested Regressiontest automated: UI High automation degree from the start Dominated by nonui access nonui S 1 S 2 S 3 Releases/Sprints t One-time (manually) Test of new functionality with regressive error potential (manually) Regressiontest (automated) Copyright by TRICENTIS, all rights reserved 24

25 Testing Challenges: o Move towards Agile software development o Cost and scheduling pressure o All previous attempts at automation have failed Solution: Model-based test automation. Results: o Core regression suite implemented in less than 3 months 1,009 test cases 33,797 test actions GBST Shares Settles close to 50% of all trades on the Australian Stock Market, ~ AU$ 2.5 billion every day 35 end-to-end business process scenarios, multiple day scenarios 2,511 verification points o Execution effort down by 80% (from 535 h to less than 100 h). o Annual cost reduction exceeds 50%. Copyright by TRICENTIS, all rights reserved 25

26 German Payback System vendor, hosting payback systems for Lufthansa, Deutsche Bahn, etc. Testing Challenges: o Agile development with continuous integration o Very short development cycles o Multiple interfaces (Browser, Smartphone, rich client) Solution: Development provides test-accessible nonui interface (WebServices, REST) for every single use-case! enabling model-based test automation, early nonui test automation Copyright by TRICENTIS, all rights reserved 26

28 How Developers need to support Agile developers need to provide appropriate hooks: Comprehensive, public interface for each use-case allowing to fully test business logic avoiding noisy communication... being maintained! Copyright by TRICENTIS, all rights reserved 28

29 Summary Agile Testing: Volatile Backlog, Short Sprints, Value Management and Autonomous Teams Quality remains the Number 1 Concern Quality results from testing the right things and improving test coverage and is held back by the lack of testing efficiency Shift Left: establish specification by example as mandatory part of your user stories Shift Left: apply testing literally from Day 1 of the project and nonui test automation is the KEY Agile developers need to provide and maintain nonui interfaces allowing for gapless testing of business logic. Copyright by TRICENTIS, all rights reserved 29

LEAN AGILE POCKET GUIDE

SATORI CONSULTING LEAN AGILE POCKET GUIDE Software Product Development Methodology Reference Guide PURPOSE This pocket guide serves as a reference to a family of lean agile software development methodologies