Scrum and Testing The end of the test role Bryan Bakker 20 maart 2012

Transcription

1 Scrum and Testing The end of the test role Bryan Bakker 20 maart 2012 voordracht georganiseerd door Discussiegroep Software Testing met de steun van Ingenieurshuis, Antwerpen

2 Scrum and Testing... The end of the test role? Bryan Bakker ie-net March 2012 Contents Scrum Scrum and Testing More than only software Outsourcing Note: not everything I tell is part of scrum Sioux 2012 Confidential 2 1

3 About Bryan Bakker Test Architect Certifications: ISTQB, TMap, Prince2 Member of ISTQB Expert Level on Test Automation Tutor of several test related courses Domains: medical systems, professional security systems, semi-industry, electron microscopy Specialties: test automation, integration testing, design for testability, reliability testing Sioux 2012 Confidential 3 Scrum experiences Within Sioux almost all projects developed with Scrum Started on Scrum project in feb 2010 I am a software test expert Not a Scrum Expert Not a Scrum Master Sioux 2012 Confidential 4 2

4 About Sioux UTRECHT MOSCOW EINDHOVEN HERENTALS NEDERWEERT Sioux 2012 Confidential 5 Scrum Sioux 2012 Confidential 6 3

5 Well known picture Sioux 2012 Confidential 7 Wishes and facts Wishes: Customer knows what he wants Developers know how to implement it Nothing will change along the way Facts: Customer discovers what he wants Developers discover how to implement it Most things change along the way Solution: Agile development (e.g. Scrum) OK... that s a wish, and not a fact Sioux 2012 Confidential 8 4

6 Manifesto We have come to value: Individuals id and over Processes and interactions tools Working software over Comprehensive documentation Customer collaboration over Contract negotiation Responding to change over Following a plan Source: An introduction to Scrum by Mike Cohn. Sioux 2012 Confidential 9 Manifesto We have come to value: Individuals id and over Processes and interactions tools Less processes more communication More ad-hoc, problem solving Less documented processes Do not document everything in Quality Systems Manual Tools... Too complex, too expensive Not that extreme! Sioux 2012 Confidential 10 5

7 Manifesto We have come to value: Individuals id and over Processes and interactions tools Processes still useful Should help to solve a problem Quality Systems Manual can still be needed Standards, safety, evidence needed Tools needed! A lot open-source is seen Choose simple tools, that fulfil the job Source: An introduction to Scrum by Mike Cohn. Sioux 2012 Confidential 11 Manifesto We have come to value: Working software over Comprehensive documentation Customer wants working software not documentation Documentation can be in the code Spend time on software instead Not that extreme! Sioux 2012 Confidential 12 6

8 Manifesto We have come to value: Working software over Comprehensive documentation Documentation still needed, but should be useful Documentation can be mandatory (standards, safety, regulatory) Traceability, evidence (don t forget maintenance) But documentation alone is not that valued... Quality Assurance (Compliance Verification) can ensure that correct set of documentation is delivered Sioux 2012 Confidential 13 Manifesto We have come to value: Customer over Contract t collaboration negotiation Product owner is continuously involved Product owner can control the scope and priorities Do not try capture everything in a contract, this is going g to change anyway Work together Sioux 2012 Confidential 14 7

9 Manifesto We have come to value: Responding to change over Following a plan Plan is nice, but plan will change Accept the change, and embrace it Change is an opportunity (to build the right product) Sioux 2012 Confidential 15 Scrum characteristics One of the agile processes Self-organizing g teams Product progresses in a series of sprints Requirements are captured as items in a list of product backlog No specific engineering practices prescribed Incremental, iterative Sioux 2012 Confidential 16 8

10 Scrum characteristics Features are developed in order of priority Pragmatic Increments result in shippable products Daily scrum meetings to share status Time-boxed, Q over F Quick feedback from customer Customer has control on project Sioux 2012 Confidential 17 Scrum overview Daily Scrum Meeting 24 hours Sprint Backlog Backlog tasks expanded by team 2-4 weeks Product Backlog As prioritized by Product Owner Potentially Shippable Product Increment Source: Adapted from Agile Software Development with Scrum by Ken Schwaber and Mike Beedle. Sioux 2012 Confidential 18 9

11 Scrum Roles Product Owner Voice of the customer Owner of product backlog (incl. prioriorities) Decides where the team should go Not how, not how fast Scrum Master Focuses on scrum practices Removes impediments -> make sure team can work Team 5-8 persons, sit together Self organizing Owns sprint backlog Sioux 2012 Confidential 19 Scrum Artifacts Product backlog High level features of project Items at top should be clear Dynamic Sprint backlog Contains work for sprint Fixed for iteration (no changes in current sprint!) Quality more important than features Visible on scrum-board Burndown chart Shows amount of work finished in sprint Gets constant over time within team Sioux 2012 Confidential 20 10

12 Burndown chart burndown chart estimated hours estimated work remaining working hours sept 11 sept 12 sept 13 sept 14 sept 17 sept 18 sept 19 sept 20 sept 21 sept 24 sept 25 sept 26 sept 27 sept 28 sept days left in sprint Sioux 2012 Confidential 21 Scrum meetings Sprint contents meeting Which items from product backlog are added to sprint backlog Product owner decides Daily scrum Stand-up meeting Product owner not always present, but should be available Sprint review Demo Retrospective Sioux 2012 Confidential 22 11

13 Why Scrum? Flexibility for business: After each sprint: product could be shipped features can be added/removed on product backlog and priorities can be changed. Creates focus sprint can be overseen only spend preparation time on things that will be done Sioux 2012 Confidential 23 Release flexibility Problems in old approach: Quality level increases slowly during project Long periods of unstable products What would you prefer: Getting a product soon and getting an update when it is available, or Wait a long time and get the final product Sioux 2012 Confidential 24 12

14 When to use Suitable when: time-to-market to market is short desired functionality still uncertain dynamic market situation Sioux 2012 Confidential 25 The case against Requires trust Too little upfront thinking: Danger of major redesign Wrong requirements Requires good team Sioux 2012 Confidential 26 13

15 Scrum and Testing Sioux 2012 Confidential 27 Test Driven Development Not part of Scrum, based on extreme Programming But used typically in Scrum projects 1. Create a new test 2. See the new test fail 3. Write code to make test pass 4. Build, run all tests, see all tests pass 5. Refactor (cleanup) Sioux 2012 Confidential 28 14

16 Test Driven Development Performed by software developer Tests are continuously executed Code is not delivered when tests fail Testing is part of the process Quality awareness increases Automation is a must! Sioux 2012 Confidential 29 Test Driven Development Add tests to regression test Regression test is run on each build Regression detected immediately Short feedback loop Safety net at re-designs Test Engineers should become Software Engineers or look for another (non-agile) assignment! Sioux 2012 Confidential 30 15

17 Scrum and testing TDD is important TDD can be enough in small teams In combination with continuous integration very strong But test engineers can still be very useful Tester Developer Add a tester to each scrum team Tester can define and execute test cases to verify integrated functionality Sioux 2012 Confidential 31 Agile Testing Quadrants Lisa Crispin, Janet Gregory Sioux 2012 Confidential 32 16

18 Quadrant 1 Technology facing tests that support the team Unit/component level TDD is used here Focus on internal quality Instant feedback Build testability into code Fully automated Sioux 2012 Confidential 33 Quadrant 2 Business facing tests that support the team Verify correctness of complete function Integration If necessary use stubs and mocks Focus on external quality Quick feedback (during sprint) Test engineers together with customers Manual and automated Sioux 2012 Confidential 34 17

19 Quadrant 3 Business facing tests that critique the product Focus on validation and acceptance Customer important Realistic environment/use Usability Quick feedback (end of sprint) Increases confidence Sioux 2012 Confidential 35 Quadrant 4 Technology facing tests that critique the product Non-functional tests E.g. Performance, reliability, security Stress- and load testing Special expertise needed Tool support needed Probably not in early sprints And not in each sprint Sioux 2012 Confidential 36 18

20 Role of Test Engineer Test Engineers should become Software Engineers or look for another (non-agile) assignment! In fact a Test Engineer can add a lot of value In the Scrum team Move Test Engineers from independent test teams into scrum teams Sioux 2012 Confidential 37 More than only software Sioux 2012 Confidential 38 19

21 Multiple disciplines Sioux 2012 Confidential 39 Multiple disciplines - Agile Whish Product Manager Still magic, but smaller magic Feature 1 Stories SW Reqs HW Reqs FW Reqs Potentially ti Shippable Product ME Reqs Feature 2 Sioux 2012 Confidential 40 20

22 Problems/challenges Hardware and Software developed concurrently Development machine not the target Used for testing (e.g. TDD) Can differ substantially Hardware running ahead of software Problems in hardware detected late and hopefully fixed in software Or, hardware available late in project Software cannot be tested on target hardware Sioux 2012 Confidential 41 Problems/challenges Hard to scrum hardware and mechanics But: There are more deliverables than the final hardware Documents (e.g. interface specifications) Prototypes Alternative hardware (competitor, old-version) Stubs and simulators (software alternative), test interfaces Remember: we want to reduce risk as soon as possible It might be worth to spend a few euro s on extra prototypes Although in the end, they are thrown away Sioux 2012 Confidential 42 21

23 Problems/challenges In some environments Hardware/Mechanical development rather predictable Choice can be not to disturb them with a new process Software can still benefit from Scrum Whole project can still benefit from Scrum Sioux 2012 Confidential 43 System level Anyway... System test activities must still be performed As soon as possible Targets needed (might be sparse) In sprint, but often not practical Take system test out of sprint, and let separate team look at this: Independent (system) test team Sioux 2012 Confidential 44 22

24 Independent Test Team? Scrum Team: System tests Integration tests Development & Unit tests System tests Integration tests Development & Unit tests System tests Integration tests Development & Unit tests Sprint n Sprint n+1 Release Release Sprint n+2 Release Scrum Team: Independent Test Team: Integration tests Development & Unit tests Internal Release Integration tests Development & Unit tests Internal Release System tests Sprint n Sprint n+1 Integration tests Development & Unit tests Internal Release Sprint n+2 Release Sioux 2012 Confidential 45 Multiple Scrum teams Sioux 2012 Confidential 46 23

25 Multiple Scrum teams Sprint Contents Sprint Team A Potentially Shippable Product Sprint Contents Sprint Team B Potentially Shippable Product Sprint Contents Sprint Team C Potentially Shippable Product Sioux 2012 Confidential 47 Multiple Scrum teams Sprint Contents t Sprint Team A Sprint Contents t Sprint Team A Sprint Contents Sprint Team B Potentially Shippable Product Sprint Contents Sprint Team B Potentially Shippable Product Sprint Contents Sprint Team C Sprint Contents Sprint Team C Integration and System Testing Integration and System Testing Sioux 2012 Confidential 48 24

26 Role of Test Engineer Move Test Engineers from independent test teams into scrum teams Independent test engineers still needed Probably less, so some can move to scrum teams Sioux 2012 Confidential 49 What about regulations? Satisfying standards, regulations each sprint? Probably not... Mandatory at the end of development (End Game) Think of interim releases Overhead can be large (e.g. FDA approval) But benefit can be high Customer feedback Problems with standards, safety, etc detected early early risk reduction Sioux 2012 Confidential 50 25

27 Evidence traceability? When formal process is needed to release a product Safety Industry standards Regulatory approvals (e.g. FDA) Overall quality characteristics Best practise: Can be done outside the Scrum teams By independent teams Test, regulatory, quality, (hardware) standards,... Link to the Scrum teams, but still independent Sioux 2012 Confidential 51 Scrum and CMMI Scrum focuses on people and interaction. CMMI focuses on processes. Scrum focuses on working software. CMMI focuses on documentation. CMMI and Scrum can be combined, but their natural tendencies are opposite. Sioux 2012 Confidential 52 26

28 Outsourcing Sioux 2012 Confidential 53 Common issues with outsourcing Difficult communication Cultural differences Limited domain and system knowledge Requirements must be clear and very detailed Time-zone difference Not under control Focus Sioux 2012 Confidential 54 27

29 Scrum and outsourcing? Some bullets from other sheets: Self-organizing teams Daily scrum meetings to share status Requirements are captured as items in a list of product backlog Quick feedback from customer Self-steering teams The whole team takes a shared responsibility on achieving the project goal (accountability and authority). desired functionality still uncertain Don t combine Scrum and outsourcing Sioux 2012 Confidential 55 Sioux and outsourcing Moscow Availability of skilled engineers European culture English speaking Minimal time zone difference (2 hours, but...) Frequent travelling possible Of course: lower hour rate 2 customers are using this service FrontOffice and BackOffice Sioux 2012 Confidential 56 28

30 V-Model Wish, need Release User Requirements Acceptance Test System Requirements FrontOffice System Test Eindhoven Architecture Integration Test BackOffice Design Component Test Moscow Implementation Sioux 2012 Confidential 57 FO BO responsibilities FO Requirements elicitation / discussions with customer Architecture (high level) Project management BO Architecture (low level) Design Implementation & Test (simulation) FO Integration and system test (targets) Reporting Sioux 2012 Confidential 58 29

31 Outsourcing and Scrum Sprint backlog defined by FO+BO and customer Sprints of 3 weeks Daily scrums Demo s given by FO to customer Retrospectives with the team FO present at customers site (regularly) Some extra tools needed: Skype, WebEx, electronic scrum-board Sioux 2012 Confidential 59 Electronic Scrum Board Sioux 2012 Confidential 60 30

32 Advantages Daily scrums overcome multiple problems: Communications Focus Control FO-BO construction overcomes: Domain and system knowledge Requirements Don t combine Scrum and outsourcing Sioux 2012 Confidential 61 Final thoughts Scrum is not the solution for all problems You still need good people Scrum is not new Focus on team work and quick feedback (Independent) Testing still necessary Scrum possible in (multi-disc.) product development Scrum helps when outsourcing is applied Sioux 2012 Confidential 62 31

33 Questions Sioux 2012 Confidential (0) Sioux 2012 Confidential 64 32

34 Consulteer onze website ie-net vzw Ingenieurhuis Desguinlei 214, B-2018 Antwerpen 1, Tel , Fax , Dexia ING BTW BE