Agile Testing Strategy Emily Bache, SDC2010

Transcription

1 Agile Testing Strategy Emily Bache, SDC2010

2 Agile Testing Strategy why should I care? Timebox Frequent communication developers Working software testers Agile testing is more fun!

3 Talk Outline Traditional Testing strategy Agile teams include developers and testers Introducing Agile testing Quadrants Balancing the quadrants

4 Traditional testing strategy: the Wall Bug reports Code releases developers testers

5 The tester's point of view: a sock full of marbles? Idea attribution: Gerald Weinberger, Quality Software Management

6 Release to customer: how many red marbles left?

7 Which strategy wins??????

8 Traditional testing: it detects quality it does not determine it.

9 85% of defects introduced during coding

10 The longer you leave them, the harder they are to fix

11 How do you find defects? End users report them after release Testing Technical Reviews Requirements walkthroughs Code inspections

12 With agile, Testers are on the same team as devs developers testers

13 With agile, Testers are on the same team as devs. The team delivers working, tested software every iteration Timebox Frequent communication developers testers Working software

14 Healthy Task Board To Do Underway Done

15 Developers are blind to bugs in their own code Developers don t want to break their beautiful creation Subconciously use their knowledge of the implementation to avoid weak spots circa 1940, Paris, France - Blindfolded Typing Competition. by Jhaynee

16 Tests have different purposes: critique the product support developers

17 A smart mouse Hey guys, I captured the mouse! by darwin Bell

18 Tests have different focus Technology facing Business facing

19 Agile Testing Quadrants Q2 Q3 Q1 Q4 Technology Facing Original Idea by Brian Marick Critique the Product Supporting the team Business Facing

20 Agile Testing Quadrants - examples Automated Functional tests Manual Exploratory testing, Usability testing Q2 Q3 Q1 Q4 Automated Unit tests From Agile Testing by Crispin/Gregory Performance tests, Security tests Technology Facing Critique Product Supporting the team Business Facing

21 Q1: support the team, tech facing Q2 Q3 Q1 Q4 Technology Facing Critique Product Supporting the team Business Facing Automated tests using internal interfaces, classes, methods Unit & integration tests Closely linked with design/architecture work Must execute quickly

22 Q1 testing strategy Developers use TDD xunit (same language as production code) Measure progress in automated build

23 Q1: how to encourage them on your project Step 0 version control, automated build, CI. Coding dojo Programming coach who pairs with devs to teach TDD Design skills: dependency injection etc.

24 Q4: critique the product, tech facing Q2 Q3 Q1 Q4 Technology Facing Critique Product Supporting the team Business Facing Performance, security, reliability etc Usually need tool support, and often dedicated hardware Affected by architecture and deployment choices

25 Q4 testing strategy Architectural spikes to compare performance CI build runs selected timed tests Run last iteration release without restart, measure response time and memory usage. It depends on your product!

26 Q4: how to encourage them in your project Get extra set of hardware similar to production environment Find tools to support testing, or build some.

27 Q2: support the team, business facing Q2 Q3 Q1 Q4 Technology Facing Critique Product Supporting the team Business Facing Automated tests using external interfaces Closely linked to requirements Readable by business people

28 Agile Requirements User Story + Conversation + Example/ Coaching Test = Requirement From Agile Testing, Crispin, Gregory

29 Q2 testing strategy Develop custom test harness Testers write happy path/coaching tests before coding starts Add more tests as code grows Collaborate, pair program developers testers

30 Good tests should have: Readability Robustness Test Case Speed Coverage

31 Building a test harness: it's a coding project fitnesse cucumber robot texttest How easy to add a new test case? How easy to create new sort of test case? How easy to debug a failing test? How easy to refactor duplication between tests?

32 Q2: how to encourage them on your project Create developer in test role Mini project to identify suitable tools Talk to business people Make test failures visible to all devs

33 Q3: critique product, business facing Q2 Q3 Q1 Q4 Technology Facing Critique Product Supporting the team Business Facing Manual exploratory testing Usability testing Testing skills and domain knowledge crucial

34 What is exploratory testing? Cem Kaner invented the term in his book Testing Computer Software simultaneous learning, test design and test execution

35 Q3 testing strategy testers product owner salespeople support technicians Tester role on team does exploratory testing 10% of their time, on latest version. Invite people in other roles with customer perspective to do exploratory testing at iteration end Consider using formal Session Based Exploratory Testing

36 Q3 tests feed back to stakeholders Frequent communication product owner testers If you find major problems, examine why they weren't found sooner in Q2 or Q1 tests. Q3 testing should inspire new User Stories

37 Q3: how to encourage them in your project Good testers have always done this identify if one person has found a disproportionate number of important bugs Form a study group, read the literature.

38 Agile Testing Quadrants Q2 Q3 Q1 Q4 Technology Facing Original Idea by Brian Marick Critique the Product Supporting the team Business Facing

39 As headlights are to a car at night, so is the testing effort to a software development project Lesson #1 in Lessons Learned in Software Testing, Kaner, Bach, Pettichord

40 Prioritizing different sorts of tests Q2 Q3 Q1 Q4 Technology Facing Critique Product Supporting the team Business Facing Q1, Q4 are basic Q2 enables agility Q3 delights end users

41 Skills for an Agile team member: Specializing Generalists Testers: Excellent at Testing Basic programming Domain knowledge Developers: Excellent at Programming Basic testing Domain knowledge Skills transfer developers testers

42 Summary Take down the wall Unit tests are not enough on their own. Cover the Agile Test Quadrants Q2 Q3 Q1 Q4 Technology Facing Critique Product Supporting the team Business Facing

43 Questions? Agile Testing Lisa Crispin & Janet Gregory Lessons Learned In Software Testing Kaner, Bach, Pettichord. Image credits: iconspedia.com: Asher, DaPino Webdesign, Zeus Box, Mazenl77, Harwen, Flahorn, eponas-deeway