Scrum QA Assessment. John Scarborough VP System Engineering STeP-IN Summit January 2006

Transcription

1 Scrum QA Assessment John Scarborough VP System Engineering STeP-IN Summit January

2 The origins of Agile Agile methodologies were created to provide an alternative means for software product development that would avoid plan-driven methods long development cycles, high expense, and sluggish response to changing requirements. Aztec Software 2

3 The Agile Manifesto Agile Individuals & interactions Working software Customer collaboration Responding to change rather than rather than rather than rather than Plan-driven Processes and tools Comprehensive documentation Contract negotiation Following a plan Aztec Software 3

4 Scrum on the world stage In November, Microsoft not a company one thinks of when referencing implementations of Scrum or other Agile implementations -- publicly announced that it is encouraging its development groups to use Scrum project management. Aztec Software 4

5 If Scrum is so great, how come To listen to Agilistas, you d think it would work right out of the box. But no one gets Scrum right the first time they try it. Aztec Software 5

6 Orthodox and reformed scrum Anyone who improved their implementation of Scrum did it by improving processes, because Scrum is a collection of processes. They may have done so consciously or unconsciously. Consciously is better. Aztec Software 6

7 Agile Methodology: A set of processes Develop stories Plan features Plan iteration Write unit tests Develop code Run unit tests Check in code Acceptance test Release Aztec Software 7

8 Improving processes Before you can improve a process you must first understand it. Study its structure Analyze its internal and context dynamics Establish a baseline for change Aztec Software 8

9 Improving processes in the plan-driven environment Get management commitment Appraise process capability level Establish a baseline for measuring process improvement Plan process improvement according to best practices Periodically appraise improvement Aztec Software 9

10 Improving processes in the Scrum environment Get team commitment Appraise process capability level Establish a baseline for measuring process improvement Plan process improvement according to best practices Periodically appraise improvement Aztec Software 10

11 Assessment must be consistent with the spirit of Agile SW-CMM tells what to do in general terms, but does not say how to do it. The implementation of [CMM] methodologies must be aligned with the spirit of the agile philosophy and with the needs and interests of the customer and other stakeholders. --Mark Paulk, co-founder of CMM Aztec Software 11

12 Who needs assessment? Small is beautiful but it still needs improvement. Kickoff Iteration 1 The process for one release Iteration 2 Iteration 3 Iteration 4 Validation W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16 Demo 1 α release Scope Freeze Demo 2 Code Freeze β release Aztec Software 12

13 Enable continuous improvement within one sprint Group member evaluation (what are our strengths? weaknesses?) Do we need additional resources? What tools will optimize our work? What statistical measures and controls will increase our desire to succeed? Should we engage external teams for work that is not tightly coupled to the sprint backlog? Aztec Software 13

14 Previous Release Cycle Iteration 4 Certification W10 W11 W12 W13 W14 W15 W16 Can Scrum scale? The argument for discipline Demo 2 Code Freeze General Design & Planning for the Next Release Current Release Cycle Iteration 1 Iteration 2 Iteration 3 Iteration 4 Certification W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16 Kickoff Demo 1 Scope Freeze Demo 2 Code Freeze General Design & Planning for the Next Release Next Release Cycle Iteration 1 Iteration 2 W1 W2 W3 W4 W5 W6 Kickoff Demo 1 Aztec Software 14

15 Enable continuous improvement from sprint to sprint Elapsed time, usecase to story Testability of requirements (stories) % of code for which there are unit tests # Sev 1 bugs per line of checked code # features punted to next release Conformity to agreed-upon protocols Aztec Software 15

16 An example of Scrum QA Assessment Se r W t Criteri a Low (1) Med (3) High (5) CC R Every test case has been reviewed for completeness Test cases are never reviewed for completeness Test cases are irregularly reviewed for completeness All test cases are reviewed for completeness 4 2 Criteria specific to the project are each assigned a range of scores for level of realization. A capability index is derived through comparing the sum of scores to the total possible. Aztec Software 16

17 Profiling by group, by process, by component Rating Group A Group B Group C Group D Aztec Software 17

18 Formal advantages of Scrum QA Assessment Provides staged improvement: a road map that an organization can follow, step by sequenced step, toward its chosen goal. Provides continuous improvement: emphasis on growth and improvement in individual process areas Provides assessment specificity: criteria are based on specific business objectives driving the Scrum Aztec Software 18

19 Scrum QA Assessment is dynamic Scrum QA Assessment s specificity and the integration of continuous and staged improvement enable the Scrum Master to update multiple assessments as each process improvement goal is achieved. The effect of a single process update is immediately registered for: Group cumulative score Process area score Specific process score Aztec Software 19

20 Scrum QA Assessment Aids Self-Organization It is not necessary to bring in outside experts on a periodic basis; the expert is very local it may be a test manager coordinating with Scrum Masters in his division, or it may be a Scrum Master assessing his or her own group. Therefore it is consistent with Scrum s insistence on self-management and self-organization. Aztec Software 20

21 CONCLUSION Scrum QA Assessment is a powerful tool for Scrum Masters The Scrum Master and test manager can use Scrum QA Assessment to understand the team s or organization s current state strengths and weaknesses, which will provide the data required to make crucial decisions in sourcing, provisioning, and scheduling. Aztec Software 21

22 THANK YOU 22