AGILE PRODUCTIVITY METRICS

BIO PRESENTATION WK2 6/28/2006 10:00 AM AGILE PRODUCTIVITY METRICS Michael Mah QSM Associates, Inc. Better Software Conference June 26 29, 2006 Las Vegas, NV USA

Michael Mah Michael Mah is a contributing author of IT Measurement, Advice from the Experts and the upcoming book, Optimal Friction, People Dynamics at Work in the Information Age. Michael also publishes his writing on-line through the Cutter Consortium. His areas of expertise include organizational development, IT negotiation, software project estimation, productivity benchmarking, outsourcing, risk management, and project runway prevention. Michael has been a keynote and featured speaker for the Carnegie Mellon Software Engineering Process Group Conference, the Better Software Conference & EXPO, the Cutter Summit series, and numerous Project Management Institute and SEI SPIN chapter meetings.

Better Software Conference & Expo June 26-29, 2006 Agile Productivity Metrics: XP and Productivity Measures What the Numbers Say Michael Mah Managing Partner QSM Associates, Inc. 75 South Church Street Pittsfield, MA 01201 413-499-0988 Fax 413-447-7322 e-mail: michael.mah@qsma.com Website: www.qsma.com Blog: www.optimalfriction.com 4/19/2006 1

Manifesto for Agile Software Development We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools working software over comprehensive documentation Customer collaboration over contract negotiation responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. 2001 Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas, Martin Fowler 4/19/2006 2

Frothy eloquence neither convinces nor satisfies me. I am from Missouri. You have got to show me. - Missouri Congressman Willard Duncan Vandiver, 1899 4/19/2006 3

4/19/2006 4

Industry Data from the QSM SLIM-Metrics Database Spans 20+ years Large heterogeneous database contains over 7,000+ projects Represents over 685+ million SLOC, 7+ million function points, over 600 languages, from 500+ organizations in 18 countries Adding 200 400 projects/year 4/19/2006 5

4/19/2006 6

Agile Measurement Approach 1 - COLLECT AND VALIDATE PROJECT DATA 2 - ANALYZE PROJECTS USING QSM REFERENCE DATABASE Overview of Database Number of Projects vs Division Avg, Min, Max PI Grade vs Division Desktop Faxes 5.0 Desktop Faxes High End Systems 3.0 Division High End Systems Division Mobile Coms 3.0 Mobile Coms 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Number of Projects Average Value of Metrics 0 5 10 15 20 25 30 35 40 Number of Projects Life Cycle Effort vs ESLOC 100 New Percent SLOC Modified % SLOC Unmodified % SLOC 12 35 56 10 1 0.1 Life Effort (MHR) (thousands) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 Number of Projects 1 10 100 1000 ESLOC (thousands) 0.01 ALL Systems Fax Corp 1997base Avg. 4 - DOCUMENT RESULTS 3 - DETERMINE PROCESS METRICS & PROJECT POSITIONING 4/19/2006 7

We ve got to make our deadline or kill our kids, Sam told his team. 4/19/2006 8

4/19/2006 9

Industrial XP Environment 4/19/2006 10

Industrial XP Environment 4/19/2006 11

Industrial XP Environment Photo courtesy of Joshua Kerievsky 4/19/2006 12

Industrial XP Environment Photo courtesy of Joshua Kerievsky 4/19/2006 13

Traditional Release Whiteboard Sketch Traditional Release 1 4/19/2006 14

Digitized Replica Using SLIM Staffing & Probability Analysis R&D C&T 1 Avg Staff (people) <Single Goal - MBI 1.1946> 3 2 4 5 6 7 8 25 Milestones 0 - CSR 1 - SRR 2 - HLDR 3 - LLDR 4 - CUT 5 - IC 6 - STC 7 - UAT 8 - FCR 9-99R 10-99.9R 20 15 10 Avg Staff (people) 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep '01 '02 '03 0 SOLUTION PANEL <Single Goal - MBI 1.1946> Duration Effort Cost Peak Staff MTTD C&T 24.0 347 5893 19.5 0.3 Life Cycle 27.0 397 6741 19.5 0.3 Months PM $ (K) people Days Start Date 9/29/2001 7/1/2001 PI=17.0 MBI=1.2 Eff SLOC=376022 CONTROL PANEL <Single Goal - MBI 1.1946> 17.0 19.5 376 13.6 20.4 PI 15.6 23.4 Peak Staff 301 451 Eff SLOC (K) Project: Condor 4/19/2006 15

XP Releases Whiteboard Sketch Agile 1 Agile 3 Agile 2 4/19/2006 16

Digitized Replica Using SLIM tm Staffing & Probability Analysis R&D C&T Avg Staff (people) <XP Rel. 3.0> 1 3 2 4 5 6 78 10 Milestones 0 - CSR 1 - SRR 2 - HLDR 3 - LLDR 4 - CUT 5 - IC 6 - STC 7 - UAT 8 - FCR 9-99R 10-99.9R 8 6 4 2 Avg Staff (people) Apr '02 1 2 3 4 5 6 7 8 9 May Jun Jul Aug Sep Oct Nov Dec Jan Feb '03 Mar 0 Duration Effort Cost Peak Staff MTTD Start Date SOLUTION PANEL <XP Rel. 3.0> C&T Life Cycle 6.0 9.0 33 69 561 1173 8.4 8.4 1.8 1.8 8/31/2002 6/1/2002 PI=20.7 MBI=3.8 Eff SLOC=67023 Months PM $ (K) people Days 20.7 16.5 24.8 PI CONTROL PANEL <XP Rel. 3.0> 8.4 6.7 10.1 Peak Staff 67 54 80 Eff SLOC (K) Project: Elan 3.0 4/19/2006 17

Digitized Replica Using SLIM tm Staffing & Probability Analysis R&D C&T 1 2 Avg Staff (people) <XP Rel. 3.2> 3 4 5 6 7 8 10 Milestones 0 - CSR 1 - SRR 2 - HLDR 3 - LLDR 4 - CUT 5 - IC 6 - STC 7 - UAT 8 - FCR 9-99R 10-99.9R 8 6 4 2 Avg Staff (people) Jan '03 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep '04 0 Duration Effort Cost Peak Staff MTTD Start Date SOLUTION PANEL <XP Rel. 3.2> C&T Life Cycle 15.0 19.0 88 109 1496 1854 8.2 8.2-0.0-0.0 6/30/2003 3/1/2003 PI=14.8 MBI=1.3 Eff SLOC=76276 Months PM $ (K) people Days 14.8 11.9 17.8 PI CONTROL PANEL <XP Rel. 3.2> 8.2 6.6 9.8 Peak Staff 76 61 92 Eff SLOC (K) Project: Elan 3.2 4/19/2006 18

4/19/2006 19

David Tara...................................... 4/19/2006 20

4/19/2006 21

4/19/2006 22

Schedule Comparison C&T Duration (Months) vs Effective SLOC 100 Fast Schedules on Traditional Projects Even Faster on Agile Projects Traditional 1 Agile 2 Agile 1 Traditional 2 10 C&T Duration (Months) Agile 3 1 10 100 1000 Effective SLOC (thousands) All Systems Special Project QSM 2002 Scientific Avg. Line Style 1 Sigma Line Style 4/19/2006 23

Defect Comparison Errors SIT-FOC vs Effective SLOC High Defects on Traditional Projects Low Defects on Agile Projects Traditional 1 10000 1000 Traditional 2 Agile 1 100 Errors SIT-FOC Agile 3 Agile 2 10 1 10 100 1000 Effective SLOC (thousands) All Sy stems Special Project QSM 2002 Scientific Av g. Line Sty le 1 Sigma Line Sty le 4/19/2006 24

Example PI Calculation Size = 376,022 SLOC Condor PI = SIZE TIME EFFORT = 17 * Effort = 347 Person- Months Time = 24 Months 4/19/2006 25

Business Scientific System Productivity Index (PI) (industry values by application type) Information Process Control Engineering Telecommunications Command and Control Real Time Avionics Real Time Microcode 0 2 4 6 8 10 12 14 16 18 20 22 24 Productivity Index (PI) w/ ±1 Standard Deviation 4/19/2006 26

Before vs. After Summary Previous Performance Current Performance Percent Improvement Project Cost $2.8 Million $1.1 Million 61% Project Schedule 18 months 13.5 months 24% Cumulative Defects 2,270 381 83% Staffing 18 11 39% * Using average project size of 150,000 lines of new and modified code 4/19/2006 27

4/19/2006 28

4/19/2006 29

4/19/2006 30

Metrics Tools Used for this Analysis SLIM- Estimate Size, Schedule, Cost & Quality Estimating SLIM- Metrics Metrics Repository & Analysis Statistical Process Control & Adaptive Forecasting SLIM- Control 4/19/2006 31

Without metrics, you re just another person with a different opinion. 4/19/2006 32

Recommended Reading Honore, Carl, In Praise of Slowness, How A Worldwide Movement is Challenging the Cult of Speed, 2004 HarperSanFrancisco Gladwell, Malcolm, Blink; The Power of Thinking Without Thinking 2005 Little, Brown. Mah, Michael, The Making of the Agile IT Executive Business IT Strategies Advisory Executive Report Vol 6 Number 10. 2004 Cutter Information Corp. Putnam, Lawrence H., and Myers, Ware, Five Core Metrics, The Intelligence Behind Successful Software Management 2003 Dorset House Publishers. 4/19/2006 33

Recommended Reading Demarco, Tom and Tim Lister Waltzing With Bears, Managing Risk on Software Projects 2003 Dorset House. Ury, William, Getting Past NO 1993 Bantam. Fisher, Roger and Alan Sharp, Getting It Done, How to Lead When You re Not in Charge, 1998 HarperCollins. Heen, Sheila, Doug Stone and Bruce Patton Difficult Conversations - How to Discuss What Matters Most, 1999 Viking/Penguin. 4/19/2006 34

Recommended Web Resources Blogosphere: www.optimalfriction.com QSM Associates Web Library/Resource Center: www.qsma.com 4/19/2006 35

For Additional Information Contact: Michael Mah Managing Partner QSM Associates Inc. Clocktower Building 75 So. Church St., Suite 600 Pittsfield, MA 01201 Email: michael.mah@qsma.com Web url: www.qsma.com 4/19/2006 36