Agile Methods Barry Boehm, CS 510 Lecture Fall 2001 (boehm@sunset.usc.edu) (http://sunset.usc.edu) Outline Silver bullets and lead bullets Information technology trends The dwindling lead-bullet niche Underlying world-view trends Agile methods Example: extreme Programming (XP) Counterpoint: A skeptical view Relations to CMM and CMMI How much planning is enough? -CSE 2 The Silver Bullet Fantasy (Brooks, 1986) Example: Conway s Law and its Converse Even if we can t define what the software problem werewolf is, And we ve seen that lead, bronze, iron, and steel bullets can t kill it, We re sure there s a silver bullet that can ****** Fallacy: We can fix things we understand Accidental software problems But we can t fix things we don t understand Essential software problems Conway s Law (Datamation, 1968), extended The structure of a computer program... -CSE 3 -CSE 4 Example: Conway s Law and its Converse Conway s Law (Datamation, 1968), extended The structure of a computer program... reflects the structure of the organizations that build and use it Converse of Conway s Law We will learn how to build perfectly functioning software -CSE 5 -CSE 6 1
Converse of Conway s Law We will learn how to build perfectly functioning software As soon as we learn how to build perfectly functioning organizations The Lead Bullet Expectation If a lead bullet can kill a software-problem wolf this year, It will be able to do it next year too Counterexamples Waterfall model of the software process Pre-WYSIWYG word processing architecture Pre-Web book sales management applications Key drivers: technology, economics, humanization -CSE 7 -CSE 8 Information Technology Trends Lead Bullets with Dwindling Niches Traditional Development Standalone systems Stable requirements Rqts. determine capabilities Control over evolution Enough time to keep stable Value-insensitive process models Current/Future Trends Everything connected Rapid requirements change COTS capabilities determine rqts. No control over COTS evolution Ever-decreasing cycle times Value-oriented process models Complete, consistent, traceable, testable snapshot requirements Static domain architectures and enterprise architectures Heavyweight formal methods Fixed contract models of software management COTS- and value-insensitive objectoriented methods -CSE 9 -CSE 10 Core Niches for Current Lead Bullets - Still very important High-assurance, Real-time, Autonomous control systems Underlying World-View Trends Universalism Situationalism Stephen Toulmin, Cosmopolis, U. Chicago Press, 1990 Reductionism Emergence Stuart Kauffman, At Home in the Universe, Oxford U. Press, 1995. W. Brian Arthur, Increasing Returns and the New World of Business, Harvard Business Review, Jul/Aug 1996, pp. 100-109. James Highsmith, Adaptive Software Development, Dorset House, 1999. Software Focus System Focus John Thorp, The Information Paradox, M c Graw Hill, 1998. -CSE 11 -CSE 12 2
Cosmopolis: The Erosion of Modernist Philosophy Dominant since 17 th century Formal, reductionist Apply laws of cosmos to human polis Focus on written vs. oral; universal vs. particular; general vs. local; timeless vs. timely one-size-fits-all (lead bullet) solutions Strong influence on focus of computer science Weak in dealing with human behavior, rapid change Reductionism vs. Emergence Order is not imposed on complex adaptive systems; it emerges (Kauffman) Knowledge-based industries have increasing vs. decreasing returns (Arthur) Network effects, up-front costs, customer groove-in Adaptation succeeds better than optimization Adaptive model best fits future software projects (Highsmith) Balance of discipline and flexibility -CSE 13 -CSE 14 Outline Silver bullets and lead bullets Information technology trends The dwindling lead-bullet niche Underlying world-view trends Agile methods Example: extreme Programming (XP) Counterpoint: A skeptical view Relations to CMM and CMMI How much planning is enough? -CSE 15 The Agile Manifesto - I We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. -CSE 16 The Agile Manifesto II The Agile Manifesto III Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Business people and developers must work together daily throughout the project. -CSE 17 Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Working software is the primary measure of progress. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. -CSE 18 3
The Agile Manifesto IV Various Agile Methods Available Continuous attention to technical excellence and good design enhances agility. Simplicity the art of maximizing the amount of work not done is essential. The best architectures, requirements, and designs emerge from self-organizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. -CSE 19 Adaptive Software Development (ASD) Agile Modeling Crystal methods Dynamic System Development Methodology (DSDM) * extreme Programming (XP) Feature Driven Development Lean Development Scrum -CSE 20 extreme Programming (XP) Principles The 12 Practices Early Adopters XP Principles I Philosophy: Take known good practices and push them to extremes If code reviews are good, we ll review code all the time If testing is good, we ll test all the time If design is good, we ll make it part of everybody s daily business -CSE 21 -CSE 22 XP Principles II XP Principles III If simplicity is good, we ll always leave the system with the simplest design that supports its current functionality If architecture is important, everybody will work defining and refining the architecture all the time If integration testing is important, then we ll integrate and test several times a day -CSE 23 If short iterations are good, we ll make the iterations really, really short seconds and minutes and hours, not weeks and months and years If customer involvement is good, we ll make them full-time participants -CSE 24 4
XP: The 12 Practices The Planning Game The Planning Game Small Releases Metaphor Simple Design Testing Refactoring Pair Programming Collective Ownership Continuous Integration 40-hour Week On-site Customer Coding Standards -Used generatively, not imperatively -CSE 25 Use stories to facilitate knowledge transfer Put decisions in the hands of the person with the best knowledge: business decisions Customer software decisions Developer Plan only as far as your knowledge allows next iteration or next release -CSE 26 Small Releases Metaphor Supports quick feedback from users Simplify the tracking of metrics stories per iteration project velocity Increase the manageability of the project for the customer But complicate user conservation of familiarity Ground all discussions in a single shared story of how the whole system works Provide an overarching view of the project Connect program to work process -CSE 27 -CSE 28 Simple Design Design Embodies only the needed complexity and no more emphasis on top-down or bottom-up design as needed to meet this iteration s stories extra complexity removed when discovered Simpler designs are easier to modify, maintain, and describe decreases the cost of design changes But no notion of product line architecture Testing Unit tests verify the programmer s work must be done by programmer constant testing makes finding new bugs faster and easier Functional tests verify that a story is complete developed by customer tests define functional requirements -CSE 29 -CSE 30 5
Refactoring Pair Programming Procedure for implementing iterative design behavior-preserving improves communication among developers adds flexibility to the programming process Design is important do it all the time software development process is a design process But redesign much more expensive for large systems -CSE 31 All code is written by two programmers at a single machine Inspections are important, so do them all the time Increase implicit knowledge transfer Decrease cycle time, but increase effort -CSE 32 Collective Ownership Continuous Integration Everyone owns all of the code anyone can change any code anywhere no personal ownership of modules no egoless programming either Everyone is permitted access to all the code so everyone has a stake in knowing all of the code (that they will work with) Requires deserved trust But still has scalability problems The system always works there is always something to be released Similar to rapid releases fast feedback to developers on problems no big bang integration disasters -CSE 33 -CSE 34 40-hour Week On-site Customer No heroes Knowledge can only be transferred at a limited rate Work for sustained speed, not a single sprint never work overtime a second week in a row A real, live user available full-time to answer questions as they occur Programmers don t know everything Business knowledge is the key to a successful business project -CSE 35 -CSE 36 6
Coding Standards Communication occurs through the code Common standard promotes understanding of other developers code Helps promote team focus Counterpoint: A Skeptical View I Letter to Computer, Steven Rakitin, Dec. 2000 individuals and interactions over processes and tools Translation: Talking to people gives us the flexibility to do whatever we want in whatever way we want to do it. Of course, it s understood that we know what you want - even if you don't. working software over comprehensive documentation Translation: We want to spend all our time coding. Real programmers don t write documentation. -CSE 37 -CSE 38 Counterpoint: A Skeptical View II Letter to Computer, Steven Rakitin, Dec. 2000 customer collaboration over contract negotiation Translation: Let's not spend time haggling over the details, it only interferes with our ability to spend all our time coding. We ll work out the kinks once we deliver something... responding to change over following a plan Translation: Following a plan implies we would have to spend time thinking about the problem and how we might actually solve it. Why would we want to do that when we could be coding? -CSE 39 Outline Silver bullets and lead bullets Information technology trends The dwindling lead-bullet niche Underlying world-view trends Agile methods Example: extreme Programming (XP) Counterpoint: A skeptical view Relations to Software CMM and CMMI The planning spectrum Agile and Plan-Driven home grounds How much planning is enough? -CSE 40 The Planning Spectrum Agile and Plan-Driven Home Grounds Agile Home Ground Plan-Driven Home Ground Hackers XP Adaptive Sw Devel. Agile Methods CMMI Milestone Risk- Driven Models Milestone Plan-Driven Models Software CMM Inch- Pebble Ironbound Contract Agile, knowledgeable, collaborative developers Dedicated, knowledgeable, collaborative, representative, empowered customers Largely emergent requirements, rapid change Architected for current requirements Refactoring inexpensive Smaller teams, products Premium on rapid value Plan-oriented developers; mix of skills Mix of customer capability levels requirements knowable early; largely stable Architected for current and foreseeable requirements Refactoring expensive Larger teams, products Premium on high-assurance -CSE 41 -CSE 42 7
How Much Planning Is Enough? - A risk analysis approach Risk Exposure Prob (Loss) * Size (Loss) Example RE Profile: Planning Detail - Loss due to inadequate plans high P(L): inadequate plans high S(L): major problems (oversights, delays, rework) Loss financial; reputation; future prospects, For multiple sources of loss: Σ [Prob (Loss) * Size (Loss)] source sources low P(L): thorough plans low S(L): minor problems Time and Effort Invested in plans -CSE 43 -CSE 44 Example RE Profile: Planning Detail - Loss due to inadequate plans - Loss due to market share erosion Example RE Profile: Time to Ship - Sum of Risk Exposures high P(L): inadequate plans high S(L): major problems (oversights, delays, rework)) low P(L): few plan delays low S(L): early value capture high P(L): plan breakage, delay high S(L): value capture delays low P(L): thorough plans low S(L): minor problems high P(L): inadequate plans high S(L): major problems (oversights, delays, rework) Sweet Spot low P(L): few plan delays low S(L): early value capture high P(L): plan breakage, delay high S(L): value capture delays low P(L): thorough plans low S(L): minor problems Time and Effort Invested in Plans Time and Effort Invested in Plans -CSE 45 -CSE 46 Comparative RE Profile: Plan-Driven Home Ground Comparative RE Profile: Agile Home Ground Higher S(L): large system rework Plan-Driven Sweet Spot Mainstream Sweet Spot Lower S(L): easy rework Agile Sweet Spot Mainstream Sweet Spot Time and Effort Invested in Plans Time and Effort Invested in Plans -CSE 47 -CSE 48 8