What is meant by the term, Lean Software Development? Scope of this Report November 2014 This report provides a definition of Lean Software Development and explains some key characteristics. It explores the similarities and differences between Lean Software Development, Lean Manufacturing and Lean Six Sigma. Finally, it considers the extent to which traditional waterfall and agile (primarily scrum) approaches to software development can be considered as Lean Software Development. A Definition of Lean Software Development According to David Anderson, the term Lean Software Development was first coined as the title for a conference organized by the ESPRIT initiative of the European Union, in Stuttgart Germany, October 1992, and, independently, by Robert Bob Charette in 1993 as part of his work exploring better ways of managing risk in software projects. The term Lean dates to 1991, suggested by James Womack, Daniel Jones, and Daniel Roos, in their book The Machine That Changed the World: The Story of Lean Production as the English language term to describe the management approach used at Toyota (see Lean Family Tree below). The idea that Lean might be applicable in software development was established very early, only 1 to 2 years after the term was first used in association with trends in manufacturing processes and industrial engineering. In their 2nd book, published in 1995, Womack and Jones defined five core pillars of Lean Thinking. These were: Value Value Stream Flow Pull Perfection - achieved by eliminating waste. Though widely accepted as the default working definition for Lean over most of the next decade, Lean principles were still much more associated with (and acted on) in manufacturing than in software development. As Anderson goes on to say, defining Lean Software Development is challenging because there is no specific Lean Software Development method or process. A software development lifecycle process or a 2014 David Consulting Group Page 1 of 8
project management process could be said to be Lean if it was observed to be aligned with the values of the Lean Software Development movement and the principles of Lean Software Development. A working definition of Lean Software Development Mary and Tom Poppendieck gave the topic of Lean Software Development a fresh burst of life in the first decade of the 21 st century by modifying Womack and Jones five pillars into seven principles of Lean Software Development (with guidance on the main ways to apply these principles to software development): Eliminate Waste Build Quality In Create Knowledge Defer Commitment Deliver Fast Respect People Optimize the Whole We shall use this as the basis of definition for this report. Characteristics of Lean Software Development David Anderson has identified the following characteristics of Lean Software Development to identify what it is and what it isn t: There is no such thing as a single Lean Software Development process. A process could be said to be Lean if it is clearly aligned with the values and principles of Lean Software Development. Lean Software Development does not prescribe any practices, but some activities have become common. Lean organizations seek to encourage continuous improvement (also called kaizen) through visualization of workflow and work-in-progress and through an understanding of the dynamics of flow and the factors (such as bottlenecks, non-instant availability, variability, and waste) that affect it. Process improvements are suggested and justified as ways to reduce sources of variability, eliminate waste, improve flow, or improve value delivery or risk management. Lean Software Development processes will always be evolving and uniquely tailored to the organization within which they evolve. It will not be natural to simply copy a process definition from one organization to another and expect it to work in a different context. It will be unlikely that returning to an organization after a few weeks or months to find the process in use to be the same as was observed earlier. It will always be evolving. The Lean Family Tree 2014 David Consulting Group Page 2 of 8
As we have already described, Lean Software Engineering has some ancestors and some cousins best summarized by this family tree created by the Poppendiecks: The most important point to note for the purposes of this report is that while Lean thinking for software development has evolved from the work in manufacturing, operations and supply chains, these are no more than cousins because one of the two underlying assumptions about inputs, or more strictly input variance, is very different for the product development/software development branch of the family To understand this critical difference, we need to step back a little to review fundamentals. The principles of Lean management at Toyota are very subtle. The single word waste in English is described more richly with three Japanese terms: Muda literally meaning waste but implying non-value-added activity Mura meaning unevenness and interpreted as variability in flow Muri meaning overburdening or unreasonableness The major difference between Lean manufacturing and Lean product development/software engineering lies in the concept of Mura. In manufacturing and operations, Mura or variability of flow (or often just variability) is a parameter which can and should be controlled within established parameters. There is an assumption of uniformity of inputs entering the same production line or process which will nonetheless have some manageable variance from that uniformity. Indeed, this is one of the foundations of Lean six sigma. In product and software development there can be no assumption of uniformity of inputs - every product or software project entering the same production Line or development team will be different. This variability can lead to good outcomes in the form of creative solutions. 2014 David Consulting Group Page 3 of 8
Lean manufacturing broadly assumes that the same processes are performed on the same inputs yielding the same outputs where Lean software development assumes that the same processes are applied to different inputs yielding different outputs. The 7 Principles of Lean Software Development 1. Eliminate Waste The three biggest wastes in software development are: a. Extra Features We need a process that allows us to develop just those 20% of the features that give 80% of the value b. Churn If you have requirements churn, you are specifying too early. If you have test and fix cycles, you are testing too late. c. Crossing boundaries Organizational boundaries can increase costs by 25% or more. They create buffers (queues) that slow down response time and interfere with communication 2. Build Quality In if you routinely find defects in your verification process, your development process is defective a. Mistake-proof code with Test-Driven Development (TDD) write executable specifications instead of requirements b. Stop building legacy code redefine legacy code to be code that lacks automated unit and acceptance tests c. The big bang release is obsolete use continuous integration and nested synchronization to develop, test and release (or at least be ready to release) incrementally 3. Create Knowledge Planning is useful insofar as it builds knowledge. Learning is essential. a. Use the scientific method hypothesize, experiment, learn and repeat in as short cycles as possible b. Standards exist to be challenged and improved c. Predictable performance is driven by the right sort of feedback A predictable organization does not guess about the future and call it a plan; it develops the capacity to rapidly respond to the future as it unfolds based upon the scientific method 4. Defer Commitment Abolish the idea that it is a good idea to start development with a complete specification a. Break dependencies System Architecture should support the addition of any feature at any time b. Maintain Options Think of code as an experiment make it change-tolerant c. Schedule irreversible decisions at the last responsible moment Learn as much as possible before making irreversible decisions. 5. Deliver Fast lists and queues are buffers between organizations and/or teams and/or processes that slow things down a. Rapid delivery, high quality and low cost are fully compatible b. Queueing theory applies to development not just to servers c. Limit work to capacity establish a reliable, repeatable velocity with iterative development/. Aggressively limit the size of queues (including the input queue) to your capacity to deliver 2014 David Consulting Group Page 4 of 8
6. Respect People Engaged, thinking people provide the most sustainable competitive advantage. a. Teams thrive on pride, commitment, trust and applause b. Provide effective leadership c. Respect partners 7. Optimize the Whole a. Focus on the entire value stream local optimizations can, and often do, hurt the whole i. From concept to cash ii. From customer request to deployed software b. Deliver a complete product not just the software. Complete products are build by complete teams c. Measure UP i. Measure process capability with cycle time ii. Measure team performance with delivered business value iii. Measure customer satisfaction with a net promoter score. Practices associated with Lean Software Development As David Anderson has pointed out, Lean Software Development does not prescribe practices. It is more important to demonstrate that actual process definitions are aligned with the principles and values. However, a number of practices are being commonly adopted: Cumulative Flow Diagrams Cumulative flow diagrams plot an area graph of cumulative work items in each state of a workflow. They are rich in information and can be used to derive the mean cycle time between steps in a process as well as the throughput rate (or velocity ). Practitioners can learn to recognize patterns of dysfunction in the process displayed in the area graph such as bottlenecks and observe the effects of mura (variability in flow). Visual Controls Lean Software Development teams use physical boards, or projections of electronic visualization systems, to visualize work and observe its flow. Such visualizations help team members observe work-inprogress accumulating and enable them to see bottlenecks and the effects of mura. Visual controls also enable team members to self-organize to pick work and collaborate together without planning or specific management direction or intervention. These visual controls are often referred to as card walls or sometimes (incorrectly but unstoppably) as kanban boards. Virtual Kanban Systems A kanban system is a practice adopted from Lean manufacturing. It uses a system of physical cards to limit the quantity of work-in-progress at any given stage in the workflow. Such work-in-progress limited systems create a pull where new work is started only when there are free kanban indicating that new work can be pulled into a particular state and work can progress on it. In Lean Software Development, the kanban are virtual and often tracked by setting a maximum number for a given step in the workflow of a work item type. In some implementations, electronic systems keep track of the virtual kanban and provide a signal when new work can be started. The signal can be visual or in the form of an alert such as an email. Virtual kanban systems are often combined with visual controls to provide a visual virtual 2014 David Consulting Group Page 5 of 8
kanban system representing the workflow of one or several work item types. Such systems are often referred to as kanban boards or electronic kanban systems. Small Batch Sizes Lean Software Development requires that work is either undertaken in small batches, often referred to as iterations or increments. A small batch of work would typically be considered a batch that can be undertaken by a small team of 8 people or less in under 2 weeks. Small batches require frequent interaction with business owners to replenish the backlog or queue or work. They also require a capability to release frequently. To enable frequent interaction with business people and frequent delivery, it is necessary to shrink the transaction and coordination costs of both activities. A common way to achieve this is the use of automation. Automation Lean Software Development expects a high level of automation to economically enable small batches and to encourage high quality and the reduction of failure demand. The use of automated testing, automated deployment, and software factories to automate the deployment of design patterns and creation of repetitive low variability sections of source code will all be commonplace in Lean Software Development processes. Kaizen Events In Lean literature, the term kaizen means continuous improvement and a kaizen event is the act of making a change to a process or tool that hopefully results in an improvement. Daily standup meetings Managers will encourage the emergence of kaizen events after daily standup meetings in order to drive adoption of Lean within their organization. Retrospectives Project teams may schedule regular meetings to reflect on recent performance. These are often done after specific project deliverables are complete or after time-boxed increments of development known as iterations or sprints in Agile software development. Operations Reviews An operations review is typically larger than a retrospective and includes representatives from a whole value stream. It can be thought of as a retrospective for all the teams together. Lean Software Development and Waterfall Well, it s simple isn t it? If you want to do Lean software development, you must use Agile? Well, not quite. For a start, the role based steps of waterfall development look a lot like the process steps of a manufacturing or process production line and Lean was founded in manufacturing so there must be a way to do Lean waterfall. Why would you try? Because waterfall software development still has a role to play. Not all software development lends itself to an Agile approach. 2014 David Consulting Group Page 6 of 8
Some of the key practices from Lean software development can be applied in a waterfall environment. Starting with a kanban board and cumulative flow charts can set the scene for the introduction of workin-progress (WIP) limits at the different steps in the waterfall process. Introducing a WIP limit at the very start of the waterfall process can assert discipline around working on the small batch sizes that drive much of Lean. Starting from the Lean Software Engineering Practices, it becomes easier to gradually introduce more and more compliance with Lean Software Engineering Principles. Is this all just driving a waterfall model to an Agile model? Perhaps but not necessarily and, in any case, it may make for a softer culture transformation than a big bang move to agile since it should bring with it greater understanding of why Agile works. Reinertsen spells out simply and effectively how and why the Lean principles work without ever mentioning Agile. Lean Software Development and Agile So, if you are doing Agile then you must be doing Lean? Well, on the whole, yes but not necessarily 100%. It s certainly true that there has been much cross-pollination between Lean Software development and Agile in the form of Scrum. Further, the Agile Manifesto and its associated 12 principles bear a striking resemblance to the Lean Software Engineering Principles. The Scaled Agile Framework (SAFe) embodies Lean Software Development principles in its design for scaling Scrum. It is certainly true that Scrum is designed to impose small batch WIP limits based on the capacity of the team (and set by the team). Scrum limits (eliminates?) waste from handoffs (e.g. queues between coding and testing). However, the very need for building something like SAFe highlights the facts that many Agile development organizations are not as Lean as they could be. Scrum optimizes work for one to a few scrum teams and sometimes this is adequate but, for value streams with many teams, scrum can sometimes optimize the individual team at the expense of the optimization of whole value stream. Conclusions There are many sources for definitions and opinions on the meaning of the term Lean Software Development. This is a case where a few sources that are respected and established in the industry are better than crowd sourcing a definition. Further reading will be worthwhile but only after digesting this short report and reviewing the sources listed below. Agile software development overlaps significantly with Lean Software Engineering but they are not the same. Waterfall software development can be implemented in accordance with and benefit from Lean Software Development Practices and Principles. Sources MSDN Library, Lean Software Development by David J. Anderson, http://msdn.microsoft.com/en-us/library/hh533841.aspx Implementing Lean Software Development From Concept to Cash, Mary and Tom Poppendieck, Addison Wesley, 2007. [Or, indeed, any book by Mary and Tom on this topic!] 2014 David Consulting Group Page 7 of 8
http://leansystemssociety.org/ The Principles of Product Development Flow Second Generation Lean Product Development, Donald G. Reinertsen, Celeritas Publishing, 2009. The Agile Manifesto: http://agilemanifesto.org/ 2014 David Consulting Group Page 8 of 8