Governments information technology

So l u t i o n s Blending Agile and Lean Thinking for More Efficient IT Development By Harry Kenworthy Agile development and Lean management can lead to more cost-effective, timely production of information technology that better meets users needs. Governments information technology projects don t always deliver the promised results. Combine performance problems with a reputation for being late and over budget, and it s easy to see why officials are often reluctant to take on major IT initiatives. The problem isn t inherent in government projects, however; it s caused by the traditional waterfall approach to project planning. And it can be minimized by using a different kind of project planning, a Lean approach known as Agile development. Agile development and Lean management can lead to more cost-effective, timely production of information technology that better meets users needs. TRADITIONAL WATERFALL DEVELOPMENT Waterfall development has been the dominant approach among IT professionals for years. This approach is about trying to understand the customer s new needs and then working on the development for months, after which the final output is released a top-to-bottom process. Customer feedback is given when the overall project is completed. One study indicates that 45 percent of the features in a typical system are never used, and only 7 percent are always used (see Exhibit 1). And overall, success rates for IT development, based on a sampling of multiple publications and studies, are not good: n Approximately 31 percent of projects get cancelled. n Approximately 66 percent of projects don t meet customer needs and are therefore considered failures. n More than 50 percent of projects exceed their budgets by 200 percent. n Deliverables are not well planned or well managed. n Project managers could use more training. n Approximately 10 percent of the developed code was actually used. n Approximately 82 percent of projects cite waterfall practices as the primary reason for failure. Many articles have been written about chaos theory and how it relates to waterfall software development. This is because so many uncertainties and variables can come into play when software is developed. Also, some of the assumptions waterfall development makes are unrealistic, such as the idea that customer needs being clearly defined upon going into the project, and the needs of the client department are thoroughly vetted and agreed upon, and that the requirements can be accurately determined at the beginning of the project. Waterfall moves along a straight path, based on initial inputs and assumptions, so errors or miscalculations made at the outset of the project will be included in the final product. Waterfall also assumes that timeframes and budgets are easy to 66 Government Finance Review April 2014

predict in the beginning; many waterfall developments fall in quarterly timeframes for progress and outputs, which is not nearly the frequency needed for reviewing progress and alignment with customer needs, and soliciting customer feedback (which is the final step in a waterfall process, after the project is completed). THE EVOLUTION OF AGILE In 2001, a group of 17 IT professional gathered to develop an alternative to the waterfall approach, which they called Agile. 1 The group agreed to an Agile Manifesto (see Exhibit 2), which included several values aimed at making IT development more effective: 1. Individuals and interactions should take precedence over processes and tools. 2. Working software is the desired output, as opposed to comprehensive documentation. 3. Customer collaboration is superior to negotiating contracts with clients. 4. Responding to change is more important than blindly following a plan. Exhibit 3 illustrates an example of the difference between a waterfall and an Agile approach. The group developed the following key aspects of the system and terminology concepts (see Exhibit 4). These can be summarized as follows: 1. The overall software client needs would be broken down to segments (known as stories) that are prioritized by their importance for development; this is known as backlog grooming. The team would do the development based on the customer s prioritized needs. Exhibit 1: The Features and Functions that are Actually Used in a Typical System 16% 13% Source: Standish Group. 7% 19% 45% Exhibit 2: The Principles of the Agile Manifesto 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. 2. Welcome changing requirements, even late in development. Agile processes harness change for the customer s competitive advantage. 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 4. Business people and developers must work together daily throughout the project. 5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 7. Working software is the primary measure of progress. n Never n Rarely n Sometimes n Often n Always 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 9. Continuous attention to technical excellence and good design enhances agility. 10. Simplicity the art of maximizing the amount of work not done is essential. 11. The best architectures, requirements, and designs emerge from self-organizing teams. 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. April 2014 Government Finance Review 67

Exhibit 3: A Waterfall Approach versus an Agile Approach Waterfall: Entire Timeframe 100 Percent Time Start Development Agile Agile Provides: n Decreased Risk n More n Less Confusion n Greater Employee Satisfaction Long Duration Before Results are Produced Exhibit 4: Overview of an Agile Project n More Points n More Time with Clients n Less Time Needed Before Adjustments or Corrections are Made 20 Percent Time 20 Percent Time 20 Percent Time 20 Percent Time 20 Percent Time Short Release Short Release Short Release Short Release Short Release Product Owner Team: Prioritize project requirements. Product Backlog Stories Sprint Backlog Product Owner and Sprint Team: Select top prioritized requirements to be completed for the Sprint. 24 Hours 2-4 Weeks Sprint Scrums are held for 15 minutes each morning to review previous day s progress and establish plans for current day. Product Sprint Team delivers a working functional product = completes the Sprint backlog. 2. Stories (a segment or element the backlog development needs) would then be developed. The suggested story structure is: As a <user type> I want to <do some action> so that <desired result> This helps the development team identify the user, action, and required result in a request, and it is a simple way of writing requests that anyone can understand (e.g., As a user I want a tool bar on the screen so that I can easily drop in changes ). 3. The prioritized sections would be broken into sections, or stories, lasting 1-4 weeks; these are called Sprints, and each Sprint ends with a working functional product for customer review. Each Sprint team, normally of six to ten people, would be dedicated full time. (There could be more team members, based on the complexity of the software development, but smaller teams work better, in general.) Ideally, all team members should be at the same site for ease of communication and coordination. 4. A Sprint planning meeting is then held with the assigned Sprint team to determine what details are required to perform the work for the current Sprint. This includes developing a detailed plan with elements and timeframes. Various techniques can be used to determine a consensus for the team time requirements, such as planning poker. 2 5. The Sprint is then broken down to daily assignments for each team member. Each Sprint should include a daily review of progress, called a Scrum, for approximately 15 min- 68 Government Finance Review April 2014

utes at the beginning of each day to review what was completed the day before, what the team expects to complete on the current day, and any impediments that team members have experienced. 6. At the end of each Sprint, the customer would review the functional output and review the prioritized product backlog. The backlog priorities could change based on the output of each sprint. 7. The team leader, or Scrum master, monitors project progress, provides support to the team, and helps remove obstacles that team members may be experiencing. INTEGRATING LEAN WITH AGILE Agile is clearly aligned with the principles of Lean, which are: 1. Add value for the customer. the desired outcomes; the customer s team or department also needs to be completely on-board with the desired outcomes. Many times, this is not the case. More time should be spent on the front end of any IT software development to clearly specify what outcomes are required and to make sure that everyone involved has the same clear understanding of all the appropriate operational definitions. Agile offers an advantage for customer deliverables that are not well defined, in that these issues will be discovered much earlier in the software development process. Because Agile breaks down the project development into key stories that are then prioritized, customer communications are significantly better than in a traditional waterfall process. Agile will produce the first demonstration deliverable within two to four weeks for the customer to evaluate, along with the opportunity to look at the remaining stories and priorities in the backlog that still need to be developed. A project charter document can be adapted to support the Agile process (see Exhibit 5). The customer fills out the project charter and delivers it to the Agile team, providing a good process for clarifying and verifying any changes that need to be made. Mapping Key Value Systems, or Learning to See. Too many projects are initiated without a clear understanding of how the current process operates, including all of the process wastes and inefficiencies. Nothing should be automated until the current process has gone through a Lean review to remove wastes; automating a junk process provides automated junk. Understanding the process and its outcomes happens in the actual work area, where the 2. Learn to See using value mapping streams. Exhibit 5: An Agile-Lean Project Charter 3. Create flow the goal is one-stop shopping. 4. Keep pace with the customer demand rate pull from the customer. 5. Aim for perfection get it right the first time, eliminating errors and rework. Adding Value for the Customer. This is all about clearly understanding what the customer wants to change, fix, improve, or enhance. Any IT development project should ensure that the customer has a clear, comprehensive understanding of what they want to do and what the desired outcomes are. It s not enough for a customer to describe April 2014 Government Finance Review 69

action is, and not with a few high level managers having a discussion in a conference room. When managers take the time to go to the work area and understand in great detail what actually happens, they learn to see which includes being amazed at how things are really done, including the amount of variation. Creating Flow. A process should be designed with as few handoffs as possible before the software is created. Employees often say we have to do it this way, but don t just accept that assertion; look for the facts that support it, and make sure than any existing legislation/mandates are truly valid. Tribal knowledge, along with how effectively or ineffectively training is transmitted over time, creates beliefs about why we have to do it this way that usually aren t true. The first step in getting past these beliefs is to clearly understand the current state of how things are actually done, in as much detail as possible. Then, examine the current state for wastes such as errors and rework, and ask workers about their areas of frustration and any ideas they may have to improve the process. This information allows the organization to design a streamlined and effective future state. This future state design should be tested using an iterative process called the Plan-Do-Check-Act/Adjust cycle. Once these steps have been taken, an Agile development process can begin. Keeping Pace with the Customer Demand Rate, or Pulling from the Customer. Agile is aligned with customer needs by creating useable output that can be demonstrated and tested in much shorter cycle times rather than waiting many months for a waterfall software output that might have missed the customer s mark. Getting It Right the First Time. Each step of the development is tested to ensure that what is being designed is what the customer needs. Looking at the current state error and rework rates allows Agile developers to take corrective actions and mistake-proof the new software. CONCLUSIONS Why, then, isn t Agile used more often? People, and organizations, are resistant to change. In a magazine article, Steve Denning cited a number of perceived objections to Agile. 3 Following are some on the more frequently encountered objections, followed by counterarguments: n Agile is only for stars. Employees are able to use and embrace Agile when they see how it works and what it accomplishes. n Agile doesn t fit our organizational culture. These objections can be countered by support from management; culture is driven by leadership. n Agile only works for small projects, and ours are big. Large projects can be divided in smaller pieces to which Agile can be applied. Agile requires co-location, and our staff is geographically dispersed. Although it is ideal for Agile teams to be at the same site, modern technology such as video conferencing makes communication easy. n Agile lacks project management processes. Agile can be integrated with Lean, PERT, GANNT, and other process methodologies. n Our individual accountability systems don t fit Agile. Accountability systems are driven by management and can be changed. n Agile is just a fad. Agile is an effective approach to software development that combines greater discipline with creativity and execution. n There are better ideas than Agile. All organizations should understand and build on best practices and steal shamelessly, but legally. n Nothing new here. The Agile approach is rooted in project management approaches, combined with Lean. This combination is what makes Agile effective. n Not a fair comparison. Agile is a mindset, not a method. Changing mindsets regarding software development open up more highly effective approaches (such as Agile) and provide opportunities to deliver better software, on or before due dates, at or less than budgeted costs. Agile is a Leaner approach to developing software than the traditional waterfall approach, creating more feedback and thus better alignment with the customer s needs. For the same reasons, Agile decreases risk and creates less confusion and greater employee satisfaction. With Agile, organizations use much less resource time discovering the need for corrections or adjustments. Agile helps organizations deliver better and more successful projects, faster, and at lower costs. There are challeng- 70 Government Finance Review April 2014

es, of course. Leaders and management need to have a clearer understanding of what Agile is, what it does, and its associated benefits than most currently do. Agile is also more successful with a greater level of communication from leaders and management. It also creates a greater need for customer interface and feedback loops. There is really no reason not to explore and embrace Agile for software development. With proper guidance and management support, the risks are extremely low, and the benefits are great. y Notes 1. See the Manifesto for Agile Software Development at http://agilemanifesto.org/. 2. Planning Poker is available at no charge from sources such as MountainGoatSoftware.com. 3. Steve Denning, The Case Against Agile: Ten Perennial Management Objections, Forbes, April 17, 2012. HARRY KENWORTHY is principal and manager of the Quality and Productivity Improvement Center (QPIC, LLC), a consulting organization he founded in 1984. He was one of the first practitioners to apply Lean in the government sector, and his consulting work has included numerous government processes that have been improved by removing waste, reducing costs, or increasing revenues in a variety of operational steps while reducing overall process cycle times and improving customer service. QPIC s business is with cities, and state and federal agencies implementing Lean in government. Kenworthy worked with W. Edwards Deming in 1983-85 on a series of seminars throughout the United States, sponsored by MIT, and he has spoken at more than 90 conferences on quality, productivity, Lean, and Six Sigma, as well as writing for several magazines. Capital One Bank has gold-star-worthy industry experts ready to help you operate more effectively and efficiently. Our full banking service offers cash management, investing and lending solutions. So team up with the Government Banking experts at Capital One, a top-10 U.S. Bank. It s the smartest choice you can make. Chip Motil, Head of Government Banking, chip.motil@capitalone.com, (631) 531-2325 Karen Bauer, NY/NJ Market Manager, karen.bauer@capitalone.com or (631)531-2669 Tammy Leisen, Relationship Manager, tammy.leisen@capitalone.com or (631)531-2324 Nefi Pongas, Relationship Manager, nefi.pongas@capitalone.com or (631)531-2349 Wayne Kuss, Relationship Manager, wayne.kuss@capitalone.com or (973)439-7621 Products and services offered by Capital One, N.A., Member FDIC. 2012 Capital One. Capital One is a federally registered service mark. All rights reserved. April 2014 Government Finance Review 71