Agile Software Development Use case for Agile Software Development Methodology in an Oil and Gas Exploration environment. White Paper
Introduction No matter what business you are in, there are critical systems, processes, and work flows controlled by computer software. The users have a wish list about how the software could be improved to help them do their jobs faster, cheaper, or more accurately. Often the wishes are complex or conflicting, and the present setup is not well documented. You would like a methodology, exploratory in nature, that allows you to discover a path towards better software without wasting time on arduous documentation. You would like a methodology as flexible as the team you employ, the very team generating these use cases. You would like a methodology for getting these modifications into production. You would like a methodology as nimble as you are. What you need is Agile. Agile promotes adaptive planning, evolutionary development and delivery, and a timeboxed iterative approach. Agile Primer Agile software development represents a sequence of iterative steps that leverage the collaboration of interdisciplinary self-organizing teams to develop requirements and incrementally arrive at solutions. A common description of Agile is that it promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change. It is a conceptual framework that promotes foreseen interactions throughout the development cycle. To better understand Agile, let s first take a brief look at the more classic waterfall method. The Waterfall (Classic) Method The Waterfall method has been around since the early days of software design. A borrowed methodology from manufacturing and distribution, it takes a linear approach to development. This classic method involves gathering every possible requirement up front and proceeding stepwise through design and implementation. The basic steps of the Waterfall Method are: 1 Requirements specification Design Coding Integration Testing and debugging Installation Maintenance Lessons learned through development are documented, not integrated into the current iteration, and used to inform subsequent redesigns. This approach focuses on delivering a fixed work scope while limiting scope creep, and there are situations in which it functions quite well. 1 Royce, Winston W. Article published in 1970, Managing the Development of Large Software Systems Waterfall development is well suited to projects where there is limited complexity, the requirements are well known at the outset, and the software is not highly integrated with other business processes. Imagine a development project to design a system for recording grades for an elementary school. Waterfall can be a great method for delivering this type of project on a predictable cost and schedule.
Waterfall Characteristics Requirements & Tasks are well defined at outset. Methodology is highly document driven. Project roles are highly structured and well defined. Communication is through PM and Sponsor. Development cycles are typically long. Project moves forward only after current step is perfected. Progression always moves forward, never backward The Cone of Uncertainty Before we explore Agile software development methodology, we must first understand how knowledge unfolds during the course of a project. Project Management frameworks look at this knowledge expansion inversely as a diminishing uncertainty. At the time of project conception, we know little and uncertainty is high. As we move through the period of definition and requirements gathering, we begin to know more. The process of design and coding gives us more information, further diminishing uncertainty. By the time of project completion we have reduced uncertainty almost completely. This process can be represented by a Cone of Uncertainty. You can see that the Waterfall method drives the bulk of project decisions at the time when uncertainty is greatest, which for a project of any complexity is a risky approach. Given the level of project complexity in exploration software, the rapidly changing science and technology in this area, and our inability to know everything before the start of a project, how can we develop software for this industry that hits the mark, contains costs, and benefits from the increasing knowledge gleaned over the project life? Enter Agile.
The Agile Manifesto 2 In 2001 seventeen software developers with a history of working in lightweight project management environments met to talk, ski, relax, and try to find common ground. The result of that meeting was the Manifesto for Agile Software Development which describes best practices in their field. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over process and tools Working product over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan While there is value in the items on the right, we value the items on the left more. Shared Vision Agile software development depends heavily on a cohesive vision shared by the customer and the development team. Working closely together, the team can iterate quickly through project cycles. Roles under a Bluware Agile project look generally like this: Upper management: the financial buyer who decides to invest budget in development Customer: person at client with the problem to be solved Product Owner: the single business voice available daily to make all business decisions. This role replaces the documentation of more traditional methods Scrum Master: consulting project manager, working side by side with Product Owner Scrum Team: self-directed development team which may be a mix of Bluware and client employees with cross-functional skills that divide the work in accordance with their abilities Together, following the shared vision, the team goes through the Agile process. Agile Process The process and terms below describe the Agile method referred to as Scrum, the most common methodology Bluware employs for software development. Any meetings are public and include the Product Owner, Scrum Master and Team to foster transparent communication. User stories capture the ideas and use cases presented by end users in everyday business language. They are generally short and capture the idea of a problem and the benefit from solving it. They generally follow a format of As a [role], I want [solution] so I can [benefit]. These user stories help the customer decide which features reach the product backlog. Product backlog contains the user stories the customer is considering for development into the soft- ware. Features chosen for development in a project move to the Release Plan. 2 The Agile Manifesto; http://agilemanifesto.org/ The Release Plan contains the user stories approved for development in the current project, along with acceptance criteria for each story to clarify how the team will test whether the software is working.
The Sprint Planning Meeting is the forum for deciding which items and features from the Release Plan are going into the next development cycle, or sprint. The features for the next cycle are put into the Sprint Backlog. Each Sprint is 1-4 weeks long with daily, face-to-face Scrum Meetings to discuss what each person did the previous day, what they plan to do today, any roadblocks, and an estimate of time remaining. These meetings run less than 15 minutes, and provide both direct feedback into the current sprint and indirect feedback to the product backlog, helping determine which features move to the next in the sprint backlog. At the end of each sprint delivery of working software defines success. The team conducts a sprint review to demonstrate the software, to learn from successes and mistakes and to incorporate that learning into the next sprint planning meeting. Each step of the process works together to give the customer control of the whole project - not just the very beginning of it. Daily Scrum Meeting Indirect Feedback 24 hours 2-4 weeks Wish List User Stories Product Backlog Sprint Planning Sprint Backlog Potentially Shippable Product Increment Direct Feedback Process Improvement Sprint Review Customer Experience Developer Experience Agile Values The writers of the Agile Manifesto developed several principles summed up in the following values: Communication. Collaborate and talk face-to-face. Proximity matters. Simplicity. Do the simplest thing that will work. Simple rules lead to complex behavior. Complex rules lead to [simple] behavior. - Dee Hock, founder and former CEO of VISA Respect. Care about each other. Care about the project. Courage. Honesty results in effective action, fear of bad news results in paralysis. Feedback. The sooner we know about something, the sooner we can adapt and improve.
For complex projects Agile methodology offers numerous benefits to the software development team and to the customer Agile Illustration Agile is the perfect development methodology for creative projects, mission critical projects, new technology introductions, new process designs, and projects with poorly defined goals. Science moves at such a rapid pace that it is almost impossible to know everything you need to know before the start of a project. Compounding this factor, science and technology often change in the middle of a project. In a waterfall project you must include any innovations at the start or wait until launch of the next project. With Agile, however, you can adapt to innovations comparatively on the fly gaining the advantage of new technology when you need it most. Bluware s work in the following case illustrates Agile on a real project. The project: Spectral decomposition plug-in component The team: Sprint duration: 6 people; both full and part time four weeks (variable to coordinate with release dates of other software). Number of sprints: 12 Project duration: 12 months Project description: The project goal was to develop a software component that performed spectral decomposition on seismic data. The software used single trace for selecting algorithm and parameters, with rapid computation speed, and then performed computation for an entire event to do quality control over the entire surface. Finally, spectral decomposition was done over an entire volume. This stepwise progression meant less time wasted processing volumes with the wrong algorithms. The customer s research group had developed multiple specialized algorithms to allow for different types of spectral decomposition. Bluware converted these research computations from MATLAB to industrial strength C++ for use outside of the research environment. The team developed a full, interactive user interface with substantial graphics to allow users to select appropriate algorithm and associated parameters, and to visualize quality control results. Special challenges: This was the customer s first Agile project, especially for Product Owner. Moving to a new development methodology created some fear of project failure for the client. This fear was overcome by a significant education process, longer sprints tending toward 4 weeks rather than 1, and ulti- mately successful delivery. The Product Owner doubted that Agile would work and was concerned about a loss of control. Is- sues were resolved through a highly responsive development team and the delivery of functional software at the end of each sprint to allow the Product Owner to offer feedback on the adjustment of algorithms and the user interface. In a traditional waterfall method, this input would not have been possible until project end. Scalability from 1 to 2 to 3 dimensional computed regions result in associated performance degradation. The multiple, short sprints inherent to Agile resolved the dimensional scale issue. The Product Owner was able to discover the issues during development and optimize algorithms and multi-threading application in later sprints after key functionality was delivered.
The software component needed to plug-in to multiple versions of product. Bluware resolved this issue by working closely with other product development teams, and maintaining a flexible architecture. Requirements like the choice of algorithms, stability issues and performance were not fully under- stood and were still being worked out during the early phases of the project. Agile methodology resolved this issue with the team significantly altering the third and fourth sprints to focus around these customer issues, allowing new features to be put in front of other planned features from the backlog. The Agile method was responsible for both uncovering and resolving this challenge. Agile Benefits For complex projects Agile methodology offers numerous benefits to the software development team and the customer commissioning the creation of the software. Improved control Users create wish lists of requests that Customer gathers and high grades. Customer directs what features are put into development. Customer prioritizes feature enhancements. Customer adds or deletes features from product backlog as priorities change. Customer participates in daily scrum to monitor progress, obstacles, risks, and changes. Customer receives working product at end of each sprint to inspect and deploy. Faster results Working software results from every sprint. The most critical features are delivered first. Numerous, small, reversible decisions are made quickly, keeping team moving forward. Higher quality Quality is incorporated at each iteration, not bolted on at the end. Feedback is closer to real-time Less waste Unnecessary features are eliminated early. Less time is wasted finding people and information. Scope creep is replaced by flexible development that is responsive to ongoing changes. Reduced time is spent producing dead documents in favor of living artifacts. Release process is simplified (i.e. testing, coding, PM, release engineering, etc.) Better product Frequent, structured communication with Product Owner means the end result is closer to customer vision. Change is welcome, so finished product incorporates ideas that arise after development launches. Developers are free to think creatively throughout the project, which yields better solutions.
Lower risk The process ferrets out failure early before a lot of effort is invested. Risks are brought forward rather than delayed reducing the cost of making adjustments. Complexity is mitigated by breaking down large efforts into simpler mini-projects called sprints. Early and regular process improvement and course correction occurs through frequent inspection. Risk Seriousness Inception Waterfall Iterative Waterfall Integration & Test Period Elaboration Construction Time Requirements Design Implementation Testing Reduced cost Customer invests only in features that are valuable because the Product Owner is involved in the daily Scrum Meetings. Quick delivery cycles allow client to capture value early to offset project cost. Waste reduction reduces time spent on low value activities. Fail early, fail often reduces costs of errors.