Are waterfall and agile project management techniques mutually exclusive? by Eve Mitchell, PwC. 22 MARCH

Transcription

1 Are waterfall and agile project management techniques mutually exclusive? by Eve Mitchell, PwC 22 MARCH

2 Projects need to be managed to be successful Change is a ubiquitous feature of modern life. Organisations across the globe are changing their working practices and business strategies to embrace the complexity and interconnected nature of a rapidly changing business environment and a shifting global economy. New delivery models often include suppliers, customers, vendors, partnerships and even competitors. Through these changed structures and practices organisations are becoming more able to address the pressures of rapid change, global competition and increasing complexity. The traditional waterfall project management methodology assumes that a project is finite with a definite beginning and end; that projects need to be managed to be successful; and that the events affecting the project are predictable. In addition, with this traditional methodology, once a phase is finished it is thought that it will not be revisited. The strengths of this approach are that it provides clear steps for development and has a structure which allows the project to be broken into manageable stages for more accurate planning. However, its limitations have been experienced by many project managers - projects rarely follow the given sequential flow, and customers often find it difficult to state all of their requirements early in the project often resulting in dreaded scope creep, or worse the project being seen as a failure as it hasn t met all the customer s requirements. In contrast, agile project management adopts the hypothesis that we don t know everything at the start of a project, and even if we think we know, this can be subject to many changes. Agile projects are structured in order to speed up the rate of learning for the team and organisation getting customer feedback early, identifying and ironing out technical difficulties, testing early and often. Through a constant cycle of learning and adaptation the agile team produces value to the customer, and learns to do so more and more effectively at each iteration. If the principle that each project begins with many uncertainties and that learning is good is embraced, we can in turn look to find ways of learning more effectively, and to incorporate this into the project as it moves towards its end point even to a point where we reward learning as opposed to admonishing change. The principles of agile The agile methods use a variety of tactics to build a project environment that is indeed agile. There is no one set of practices that are definitively agile there are a dozen different methods that are included under this banner, each including unique practices. The Agile Manifesto describes the underpinning value system which results in the key behaviours listed below: l Early delivery of value l Client centric l Different process for delivery l Success criteria l Quality criteria So, are waterfall and agile methodologies mutually exclusive? Through working with a range of clients across a number of different sectors I have had the opportunity to observe organisations who would consider they are guided by waterfall principles unwittingly also employing agile practices. I am sure you will recognise some of these scenarios: l Top priority projects having a designated war room in order to co-locate team members to allow sharing of ideas, brainstorming, improve communications and strengthen cross-functional relationships. l Daily project team meetings to describe what happened the day before and what is planned today a home grown concept of a daily SCRUM or stand up. These daily meetings give focus to the day and bring resolution to issues quickly and collaboratively. l Teams that produce work that is immediately tested by other members of the team a continual straw man improvement system which means that the solution is continuously being improved and proven out. l A team stepping into a project that has failed where the deliverable is due to the client with no time to plan and develop fully, therefore a real understanding of what value the client is looking for and that what is given with further improvements ongoing a continual beta-testing. Obviously these don t reflect a fully developed agile programme, but they do represent fragments of where we see agile in every day project management, and represent that organisations are often willing to use agile tools in one form or other. Case study To illustrate this further I would like to share a case study of an organisation that needed to re-establish itself as a leader in a transformed financial market. It required the early delivery of business benefit from its new channel website, but when the project was planned using the organisation s standard waterfall approach it showed the requirements gathering phase would still be under way when the first go live date was required by the business, posing a significant commercial risk. The suggestion of using some agile techniques was posed as it would cater for the early realisation of business benefit through incremental delivery. Approach In order to make the move towards a more agile approach in implementation the Dynamic Systems Development Method Atern (DSDM) approach was used. DSDM is a project delivery framework which is regarded as being agile in that it uses concepts such as iterative and incremental development, dynamic change control, collaboration and on-time delivery, but was originally designed to integrate with PRINCE2 thus more palatable for waterfall organisations MARCH

3 Governance and control The organisation s governance structure was derived from PRINCE2. This methodology is particularly strong in the area of project governance as it provides an overarching structure of governance to a project by establishing a project board which directs the manager using management by exception, further enhanced by positioning the business case as the driving force of the project. However, this is not in conflict with agile as all of these strengths stay in place. The first challenge was to convince the management team to change the time for releasing funding for the project. In its current practice funding was released at the end of start-up and initiate a project. The total project funding had been agreed by the original waterfall business case, but in order to support delivery funding needed to be released incrementally. The governance board agreed to this, allowing iterative design documents at governance gateways. This allowed the project to commence exploration and engineering without knowing all the detailed requirements of the project. As an FSA regulated business it was assumed that the risk and compliance teams would challenge the methodology, as it was believed that these teams would be nervous of an agile approach as they would infer it meant less documentation and less rigour. In order to address this meetings were set up to show concerned parties that the same governance gates, structures and documents would be used the organisation s waterfall documentation were very similar to agile artefacts, just named differently. Through the use of team rooms, enterprise tools, daily meetings, predictive tools such as burn-down charts and the inclusion of the business in the project team, it was demonstrated that there was more control than on other projects. The use of a time box a four week time period equating to 100 days development and 10 days testing - was used in each iteration. Within each time box the teams would investigate, refine and consolidate their deliverable for that iteration, allowing them to fail fast and change direction in a controlled way at the end of each time box as necessary. This tool combined with visible plans which all the team were able to talk about and describe, helped gain the support of the governance teams. Of particular value was the concept of failing fast as the risk team had just audited a project that had run for three years with no deliverable in sight. Business need and buy-in In order to gain buy-in, examples of where projects had failed within the organisation due to lack of business involvement were shared with the senior team. Based on this, and an avoidance of using agile terminology, the senior management team agreed that close involvement of the business was critical to the success of the project putting the customer at the centre of the project. To introduce agile to the wider project group, two separate briefings were held to explain the differences between a waterfall and a DSDM approach. The concept of project variables and in particular the principle of flexing the features and not time, cost or quality were explained. A qualified DSDM trainer was also engaged so that the entire project team could gain a common understanding of the approach to be used, and all gained DSDM Foundation Certification, including business personnel. The first challenge was to convince the management team to change the time for releasing funding for the project 24 MARCH

4 The use of modelling dramatically increased the speed of problem solving Changing attitudes and embracing new concepts There was considerable resistance to an agile approach at the start of building the technical framework as it was perceived that everything was a must have. It was presumed by the project team that flexing the features would not be possible if there were time issues. However, once examined more closely it was clear that flexibility was possible through: l Reduction of scale (for example through redundancy of systems) l Not commissioning all elements at the same time l Not gold plating systems. Timely delivery was essential to gaining market share. The business sponsor and project manager needed some additional coaching to help them demote musts to shoulds or coulds for a time box to give the flexibility of features required in the agile method. The team were also coached on the concept of a must for the project and not for the time box to help the team de-prioritise some elements of work. As part of the initial requirements gathering the organisation used customer focus groups to understand their customers needs and to inform design. They also brought the same groups into user testing. The customer would be able to see a functioning and tested development, ready for release at the end of each increment time box. Implementation of a new technology framework required to support the project would enable full incremental delivery of web changes, supported by configuration management systems. Collaboration and communication Collaboration was seen as a challenge due to the project team being spread over three countries with different cultures, with the business users and end customers in the UK, and development houses in India and South Africa. Significant attention was paid to communication issues associated with multiple locations. To address this, the organisation agreed to the following: l Regularly flying project team members to one location l Having daily stand-up calls to monitor progress using conference call facilities where all team members were not present l Using enterprise-wide tools to allow project teams to access all project artefacts l The establishment of a team room for visiting team members and UK participants. Furthermore, the role of the team lead was assigned per time box, depending on which geography the core competency for the current part of the project development was in. Every opportunity was taken to run facilitated workshops when all parties were present in one location. This enabled rich communication within the project and with the projects customers. The project manager or facilitator chaired all of the meetings and made sure all team members contributed. If decisions or discussions started to get bogged down the team were encouraged to white board or brainstorm the problem. Use of a dedicated team room to display project management artefacts also improved communications both within the project and to the whole company. The use of modelling dramatically increased the speed of problem solving. Once the group solved a problem they would photograph the solution and share it with the team for comment / feedback around the globe. Quality Using a Prioritised Requirements List (PRL), visible across the enterprise, the solution testers were able to inform the business analyst when a requirement was not good enough to test against. Testing was also introduced during each time box with business and end user involvement with the understanding that full integration / regression testing could only happen within the quarterly release cycle. Techniques Iterative Development Iterative development ensured that development stayed on track and delivered to the needs of the business: l Wireframes for web pages informed design l User experience feedback informed requirements l Prototyping through focus groups informed design Development was categorised as functional, non-functional or user experience. Requirements were monitored against a baseline, usability was tested against customer focus groups, and non-functional requirements such as speed of response, tested prior to deployment to live and informed further engineering work. MARCH

5 Time boxing Time boxing, allied with collaborative planning and estimating sessions, was key to delivering on time and on budget. During a time box the project team regularly recreated new requirements, which were placed on the PRL. At the end of the time box the PRL was assessed to see if the teams experience would move new requirements into the upcoming time box replacing previous plans. This decision was always made by the business, informed by the team. This was a change from the organisations existing way of working. Adding or removing requirements were seen as a change of scope requiring a change request and impact assessment. Working with DSDM saved at least one manday per week by not performing these tasks. Modelling Modelling was regularly used in facilitated workshops as a communications technique, most commonly using a white board or flip-chart in line with the DSDM model. Models were: l Processes l Data models l Functional diagrams l Technical diagrams l User stories or journeys If the result was needed as a product it was digitally photographed and recreated in an appropriate software package. In some cases the images were so powerful that they were created as storyboards and used as project communications tools. If they added understanding to a requirement they were added to the entry in the PRL. Prototyping Prototyping was used from the start of exploration to help design web functionality and determine usability, from web wireframes through to fully developed pages with stubs to replicate real data. Prototyping has rarely been used in the organisation, and the increase in pace and quality of requirements gathering which would have normally been done through one to-one sessions between business and analysts was a surprise to team members. Roles Several conversations were held with the business about the required seniority of the Business Sponsor in the organisation truly being able to force open closed doors. Furthermore whether they were willing to empower other team members to make decisions The empowerment conversation was challenging as the corporate culture was hierarchical and made more difficult by the ability of many people to say no rather than yes. The turning point in the conversation came when there was a realisation that through the use of time boxing any poor decision would be highlighted within the time box cycle (4 weeks) in the time box review which they would attend. Lessons learned Open communication can cause challenges By opening up all parties to all communications and changes we created some re-work in our first engineering time box. This is because one of the offshore development teams interpreted every design change to web pages during the feeder exploration time box as a programming change and therefore work to be done. The communications channel between the design team and that specific offshore team was removed, only passing information when design was signed off. Work being done that was not on the PRL Any requirement needed to be entered on the PRL, whether it was functional, non-functional or user experience. If it was not on the PRL it was not done, and more importantly not billed. A week long gap appeared between time boxes which was a one-off due to poor planning. This was filled by exploration work commissioned by a team lead without the knowledge of the project manager, even though daily stand ups were happening during this period. To ensure this didn t happen again the service requests were re-written to call out sections of the PRL as the only source of work. In addition, the team were coached any planning needed to finalise the upcoming time box occurred in the final week of the current time box. Daily stand-ups Fundamental to the success of this project and to ongoing collaboration was daily communication with teams in three countries. At least once every two time-boxes the team leads flew to the central location for a fortnight to ensure the team maintained the closeness that had been started at the kick-off. Updating the burn-down chart In the first time box the burn down was used almost as an attendance register, demonstrating the developers days worked rather than a demonstration of functionality achieved. The teams were reminded of the need to flex, which could only be achieved by knowing what was complete and what was not. The empowerment conversation was challenging as the corporate culture was hierarchical 26 MARCH

6 l Enabling an agile ethos through collaboration and communication, allowing dynamic change and accepting that detail often emerges during the project and cannot be predicted at the outset l Promoting the use of facilitated workshops and modelling I often find the need to flex my working practices to incorporate different aspects of what I consider best practice to both fit with the organisation in question, but also to deliver value to the customer. Through combining the best features of waterfall development disciplines with agile principles it is possible to get superior results. Arguably, the purpose of any project is to deliver value to stakeholders. By embracing new and different principles and techniques and applying them in the appropriate setting to deliver value, a project will be well on its way to success. In the second time box the suppliers were amazed that the business ambassador removed some of the work to be done from the time box, rather than cracking the whip. This was a major turning point in the relationship between all parties. Outcome This was a pilot project for the organisations transition to DSDM as a method of choice and was thus open to a great deal of scrutiny. Using some DSDM techniques demonstrated a level of control not seen in the organisation before, which tipped the balance of opinion in favour of using these methods. The delivery of business benefit early in the project lifecycle meant that the company s position in the market was strengthened, meeting a significant part of the business case. Conclusions So, back to the original question, are waterfall and agile project management techniques mutually exclusive? Based on the above case study, I think they are not. Integrating DSDM into a PRINCE2 environment is relatively straightforward in that both of the approaches are constructed in a similar way. They both have similar organisational structures and thus when combining the two there is no need to duplicate or compromise, more a need to take on additional concepts and components from agile and integrate them into the structure of PRINCE2 Notable enhancements include: l Introducing the mechanisms for scope tolerance using a PRL and techniques such as time-boxing l Explicit support of iterative and incremental product development 28 MARCH Eve Mitchell