Agile Challenges: Achieving self-organizing status and adopting CMMI principles

Transcription

1 Agile Challenges: Achieving self-organizing status and adopting CMMI principles Sérgio Freire Master of Software Engineering 2011/2012 Universidade de Coimbra / Carnegie Mellon University sergioaf@dei.uc.pt Abstract Agile Methodologies are one of the most popular software methodologies in the present day. The focus on team work and interactions between people is a clear differentiating factor from traditional approaches. The team should self-organize, maximizing productivity between its members. The fact that Agile focuses more on the people factor is also often confused with the fact that processes should be neglected. There s a tendency to oversimplify them or not having processes at all, which is a common misconception. This paper is a reflection on the experience of a team when adopting Scrum. It has two major goals: reflect on issues encountered when moving from a hierarchical approach to a self-organizing one and analyze the implications of using Scrum while adopting CMMI principles. Examples are provided to explain those issues and how we tackled them. We were able to improve considerably time spent making decisions and the capability of our processes. The causes for each issue are explained and the main lessons learned can be summarized as following: self-organization doesn t mean reaching a consensus with the entire team but it does assume commitment; the team should assign a responsible per process area to ensure tasks not related with the product are done; using Agile in a CMMI setting can be possible and one can even complement the other. 1. Introduction This paper is a reflection from a project of the Master of Software Engineering (MSE), a joint program from Universidade de Coimbra and Carnegie Mellon University. Both goals from this paper can be summarized as an assessment of the first Agile value: Individuals and interactions over processes and tools. Thus, this reflection focuses on answering the following questions: How can a group of individuals interact to maximize productivity? Should we and can we implement more heavyweight software processes while still using an Agile framework? 1.1 Agile Methods Agile methods have becoming increasing popular over the last decade. The main motivation for their popularity is the emerging need to quickly adapt software projects to change, which implicates a decrease in the weight of traditional development processes [1]. It s important to specify two main levels of change. First, in terms of requirements: a higher volatility may prove to be hard to manage with a plan-driven methodology. The increase of mobile and Internet-based software projects help explain this emerging trend. Both environments are known to change rapidly. Second, in terms of people: staff turnover can become a problem if knowledge is fragmented and scattered amongst different people. Solving this issue using documentation can be a very expensive approach, particularly in volatile settings. Maintaining paperwork in such context requires a high discipline and effort in order to keep all the information updated as requirements change (from specification to design). As such, the cost of moving information between people must be reduced [2]. It is also pointed out that traditional approaches were considered by many as missing the people factor and were dehumanizing [3]. P (etreme Programming) is cited as the starting point for Agile Methods [1], which became worldwide known after the Manifesto write-up [4] in The agile value propositions were summarized by a group of well known minds (Jim Highsmith, Alistair Cockburn, Kent Beck, Ron Jeffries ) as following: individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, 1

2 responding to change over following a plan. These values created a clear distinction between Agile Methods and more traditional process frameworks, generating a new path for software development. 1.2 Project and Team (ConnecTeam) Since project context and our team s background had an influence in our Scrum adoption, it will be presented a description of both in this section. Our project was developed for the largest IT Company in Portugal. Their business goal was to collect and communicate project information in a uniform way. At a technical level, we had to integrate three different pieces of technology into one solution: Microsoft Team Foundation Server 2012, our client s communication portal based on Microsoft Sharepoint Server and Microsoft Project Server. Our team is composed by 7 members, all from Portugal. Experience in the industry varied from 5 to 10 years. For the construction phase, we ve chosen an Agile methodology: Scrum. Amongst the reasons why we chose Scrum was the need for client feedback, the requirements volatility and the focus on team communication (an issue identified in previous semesters). We had two week sprints and a total of 6 in this phase. This project had year and a half planned duration, spanning from September 11 till December 12. However, the construction phase ended by middle August 12. This is related with our decision to map MSE semesters with project phases. We used Fall 11 for Requirements Elicitation, Spring 12 for Architecture, Summer 12 for Construction and Fall 12 for Maintenance. 1.3 Reflection Structure This paper is organized in two main sections and a conclusion. Each section is divided into three subsections: 1) an introduction, 2) a description of issues encountered and 3) how we solved them. The description of each section can be found below. Section 2 contains a reflection regarding our own experience when trying to implement a selforganization approach. In Section 3, the topic Agile and CMMI is going to be covered. I will explain problems the team faced and our approach for solving them. In the Conclusions section, a summary of all the major lessons learned is presented. 2. Self-Organizing Teams 2.1 Introduction One of the key practices from Agile requires the team to self-organize: The best architectures, requirements, and designs emerge from self-organizing teams [5]. This concept has appeared in previous studies that go as far as the 50 s [6], when the coal miner industry used this approach. Self-organizing teams manage their own work and decide to which tasks they are going to commit. The team also decides who does what, which means that there s no pre-assigned leader (or project manager) assigning tasks to people. However, this doesn t mean that there is no delegation of tasks or that the team needs to decide everything as a group. What does mean is that leadership amongst such group rotates. Depending on the decision or the situation, the authority is placed in a particular team member or subgroup [7]. Amongst the characteristics of self-organizing teams is mutual trustiness, common focus and respect [2]. If successful, these teams are more effective than the typical hierarchical approach [8]. Two main reasons explain this improvement: 1) team members participate more and are more committed to the project and 2) team members are more motivated and desire responsibility [7]. As a consequence, the team is more creative and a helping behavior can be more often seen. This, in turn, will eventually lead to higher productivity and quality [9]. However, there are no guidelines to reach this level of dynamics within a team [8]. Studies about this particular topic are scarce. There is, however, a recent paper that tries to understand how a team should organize itself when moving from a traditional approach to Agile. Hoda et. al [10] concluded that when establishing a self-organizing team, its members assume informal roles that facilitate the process change. One of the most important roles is the mentor that is typically played by the Scrum Master. According to the same article, he guides and supports the team initially, helps them become confident in their use of Agile methods, and encourages continued adherence to Agile practices. It is similar to the definition of an Agile coach, an experienced person that helps the team adapting to a new concept. What we found in our experience with Scrum was that the absence of such role, the desire to reach consensus in each time decision, the type of project and our own background were some of the factors that created obstacles when trying to reach a self-organizing status. 2.2 Self-Organizing Teams: Pitfalls When we made the decision of using Scrum for the construction phase of our project, we discussed as a group if we were to have specific roles per person (as we did in previous semesters) or if we were only going to use the Scrum roles (Scrum Master, Product Owner and Team Member). We opted for the latter. This meant that, contrary to our previous experience, we were not going to have people responsible for relevant processes, such as planning, quality or configuration. 2

3 There were three main reasons that led to our decision. First, we wanted to increase the flexibility within our team. We noted several times that some process owners had overworked during the first semesters. As a consequence, some tasks didn t get made on time or people just felt that they contributed more than the rest of the team. The following graph is an example from the first semester. Team member 7 spent approximately 40% more hours in the project than team member 6. member, all of us had poor if any experience using an Agile methodology. We wanted to experiment the difficulties of using such a different method and learn ways of overcoming it. During the first sprints, it was clear that our approach was not working. Our team fought constantly to reach consensus and come to decisions that each team member felt committed to. In the following sections, the most important causes for these discussions are described. Aiming for Consensus Democracy is a device that insures we shall be governed no better than we deserve. George Bernard Shaw Figure 1 Number of Hours spent per person during the 1 st Semester By not assigning people responsible for processes, we wanted to ensure that if someone had extra work, we could easily shift the responsibility to another team member. This type of work methodology is often called as backup behavior [8]. We also used the same approach when splitting work in our team. We didn t assign people responsible for a particular component of our application, despite the fact we were working with three different technologies. We wanted to take most advantage of team flexibility. If a component needed extra work, we could allocate an additional team member to work there. The second reason was related with team feedback. We thought that discussing ideas with all team members in order to reach a decision would help achieve the best outcome. Having different points of view is something that we considered valuable, as we see in expert judgment techniques (e.g. Planning Poker, Wideband-Delphi). By not pre-assigning people for executing tasks or make a decision regarding a particular process, we could solve those issues as a group, trying to reach a consensus. We also hoped to increase commitment in our team, since everyone s opinion was heard and the decision needed the agreement of everyone. The third reason was related with the learning experience. We wanted to learn the difficulties of using this selforganizing approach, which was very different from our professional background. With the exception of one team During the Summer semester we started developing the project code, which meant that we needed new Quality Assurance and Configuration Management processes. We were also using Scrum, which implicated that Requirements, Plan and Risk Management needed to be adjusted for this new methodology. It is worth mentioning that Agile methodologies like Scrum are more focused at the project management level, which means that the largest majority of these processes didn t have any particular indication on how they should be implemented (only Requirements and Plan are explicitly covered). Deciding on their definition is up to the team or to the organization, if that s the case. During the first weeks, we had meetings with the entire team to discuss how a particular process should be executed. We started having long debates of what should be our approach. Despite the fact they were the most common discussion subjects, processes were not the only topics we argued. We also discussed as a group how we should develop a particular user story, how we should conduct the testing for that story or even how we should solve client requests. All of these team discussions eventually led to disagreements which eventually led to conflicts. Ultimately, this choice of involving everyone in decisions took a toll in our schedule. The following chart contains the time spent in meetings during the first sprint. Figure 2 Total time spent in the First Sprint 3

4 These meetings include planning, daily, review, retrospective and other team meetings with the sole purpose of taking decisions (for example, how we should approach a particular client request). Average meeting time was 2.2 hours. The largest chunk of this time belongs to the planning meetings, where conflicts were more frequent and there were more topics to be discussed. What we did wrong: We essentially made three errors: 1) we didn t let people being responsible for a particular decision; 2) we struggled too hard to reach a consensus and 3) we didn t ensure that everyone understood clearly the details of each decision. 1) We failed to apply one of the basic principles from self-organizing teams. While it is true that the team is responsible for making decisions, it is also true that there must someone or a subgroup of the team assigned for making those choices. In self-organizing teams, these groups or responsible appear naturally when something needs to get done. It is important to note that team members must be willing to voluntarily accept responsibility to a particular issue. This may be another challenge by itself. We can apply the same lesson in planning meetings. While it is true that everyone in the Scrum team should get involved in those meetings, taking responsibility for a particular user story might be more productive. This doesn t mean that the team can t provide input or that the rest of the team doesn t know what is going to be implemented. What does mean is that not every decision should be reached as a group. 2) Receiving input from the entire team (we were 7, which is a considerable number) may not be as beneficial as expected. There will always be a point of disagreement and it will be very time-consuming aiming for a consensus. The team can even be blocked by conflicts that don t allow reaching decisions. It should be referred that this new flat hierarchy played a role in this issue. There was no leader for making a decision (not a project manager or a process owner) and we had equal authority and knowledge on a particular subject, only different opinions. Lencioni summarized our problems with the following sentence: Great teams understand the danger of seeking consensus, and find ways to achieve buy-in even when complete agreement is impossible [11]. We focused too much on consensus and less on complete agreement. 3) We also should have ensured that our processes were well-defined and clearly understood by everyone. By not doing so, people often posed questions and disagreed because they had different interpretations of what was decided. By the end of each meeting, it is important to review all the decisions made and ensure that everyone is on the same page. Having an Agile coach or an experienced person that set directions or assisted our team to be aligned in this new approach could help in this case. None of our team members had the required coaching experience for this role. We can find references in the literature [10] [12] defending that this element is very important when changing from a traditional approach to a self-organizing one. Neglecting Team Results Setting a goal is not the main thing. It is deciding how you will go about achieving it and staying with that plan Tom Landry In order to realize what were our main dysfunctions, we did a Team Assessment based on Lencioni s survey [11]. This questionnaire contains 15 questions to identify 5 main dysfunctions: Absence of Trust, Fear of Conflict, Lack of Commitment, Avoidance of Accountability and Inattention to Results. For each question, team members selected one of the following options: Rarely, Sometimes, Usually and Almost Always. Each option had a particular score, from 1 (Rarely) to 4 (Almost Always). In the end, we summed the results and got a final score for each dysfunction. Full results can be seen below: Dysfunction Score Absence of Trust 7.29 Fear of Conflict 8.14 Lack of Commitment 9.29 Avoidance of Accountability 7.57 Inattention to Results 6.57 Figure 3 Lencioni s Team Assessment Results Summer Scores between 6 and 7 indicate that the dysfunction is probably a problem in the team. Inattention to Results had the lowest score and it was, in fact, an issue. This is something that we have seen in other reflections from the MSE. Each person has different values and priorities and it is up to the team to ensure we are all going in the appropriate direction. In our case this effect was seen in two situations: 1) when selecting tasks and 2) when executing them. In terms of task selection and since each person chose what they were going to work next (as self-organizing teams do), there were some important activities that were left behind. Once again, this was most visible during the first two sprints. Tasks related with Processes (Quality Assurance, Risk Management or Configuration Management) were overlooked because each team member gave more priority to development activities. However, and despite their importance, there were other activities that were at least as 4

5 important to the project. The team was aware of this, but each member was waiting for someone else to pick those tasks. This behavior is often referred as Social Loafing [13]. When executing tasks, people often kept difficulties encountered to them. Despite the fact we presented our issues during daily meetings, we rarely observed a behavior of accepting other people s contributions. Often, people didn t call for help when facing difficult problems. Looking back, it s interesting to observe that we decided many things as a group but in terms of construction tasks we had a high degree of individual autonomy. Neither of them actually contributes for the self-organizing approach. This type of behavior is closer to our own background, where the project manager assigns tasks for each team member that accepts that task as his/her unique responsibility. Ultimately, our team frequently had user stories that weren t finished by the end of the sprint. It wasn t solely because people didn t ask for support, as there were other factors involved (e.g. underestimations), but the fact that this became recurrent affected us. Clearly, it wasn t helpful for our motivation in the long term. We were only able to build a momentum in the last couple of sprints. This also helps explaining our results in terms of Lencioni s assessment. What we did wrong: to summarize, we did two things wrong: 1) we only looked at the short-term and the development tasks that were at hand and 2) we allowed that our individual goals prevailed over a team goal. 1) In spite of organizing user stories per priority, we neglected tasks related with processes. It s easy to forget about other tasks when the focus is on development. We prioritized user stories but we excluded process tasks from that prioritization. This division only contributed for letting important activities to be forgotten. We are not the first MSE team with these issues. A well known paper about explicit Risk Management in Scrum also mentions the same issue [14]. As mentioned before, despite the fact we were aware that this could happen, we still wanted to try. We were hopeful that we were able to overcome that behavior but we found it hard to do so in just a couple of months. People always tend to focus on code-related tasks because that is what matters by the end of the sprint: show our advances to the product owner. 2) We also failed to establish and commit to a clear team goal. The fact that Scrum promotes short sprints actually creates a shared goal: finish all tasks before the end of the iteration. However, the team must be fully committed to achieve it, or else that goal will be no more than a statement. In our case, the individual goal (complete my task) overcame the team goal (finish all tasks as efficiently as possible). The team rarely united to solve a problem that someone was having ( helping behavior ). Not because people didn t offered to help, but because each story was taken as an individual goal and not as a team goal. This explains the attachment people frequently had to a particular task without asking for help. Additionally, if a team fails to finish all user stories in the first sprints, then it may stop considering it a problem, but something that became part of the project. People start a new sprint feeling that they are not going to end stories by the end of it because that was what happened in all other cases. The team must fight for that objective as a team and not based on individual efforts. Adjusting to Expertise You must continue to gain expertise, but avoid thinking like an expert. Denis Waitley After solving the issues described so far, we still had a problem regarding our flexibility to conduct tasks. Scrum assumes that an Agile team is cross-functional. This means that, in theory, everyone can do any task that is in the product backlog. Again, our project had, at least, three different technologies to look at. We also used two different pieces of legacy code that needed to be understood in order to make changes. Since all of these technologies and legacy ware was complex, it was impossible for each team member to understand all of them. We found that flexibility comes with a cost. If we allowed that everyone gained knowledge in all the different components of our application, we wouldn t be forming experts within our team, losing productivity. If, on the other hand, we had too many experts we would lose the opportunity to allocate more people in features that had higher priority. In the end, the team naturally organized itself by component. We had separate people working in different components. This separation wasn t imposed. It was something that happened naturally as team members started working in a particular story. We eventually started developing experts in components of our solution that typically worked with one or two technologies at most. We also ended up having people responsible for each piece of legacy code. However, this level of expertise had an impact in Scrum ceremonies. Differences in terms of tacit knowledge were especially blatant during our planning meetings. For each feature, only a few team members actually knew the effort involved. The rest didn t even have the basic knowledge to estimate what needed to be done. When applying Planning Poker, their estimation values were based on the opinion of others, the experts. Adding this difficulty with the fact that each team member usually worked alone (as explained in the previous section), it is easy to conclude that we didn t 5

6 have a complete cross-functional team. And was it indeed a problem? What we did wrong? In fact, the real problem was trying to have a cross-functional team with all the different technologies, legacy code and components our solution had. There are contexts where it may make sense separate a team according to the different modules of a system and conduct a Scrum of Scrums. In our case, it didn t make sense having planning meetings with all team involved. Our project was based on integrating several different pieces of code and technologies. Most of our discussions revolved around technical aspects of this integration. People can only add something to the discussion if they understand the issues involved. It was frequent that in our planning meetings where 7 people were involved, only 2 or 3 members added something to the discussion of a user story. These members were the ones that had enough knowledge to discuss the implementation. This also affected our estimation process, as already discussed. We should have faced sooner the fact that to be more time effective, we needed to separate ourselves per component. However, it is debatable whether daily meetings should have involved the whole team. We still found important to know what everyone s doing. Cross-functional teams are a great concept. However, as in any other software engineering concept, it has some limitations. There are contexts where it is impossible having team members readily available to work in any task. There are knowledge gaps and learning curves that can easily hammer that objective. Agile is known to require a team of capable people [15] but even in that case, can you really have a person that can easily shift between each technical area of a project? 2.3 Self-Organizing Teams: ConnecTeam s Approach Since we didn t have any tutor or instructor to guide us in this new adaptation, we were forced to reflect and organize ourselves again. We took an intermediate approach between a completely self-organizing team with no roles and the traditional team hierarchy. We based our strategy in the following actions: Specify roles for each person, as we did in the previous semesters; Set a new decision process: this process empowered process owners to make the necessary decisions. We would only debate topics as a group when it was a team global decision; Clearly separate people per technology, forming almost separate scrum teams. Specifying Roles Creating roles for each process area proved to be much more efficient than our initial approach. Not only because we were already accustomed to that hierarchy (we have previously used it in Fall and Spring semesters) but because we stopped debating as a group and started giving more responsibility to each person. By having just one person focused on a process area also allowed to have a more coherent vision of that process. People also knew the person they needed to contact in case of doubts or suggestions for improvement. We could afford losing flexibility in terms of tasks because of four factors: 1) by this time, it was very unlikely that a team member left the project; 2) we were a large team, at least according to MSE standards, which allowed having more people per component; 3) we had a better separation of roles, which meant that no one should do more work, as happened in the first semesters; and 4) even if a particular role had more work, someone could freely pick tasks assigned to him/her from the task board. These were the roles we ended up having: Scrum Master/Plan Manager, Client Liaison (often replaced Product Owner in decisions), Quality Manager, Configuration Manager, Risk Manager and Architect. Each person had the responsibility of making decisions for their area. This doesn t mean that he/she needed to conduct all the different process tasks alone. It was frequent for a process owner to create a group for helping him/her in a particular process or to reach a decision. But the real importance here is having someone that is responsible for organizing those meetings. It was also very important communicating the decisions to the rest of the team. The owner of each process area was also responsible for doing this. We implemented this change by the third sprint and the most visible impact was related with process definition. We stopped having tasks that were forgotten during the first sprints, reducing the Social Loafing behavior. Process owners also placed pressure on ensuring that important tasks related with their area were executed. We improved our processes considerably after moving to this approach. And, most importantly, we stopped having long debates in order to reach a consensus. Setting a new Decision Process We also set a new decision process that took into consideration the new roles we had. The process is as follows: 1. Each person is responsible to ensure the activities are conducted for their area; 2. Process owner can point people to help him in his decisions, if no one volunteers; 3. When a decision must have the agreement of the entire team, a meeting should be scheduled; 6

7 4. When a decision doesn't need the agreement of the entire team, it should be communicated through Skype. This avoids interruptions. 5. If someone disagrees with the decision, he/she tries to solve the disagreement with the process owner. They can decide to schedule a team meeting if they don't reach a consensus. When a team member didn t agree, we contained the discussion between him/her and the process owner. This way, we didn t affect team meetings with these issues. It s also easier reaching a consensus or a compromise with two people than with the entire team. However, we still needed to conduct team meetings from time to time, for more global decisions. The meeting organizer should follow these steps: 1. Set the agenda and the goals for this meeting (they should be crystal clear). Don't forget the moderator and note keeper; 2. Send the alternatives and pros/cons before the meeting in order to allow people to prepare it; 3. Explain context and present the alternatives and pros/cons. If someone has another alternative, this new option is also presented during this phase. (15 mins); 4. First voting round (5 mins); 5. After the results are obtained, ask the question "Is someone against the decision?" If someone disagrees, a time boxed discussion is set. (10 mins) 6. A new voting round is made. If there are more than 2 alternatives, only the top 2 are chosen. (5 mins) 7. The results obtained are final. Majority wins (we have an odd number of elements). meetings with everyone involved. People contributed equally and were more comfortable to estimate. Results The most visible results of taking these actions can be seen in time spent in meetings, which decreased considerably after Sprint 3. We had fewer meetings and these were shorter, in average. Figure 4 Total Time Spent in Meetings in hours/sprint (Sprint 1 to 5) Once again, the focus wasn t in reaching a consensus (does everyone agrees?) but moving forward (does anyone disagree?). Meeting preparation and time-boxing were also important factors for the success of this meeting. Splitting our team As mentioned before, our team naturally took a split between the different components of our application. We decided to conduct separate planning meetings for each subgroup (there were 2 groups, one with 4 people and other with 3). Each team would come up with their tasks and estimates to the Scrum Master that was responsible for organizing everything in the task board. We still considered important having daily meetings with everyone. People could shift from one group to the other for conducting tests, for instance. This way, we still had some flexibility within our team (if a user story from Group 1 needed more testers, then Group 2 could provide someone to help them), while still allowing for people to focus on a particular technology or component. But we stopped having long planning Figure 5 Average Meeting Duration in hours/sprint (Sprint 1 to 5) Another important improvement was related with our processes. But I ll leave this topic to be further explored in the following section. 3. Agile using CMMI principles 3.1. Introduction Agile introduced a new work methodology that was a clear division from traditional plan-driven approaches. Barry Boehm analyzed this difference, defending that none of them are silver bullets, with each approach being more effective in their own sweet spots [3]. In some cases, a combination of both methodologies can even be preferable. 7

8 To summarize the different approaches, Boehm created the following spectrum to help clarify where the agile approaches are placed when comparing to the plan-driven ones. Figure 6 The planning spectrum [3]. Looking at Figure 6, it is interesting to note that CMM (the model that gave origin to CMMI) is also placed within the range of Agile Methods. There is a common debate regarding the use of CMMI principles with Agile. Can it be done? Take RUP for example. This is a process framework that is known for its guidelines and documents. However, it can be tailored enough to belong to the group of Agile Methods [1]. The short iteration planning and the requirements activities for embracing change follow some of the Agile principles. It s just a matter of specifying which documentation is essential and tailor RUP enough to turn some of its processes more light-weight. The same tailoring can be applied when using CMMI or other process improvement initiatives. It s just a matter of fitting those principles in an Agile context. CMMI specification often refers to expressions as as needed or adequate. The goal is always to use those best practices according to the context. SEI was also aware of the emergence of Agile Methods and included in CMMI s most recent version notes for those approaches [16]. In fact, Agile and CMMI can complement one another. For instance, Scrum focus on teamwork and project management but there are other process areas that don t have any guidelines. There are no indications on how to conduct Quality Assurance or Configuration Management. In Scrum, Risk is an implicit process since problems and potential issues are discussed during planning meetings and daily meetings. However, under certain contexts it may make sense to create an explicit Risk Management process. To answer the initial question: yes, it can be done. And there are multiple references in the literature that prove it [17 20]. Paulk [21] even defends that P fulfill almost all of key process areas from a CMM Level 2 organization. The only process area that isn t covered is Software Subcontract Management. In the following sections, ConnecTeam s approach and issues encountered when applying CMMI principles with Scrum are described Agile and CMMI: Pitfalls In our team we had discussions regarding the definition of our processes. Since Scrum didn t had any information on how to implement Quality Assurance, Configuration or Risk processes, we needed to implement everything from scratch and adapt it to that process framework. We found three main issues when defining these processes. First, we strived too much on having simple processes. Second, we didn t have process owners. Tasks related with processes were forgotten. This problem was already discussed in Section 2 and won t be described again. Finally, we didn t provide enough time for people to adjust to new processes. We changed processes frequently. Complexity and Agility Make things as simple as possible, but not simpler. Albert Einstein We had a debate in our team of what was Agile and what wasn t. We wanted to define processes that weren t too complex to be considered Agile. This is a common misconception [20]. It doesn t really make sense to correlate complexity with Agility. If the process is effective and it s serving the team s purposes without going against Agile principles, then there s no reason to think that we shouldn t use it. In our team, during the first two sprints, we had a clear problem with our Quality Assurance. We started by having a process that had insufficient metrics. Essentially we registered the defect and its state but that was it. We weren t collecting any metrics in terms of type, root cause or even time spent. The process was very light-weight but missed important information if we wanted to improve it. For instance, without knowing how many defects we were catching per activity (code reviews, inspections, unit testing, integration testing, and system testing) how could we know their efficiency? We were only controlling historical data in terms of bugs opened, closed or active. This information was relevant at the operational level, but hardly at the strategical. What we did wrong: We made the mistake of trying to oversimplify processes. We ignored the fact that Agile is still a much disciplined methodology. All the different meetings and their strict structure, the task board, the demonstrations to the client, the roles, test-driven development all of these activities require discipline from the team. It s important to remember the sentence that goes immediately after the values in the Manifesto: ( )while there is value in the items on the right, we value the items 8

9 on the left more [4]. If we look upon the first value ( individuals and interactions over processes and tools ), we can conclude that Agile still values processes, though still being more focused on interactions between people. We changed the process by Sprint 3, at the same time we assigned people responsible per process area. Letting the process to stabilize A truly stable system expects the unexpected, is prepared to be disrupted, waits to be transformed. Tom Robbins Even when we define a new process, there s a natural tendency to rearrange it or to improve it according to our belief. After we defined the first versions of our processes, we often used retrospective meetings to make changes to them. For instance, people felt that reporting so many tasks was taking too much time. There were also other team members that thought we were adding too much detail for defect tracking, that we didn t need all that information. Others thought that code reviews shouldn t be done before unit tests or even that they were not useful at all. We made an effort to change the processes to be more simple (as it was discussed before) without actually giving time for the process to sink in. We reacted too quickly and too radically in some cases. For instance, we had a problem with our planning process as we were underestimating considerably our user stories. In the first three sprints, we used three different approaches in our planning meetings. The last one was the more successful, where each team member estimated with 50% and 90% confidence. Despite the fact we improved our estimates in that Sprint, we still wanted to change that process. It wasn t good enough as it took too much time. We were never able to stabilize the estimation process, not even by the last Sprint. We made changes in each retrospective meeting. What we did wrong: We knew we didn t have much time to apply the perfect process, but we probably reacted too quickly to our own problems. Agile Methods promote a very dynamic environment, with the team frequently discussing risks and issues and always trying to make improvements to the product or to teamwork. We had retrospective meetings every two weeks where these topics were discussed. Making changes and improving our own processes is a good thing, but doing it frequently in each retrospective meeting won t let the process stabilize. There must be a balance between something that clearly needs improvement and something that is just part of the process implementation. We should have taken the word Agility only in terms of our product, not in terms of our processes. Process owners or even the Scrum Master should be the ones held responsible for ensuring that this stabilization occurs. It s interesting to note that Deming also called this a typical error from management, interfering in processes without anything requiring to do so [22]. We, as a self-managed team, ended up suffering from the same issue Agile and CMMI: ConnecTeam s Approach There was a shift in terms of processes from Sprint 2 to Sprint 3. We assigned people responsible for process areas by creating new roles. This was a major change. These process owners started working heavily on process definition, trying to improve them based on the information we had so far. The largest improvements were in terms of Risks, Quality and Configuration. Despite the fact we changed some of our processes during retrospective meetings, we managed to stabilize almost all of them. As a team, we understood that this was the most efficient way. Our Quality Assurance process is a good example of how to implement a more heavy weight process in Agile. We started out with a very light-weight process that didn t provide much information. We knew we needed to improve our metrics or else we wouldn t be able to assess if we were doing what was needed to ensure quality in our product. Our Quality Assurance Manager sat down with another team member to define a new process. We used GQM (Goal, Question, Metric) to specify the metrics we needed. We added new fields for defect tracking: root cause, defect type, finding activity, time estimate for solving it, actual time to fix it. This way we could validate what were the activities that were more cost-effective. Figure 7 Example of an Analysis of our Quality Activities In each planning meeting, we estimated all the tasks that needed to be done for that sprint. The estimates were made in hours and there was a limit for the amount of hours we could execute in each sprint (which is sometimes called team capacity ). The Quality process owner created the tasks to support this process (for instance, analysis of metrics, validation of data). Estimates from these activities were based in historical data. There was no need for the entire team to estimate them using Planning Poker. The 9

10 same method was applied to Planning, Configuration or Risk Management activities. Planning meetings were only used for estimating user stories. Typically, the Quality Assurance manager was the one executing support activities for that process. However, all the tasks were placed in the task board and if anyone was available, he/she could pick that task, informing the process owner. In terms of quality activities, such as unit testing, integration testing or peer reviews, these were also created during planning meetings per user story. However we estimated those activities as a percentage of development effort. During retrospective meetings, we started using data collected to discuss what needed to be improved. It was a team discussion but based on metrics from each sprint. There was no need to arrange specific meetings to discuss quality issues. Scrum already provided us the place to discuss them. As it is possible to see, we ended up having a Software Process Improvement typical approach, based on CMMI principles in an Agile context. It was particularly useful when using Scrum, since this process methodology doesn t have any guideline in terms of Quality Assurance. Results We conducted a small, informal CMMI assessment during Sprint 6 (the final sprint). We selected two points in time: Sprint 2 and Sprint 5. Sprint 2 was the last sprint with team members without roles. Sprint 5 was the most recent one that had finished. For each process, we validated if we fulfilled the goals. The practices described in the CMMI documentation were used as a reference to validate if we achieved a particular goal. For each practice we stated if it applied to our context and if we were following it or not. It was a binary decision. In the end, if we followed all the different key practices, then we reached the appropriate Capability Level. Since we are not part of an organization, Level 3 was achieved as soon as we achieved Level 2. There was no organizational standard process [16] and we didn t need to tailor it to the needs of the project. Please take note that this was an informal assessment just to validate if we were completing the goals. This is, by no means, a SCAMPI evaluation. Yes No N/A SG 1 Establish Baselines SP 1.1 Identify Configuration Items SP 1.2 Establish a Configuration Management System SP 1.3 Create or Release Baselines SG 2 Track and Control Changes SP 2.1 Track Change Requests SP 2.2 Control Configuration Items SG 3 Establish Integrity SP 3.1 Establish Configuration Management Records SP 3.2 Perform Configuration Audits Figure 8 Example of an Assessment for Configuration Management Process The results can be found in the chart below. Figure 9 Process Capability from Sprint 2 and 5 As it is possible to see, we greatly improved our process definition after solving our problems. The only one that didn t changed was Configuration Management, essentially because we didn t implement audits to our repository, despite the fact we felt we needed to conduct them. That was not part of our process, though. 6. Conclusions At the time of this writing, I m nearly completing my learning experience in the MSE. When I return to my organization I will have a list of lessons learned to be remembered when adopting Agile. I think they are all going to be useful in an organization that is still taking their first steps in these new methodologies, as our team was. The following list summarizes the main conclusions from this paper: Changing from a traditional approach to a selforganizing one isn t simple or quick. It is reported that organizations take from 1 to 2 years to make this change [12]. We tried to apply it in a smaller setting for a couple of months. Creating mentoring programs for Agile teams with the goal of reaching self-organizing status more effectively may accelerate this process; Don t aim for reaching consensus in each team decision. Team members must be willing to assume responsibility when issues emerge. A person or a subgroup of the team should be responsible for their outcome. Ensure the rest of team commits to those choices. This is the self-organizing way of doing things; If important tasks are left behind, put people responsible for those areas. This is more frequent in tasks that are not directly related with the product. This is also a key factor for successfully implementing a process in a team; Don t assume that an entire cross-functional team is always attainable. Use separate scrum teams in projects with a high degree of technological expertise; 10

11 Agile assumes that there s a need for discipline and processes. Keep things as simple as required, not simple just for the sake of being light; Retrospective meetings are meant to discuss processes and improve them but without causing disruptions. Agile and CMMI. Can it be done? Absolutely. There are no impediments to it and they can even complement one another. We ve come a long way since the first day we started using an Agile Method. Scrum is easy to understand but very hard to be executed. It requires discipline, commitment and always a good level of motivation. In the end, we overcame all the issues we encountered as a team. But it was, indeed, a challenging journey. 7. Acknowledgment I want to thank my mentors for all the support and direction given throughout this program, particularly during our Studio. They are: David Root, Ipek Ozkaya, Marco Vieira and Raul Barbosa. I also want to thank my individual advisor, Bruno Cabral. I want to thank my team for all the friendship and stimulating conversations we had during the last year and a half. I never worked with such a talented group of people before. Finally, I want to thank Novabase for providing the opportunity of entering this course. 8. References [1] P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta, Agile software development methods, VTT Electronics, [2] A. Cockburn and J. Highsmith, Agile software development, the people factor, IEEE Computer, no. November, pp , [3] B. Boehm, Get ready for agile methods, with care, IEEE Computer, vol. 35, no. 1, pp , [4] A. Cockburn, J. Highsmith, R. Jeffries, and R. C. Martin, Manifesto for Agile Software Development, pp. 2 3, [5] K. Beck, M. Beedle, and A. V. Bennekum, Principles behind the agile manifesto, Retrieved, pp. 2 3, [6] E. Trist, The evolution of socio-technical systems: a conceptual framework and an action research program, Ontario Quality of Working Life, [7] N. B. Moe, T. Dingsøyr, and T. Dybå, Understanding Self-organizing Teams in Agile Software Development, no. 3, pp , [8] N. B. Moe and T. Dingsøyr, Scrum and team effectiveness: Theory and practice, Agile Processes in Software Engineering and, no. 7465, pp , [9] M. Fenton-O Creevy, Employee involvement and the middle manager: evidence from a survey of organizations, Journal of Organizational Behavior, vol. 19, no. June 1996, pp , [10] R. Hoda, J. Noble, and S. Marshall, Organizing selforganizing teams, Proceedings of the 32nd ACM/IEEE, [11] P. Lencioni, The Five Dysfunctions of a Team. John Wiley & Sons, [12] N. B. Moe, A. Aurum, and T. Dybå, Challenges of shared decision-making: A multiple case study of agile software development, Information and Software Technology, vol. 54, no. 8, pp , Aug [13] B. Latane, K. Williams, and S. Harkins, Many hands make light the work: The causes and consequences of social loafing., Journal of Personality and Social, vol. 37, no. 6, pp , [14] C. Nelson, G. Taran, and L. L. Hinojosa, Explicit risk management in agile processes, Agile Processes in Software, pp , [15] J. Rockwood, Choose Your Weapon Wisely, Carnegie Mellon University. [16] M. Chrissis, M. Konrad, and S. Shrum, CMMI for Development: Guidelines for Process Integration and Product Improvement, Software Engineering Institute, [17] P. E. Mcmahon, Taking an Agile Organization to Higher CMMI Maturity, CrossTalk, no. January/February, pp , [18] S. Cohan and H. Glazer, An Agile Development Team s Quest for CMMI Maturity Level 5, 2009 Agile Conference, vol. 2, no. Cmm, pp , Aug [19] M. Foegen, Scrum and CMMI Does it fit together?, Wibas, no. November, [20] H. Glazer, J. Dalton, D. Anderson, M. Konrad, and S. Shrum, CMMI or Agile : Why Not Embrace Both!, Software Engineering Institute, no. November, [21] M. Paulk, Extreme programming from a CMM perspective, Software, IEEE, pp. 1 8, [22] D. J. Anderson, Stretching Agile to Fit CMMI Level 3, Agile Development, Denver, no. July,