Maintaining Quality in Agile Environment Authors : Mr. Vasu Padmanabhan, Mr. V. Arockia Jerome Presenter / Speaker : Mr. V. Arockia Jerome Banking and Financial Services, Delivery Excellence Group (DEG) TATA Consultancy Services, Chennai, INDIA 1
Description Agile methodologies and Agile frameworks in software development projects has become common practice over the last few years. Though Agile approach fosters collaborative and proactive behaviors by team members and everyone is responsible(or has the ownership) for the Quality of the Product/Process, there is need for improvement in project planning and monitoring & tracking project performance to deliver high quality product on time within budget This presentation showcases improvements made in project planning, story point estimation and ways to Quantitatively improve project performance: a) Project Planning Usage of statistical tool called Sprint Planner helps to predict the number of user stories(comprising of varying number of story points) that can be completed within the planned duration of a sprint with confidence intervals b) Story Point Estimation Establishing Workmanship Criteria brings in uniformity in the story point estimation across the teams and enables the agile teams to estimate in a uniform manner. Usage of Workmanship Criteria allows productivity comparison among the scrum teams of the same project c) Project Performance Enabling agile projects to achieve the established quality and process performance objectives, without losing the feeling for people, collaboration, and interactions that underlie all agile processes by usage of Quantitative Management techniques for effective monitoring and controlling of the agile processes 2
Learning Objective Look into Challenges with Agile Quality & Adoption of a Structured Approach (Agile Testing) leveraged through CMMI practices & enablers (Tools/ Techniques) for better predictability in Agile environment Planning 1 Exposure on the usage of statistical techniques / models for Sprint Planning 2 Technique for productivity comparison among the scrum teams of the same Productivity project (Workmanship Criteria) CMMI Practices 3 Exposure on the usage of statistical techniques for effectively generating and managing the impediments using Control Charts Applying QPM / CAR / OPM practices in Agile Projects 4 Agile Institutionalization Institutionalization of Agile in Organization through Agile Testing / Governance / Tools that puts Quality at the Top of the Stack 3
Agile Quality 1 Scope Creep / Delay (Backlog not ready at beginning of Sprint (Definition of Ready) 2 Inadequate time to prepare test plans 3 Minimal requirements documentation to prepare the test cases Challenges 5 4 Highly compressed test execution cycles Minimal time for regression Testing. Software not tested at end of Sprint (DoD) 6 Change of role from being a gatekeeper of quality to being a partner in quality 7 Often required to play a semi-developer role. Lack of support (Client / Customer / End-user support) 4
Why do we have these Quality Challenges when we are Agile? 1 Planning and Tracking 2 Measurement Missed Key and successful Practices 3 Product Quality through Process Performance 4 Institutionalization (Organizational learning / Improvements / Best Practices) 5
Improvement in Project Planning Challenge Solution Presence of Scope Slippages by Agile teams for majority of the Sprints. This arises the need for Statistical tool for effective Sprint planning. Sprint Planner is a statistical tool which predicts how many stories comprising of varying number of story points can be completed within the planned duration of a sprint. This will help the projects to choose the right number of stories for each sprint. Users can specify the confidence level at which they want to check if the given number of stories for the sprint can be completed Users may also specify the minimum and maximum possible effort for each story based on their experience; this will make the model prediction more accurate Benefits Sprint Planner Model has helped Agile teams of BFS vertical to decide on the User Stories commitment for an Iteration based on the completion possibility Simulation indicates that there is greater possibility that 95% User Stories can be completed based on the estimated Story Points Effective deployment of Sprint Planner Model has enhanced the ability to burn out planned scope in each iteration. 6
Example for Usage of Sprint Planner Scenario 1 With estimated sprint effort of 320 PD, only 7 of the 8 stories can be completed with a confidence of 97.5% where as all 8 stories can be completed with 50% confidence Scenario 2 With estimated sprint effort of 320 PD, and a Risk Factor of 0.6 only 6 of the 8 stories can be completed with 50% confidence 7
Improvement in Story Point Estimation Challenge Relative nature of the story point estimation paves way for differences in estimation among the agile teams. Productivity of the agile teams of the same project cannot be compared with each other Solution Sample Workmanship Criteria: Establish the workmanship criteria for Story Point estimation across the teams. This brings in an uniformity in the story point estimation across the teams and enables the agile teams to estimate in an uniform manner Benefits Workmanship Criteria provides standardization across Scrum Teams during Story Points estimation Usage of Workmanship Criteria allows productivity comparison among the scrum teams of the same project 8
Workmanship Criteria Before Using Workmanship Criteria User Stories Story Points Hours US785 5 128 US784 5 18.5 US912 8 56 US920 5 54 US896 8 52 US897 8 55 US921 5 18 US922 5 48 US911 13 59 140 120 100 80 Effort (hrs) 60 40 20 0 y = 0.65x + 49.8 R² = 0.0031 0 5 10 15 Story Points No strong correlation between the story points and the effort consumed for implementation for the user stories. Productivity of various sprint teams of the same project could not be compared After Using Workmanship Criteria Effort (hrs) 70 60 50 40 30 20 10 0 y = 3.2868x + 23.913 R² = 0.6128 0 5 10 15 Story Points Presence of Strong Correlation between effort and Story Points infers standardization across Scrum Teams during Story Points estimation.thus usage of workmanship criteria allows productivity comparison among the scrum teams of the same project 9
Improvement in Project Performance Challenge Solution Enable agile projects to achieve the established quality and process performance objectives, without losing the feeling for people, collaboration, and interactions that underlie all agile processes Use the following statistical techniques for generating and managing the Impediments: Effort Users Slippages may also / Velocities specify are the monitored minimum for and User maximum Stories possible in Sprint cycles effort for using each Control story based Charts. on their Improvements experience; this are will planned make the by model Agile teams prediction in order to more reduce accurate variations During the course of Sprint Cycle, Burn Down Charts are plotted to monitor the effort burned Prediction Intervals using Regression are plotted to check the Burn down variations within acceptable band Benefits Quantitative Management techniques allows effective monitoring and controlling of the agile process. 10
Examples for Usage of Control Charts Control Charts are used for analyzing variations in the Velocity (Sprint on- Sprint) in a Leading Organization Release - Sprint Wise Story Points Project X Sprint 1 185 Project X Sprint 2 134 Project X Sprint 3 207 Project Y Sprint 1 205 Project Y Sprint 2 218 Project Y Sprint 3 171 Project Z Sprint 1 293 Project Z Sprint 2 318 Velocity is improving compared to Previous release due to improvements in Agile Practices (Co location of BA s, Tool enablers like RTC etc.) Control Charts are used for analyzing variations in the Velocity (Sprint on- Sprint) in a Leading Organization Team velocity improved consistently 11
Examples for Analysis of Burndown Charts Improvement when compared with Sprint 1 - User Stories were finalized and not added at the last week of the Sprint Effort is negative because team has worked for more than what they have committed at the beginning New User Stories were added during the Sprint cycle 12
Applying CMMI QPM / CAR /OPM Practices in Agile Projects Understanding relationship between Work-In Progress & Cycle Time quantitatively The graph here shows the impact of the work-in-progress levels on the cycle time. The work-in-progress tasks are plotted on the left Y-axis (column graph) while the cycle time is plotted on the right Y-axis (line graph) Teams understand the relationship between work-in-progress and average completion rate Analysis : Assuming capacity remains the same, a higher work-in-progress tasks would now mean that people now have the opportunity to multitask. This in turn drives down the average completion rate and increases your cycle time. Examine root causes using Lean and Six Sigma tools. Kick start improvement efforts for the team that has the longest cycle time Two reasons why the cycle time graph spikes: Increase in work-in-progress Non-completion of any work-in-progress tasks on a given day significant variation in the cycle time values over time. Part of the reason is that project s capacity also varies. Enables increase work-inprogress Cumulative flow diagram will illustrate how smooth Agile Team flow is and how Work In-Progress affects the overall lead time Spot variances while they can still be easily addressed 13
Agile Testing Agile Testing Agile Testing Architecture 1. User feedback is instantaneous 2. No developers vs testers in Agile. Agile follows a collaborative test Model 3. Technical debt are not let to accumulate. Problems are solved before they stack up 4. Test Driven Development puts quality at top of the stack 5. Agile encourages Test Automation Write test once and configuration manage them Run frequently with a click No human input Output which can be parsed by the machine Requirements / Testing Done Iteratively 14 14
Shift required when we move to Agile Testing Quality is responsibility of the ENTIRE Scrum Team. Quality Assurance is made part of the team in Agile Quality Gate does NOT only mean System Testing but the entire application Continuous Integration will Lead to Continuous QA Progressive inclusion of System Integration test cases from sprint to sprint Prioritize Test Cases by Risk and Frequency of Feature Usage Utilization of Sprint Retrospective to identify Where in the Scrum process did failure of Quality occur (Root Cause Analysis) 15
Better Tools Agile Management Tools Rally, RTC, Version One Continuous Integration Jenkins, Cruise Control Development Tools Static Code Analyzer (Java PMD, Checkstyle, Find bugs etc ;.Net/Microsoft ReSharper) Distributed Collaborative Source Code Editing (Eclipse- Saros ; Visual Studio VS Anywhere) Automatic Testing Agile Testing Tools (Functional Testing tools, Code Coverage Analysis tools, Test Runner tools, Unit Testing tools) Issue Tracking Tools (Jira, Bugzilla, etc) Test Management Tools (HP Quality Center, Xstudio, TestLink, etc) Software Configuration Management Subversion, Mercurial, Git, ClearCase, Microsoft TFS Communication Tools Telephone and Conference Call, Videoconference, Web Ex 16
Conclusion What worked well for maintaining Quality in Agile Environment? Transition Functionality Driven Usage Scenario Driven Reactive Proactive Unplanned Rework and Overheads Planned Automated rigor and Learning Focus Procedural Milestones and Triages One Team One Goal and Single definition on Done 17
Conclusion contd Deliver to Current Priorities and Need and Know how Shortened Feedback Loops Elimination of Waste Inspect and Adapt Product Quality 18
Vasu Padmanabhan 91 44 67424924 vasu.p@tcs.com www.tcs.com Thank You Acknowledgements Special thanks to our BFS Head- Mr. K Krithivasan Thanks to the Delivery Heads and teams without whom this would not be possible V. Arockia Jerome 91-9600085149 arockia.jerome@tcs.com www.tcs.com