Lean Agile Transformation 29 th May 2013 Shane O Flynn Global Vice President of Engineering Enterprise Ireland Driving Competitiveness Dublin Castle
Who are Openet? Openet software enables new business models that innovate how people, machines, and services interact with the world s largest networks Selected Customers: Network Service Provider Focus Best in Class Policy and Charging Control 80 customers, 28 countries Over 10 Billion events per day 900+ employees, Offices in Ireland (HQ), US, Brazil, & Malaysia Founded 1999 Openet #1 in Policy Management 2011 & 2012 2
The Engineering Challenge 4 Product Lines 11 Sub-Categories ~2.5 Million Lines of Code ~300 Product Engineers 4 Development Centre s 2 Outsourced Partners Work from Home policy in place Engineering work done in 11 different time zones 3
The Waterfall Model of Software Development Requirements Design Implement Test Doesn t exist! Maintain V1.29 Requirements document that d be 29 iterations! We ve all been agile for years, we just didn t know it! 4
Our Car Crash Waterfall Project New Product Add-On developed using Waterfall-ish method Kicked Off in March 2011 Baseline target completion date was October 2011 Project ran until April 2012 100% over budget & 6 months late. Main Issues: High dependency on manual testing Lack of automated testing, functional & non-functional Attempted to do Agile but requirements written as User Stories with no Acceptance Criteria, and executed in Waterfall framework Legacy code issues technical debt / refactoring was complex & difficult to test. High risk Integration testing executed in the final Phase. Performance Testing executed in final Phase. New Product, new domain & new teams. 5
The road less travelled the path to Lean Engineering Product Management & Customer Demand is accelerating Product lines are increasing Waterfall wasn t working (for us!) Mantra - Add more Engineers Was there another way? 6
Noble Goals Get re-organised Use Lean Methods Product Quality & Delivery efficiency Increase backlog agility JIT decision making. Shorten our release cycle times. Learn Fast! bka Fail Fast! 7
Lean Software Development Lean development can be summarized by seven principles, very close in concept to lean manufacturing principles: Eliminate waste Amplify learning Decide as late as possible Deliver as fast as possible Empower the team Build integrity in Kanban is a Lean tool. Combine Scrum and Kanban ScrumBan 8
Kanban Process / Lean Software Do not send defective products to the subsequent process The subsequent process comes to withdraw only what is needed Produce only the exact quantity withdrawn by the subsequent process Level the production Kanban is a means to fine tuning Stabilize and rationalize the process 9
Lean Product Release Cycle 10
3 Major Changes for Openet Feature Teams ScrumBan & Done Time boxed Sprints 11
Feature Teams Self organising atomic units Work assigned to team, not people Long lived (12-18 months) Generally 1 Feature Lead to run the team ~3 developers ~2 QA (including Performance expertise) Co-located Generalising specialists Assigned true specialists on an ad-hoc basis 12
Scaling Feature teams VP, Delivery PMO PMO Trainer(s) Training Delivery Program PMO Architect(s), Tech Writer(s), Architect(s), Trainer(s), Tech Tech Writer(s), Architect(s), Mentor(s) Trainer(s), Tech Tech Writer(s), Mentor(s), etc. Trainer(s), Tech Mentor(s), etc. Strategic Program #n Strategic Program #2 PMO Architect(s), Tech Writer(s), Trainer(s), Tech Mentor(s), etc. Strategic Program #1 Tactical Program 13
Definition of Done Engineering Release 14
Using ScrumBan to get to Done 15
Definition of Done Done General Availability 16
Automated Testing & Test Driven Development Automated Test Systems Functional System E2E Performance Close-out Sprint 120 days down to ~60 days Target is 20 days (7 elapsed) Continuous Build Continuous Integration System Linked to Automated Test Systems Build cycle ~6 hours (from >24) Code breaks immediately visible Minutes or hours not weeks Fully virtualized environments 17
Success? Absolutely! Openet Interaction Gateway Pilot Project Delivered on-time, within budget constraints Engineering Build was actually 2 weeks early! 3 Post-ship defects discovered 2 already fixed & patched prior to being reported 3 Major changes of direction during lifecycle Target first customer changed over course of development project Initial Use Case changed significantly No impact to final timelines 18
Challenges Coordination & Roll out Education & mind-set changes Resistance to Change Communications User Stories & Acceptance Criteria Breaking things down to small sizes Estimation Distributed Teams harder Project Management Metrics 19
Metrics with Agile Phase 1 Metrics: Velocity: How many Story Points are delivered per Sprint, per Team & overall, per Program. Used to calculate future commitments to customers. Commitment Diligence: Measure of delivered work versus how much work the team said they would deliver for each Sprint. > 100% means they ve under committed originally but taken on more. < 100% means they ve over committed originally and didn t complete everything. 100% prediction accuracy. Target is between 80% - 120%. Phase 2 Metrics, (to be implemented): Focus Factor gives us a measure of disruption to the team. Adopted Work additional work taken on during the Sprint from the Backlog. Found Work additional work completed during the Sprint for items on the Sprint Backlog. Accuracy of Estimation margin of error on Story Points. Original versus Actual. 20
CM Program Metrics 60 50 40 30 CM Velocity 35.5 27 54 20 20 10 14 13.5 0 1 2 3 4 5 6 Months CM Commitment Diligence 140% 123% 120% 100% 80% 60% 40% 20% 84% 72% 55% 75% 80% Increasing Velocity & Increasing Commitment Diligence showing a very positive trend 0% 1 2 3 4 5 6 Months 21
25 20 15 10 5 0 IG Program Metrics 90% 13 IG Velocity 24 1 2 3 4 5 19 Months - Over the last 3 months, IG team sizes have decreased due to natural attrition hence decreased velocity. Looking at Velocity per person, it is on the rise. 17 13 160% 140% 120% 100% 80% 60% 40% 20% 0% Avg. Velocity per person IG Commitment Diligence 5 4 3 2 1 0 114% 119% 91% 150% 1 2 3 4 5 2.7 3.3 Months IG Velocity - Average Per Person 4.0 3.7 1 2 3 4 5 Months 4.6 22
FW Velocity FW Program 45 40 35 38 38 41 Metrics 30 25 20 15 19 10 5 0 1 2 3 4 Months FW Commitment Diligence Again Increasing Velocity & Increasing Commitment Diligence showing a very positive trend 120% 100% 80% 60% 40% 20% 0% 101% 80% 81% 69% 1 2 3 4 Months 23
Phase 1 Rollout Complete ~12 months Prior to Lean implementation, we didn t measure well. Hard to compare our improvements to our Waterfall world. Based on subjective opinion, what we re doing now is better. Entirely new way for Teams to work & deliver working software. We now have a measureable machine. Metrics will now be the basis of decisions. Metrics will now back up subjective opinions. Metrics will now support the efficient, lean growth of Engineering within Openet 24
Phase 2 Lean Transformation Expand & develop our metrics. Continuously measure. Continuously review. Continuously refine. Continuously improve. Further coaching & training for teams in specific areas. 25
Immediate Benefits (Measured) (Built in) 26
Questions? 27