Modern Risk Management with Kanban

Transcription

1 Modern Risk Management with Kanban Eric Keep Austin Agile - March 21, 2014

2 What do we mean by the term modern? mod ern adjective : based on or using the newest information, methods, or technology Source: Merriam Webster Delivery expectation is probabilistic, not speculative Commitment is deferred Real Option Theory Performance/capability are more effectively understood by observing work metrics, over anecdotal assessment Sustainability underpins improvement Manage work as we manage the rest of our business with a eye to Risk Management

3 Speculative to Probabilistic Moneyball 2011 Move away from speculative (expert) intuition (assessing capability & estimating work) if the goal is to break the mold of traditional performance in software projects Baseball isn t just numbers, it s not science. If it was anybody could do it.. But they can t b/c they don t know what we know. Adapt or die You ve got a kid in there with a degree in economics and you ve got a scout here with 30 years of experience!

4 Eric Green CEO, Zenkata IO 20 years in industry 16 years at AMD (US, Europe, Asia) 1 year CTO lean startup years activating Scrum teams with Kanban LKU Accredited Kanban Trainer Associate with David J Anderson & Associates Zenkata founded March 2013 (left ebay post in Sept) Leverage 10+ years of learning with Scrum teams around the world Explore, apply and teach cutting-edge heuristics for lean management of knowledge work and service delivery Improve people s lives through sustainable practice and improvement culture Human knowledge has evolved beyond 1 st generation Agile

5 Kanban Self Assessment Level Novice Informed Prac1cing Prac11oner Skilled Desc Curious with lihle knowledge AHended conference, read some web sites/blogs, watched videos 1 small team < 6 months Larger teams > 6 months Scaled implementanon, several teams, designed kanban systems > 18 months Novice Kanban knowledge histogram Informed PracNcing PracNNoner Skilled

6 What is Kanban?

7 Kanban A Kanban Japanese (Hiragana) meaning: Signal Card(s) Taiichi Ohno in Toyota Production System Representing a Unit of Capacity Proto-Kanban Limiting WIP by person Recurring infinite queues Personal Kanban Task level, personal productivity, family management etc. Virtual Kanban System Proper Pull System across an integrated knowledge discovery/service workflow Kanban Method (David J Anderson) Virtual Kanban Systems Evolutionary Change Management Probabilistic Forecasting and Expectation Sustainability and Improvement Culture Lean Risk Management Framework

8 Kaizen, not Kaikaku Kaizen Kanban is about this ConNnuous, Incremental, EvoluNonary Improvement Kaikaku TransformaNon (J- Curve Effect)

9 The Kanban Method or just Kanban Is A Systems Thinking approach to service delivery in creative knowledge work An Evolutionary Change Management method A Decision Support and Risk Management framework A method without methodology Is NOT A process A software development methodology A project management methodology Prescriptive

10 Kanban Method Principles 1. Start with what you do (understand) 2. Agree to pursue evolutionary change 3. Respect current process and roles 4. Encourage leadership at all levels

11 Kanban Method General Practices 1. Visualize (work, metrics, defects, risks etc.) 2. Make policies explicit (defines the system, operational and risk management practices) 3. Limit WIP (enables delivery, quality, sustainability for people) 4. Manage Flow (knowledge discovery/value stream management) 5. Implement Feedback Loops (daily, improvement, operations) 6. Improve experimentally using models (A3, 5 Whys, PDCA, Build-Measure-Learn, Cynefin, Scientific Method)

12 Hedging : a means of protection or defense (as against financial loss) (Merriam Webster) : a risk management strategy used in limiting or offsetting probability of loss from fluctuations in the prices of commodities, currencies, or securities. ( taking odds on the Pass

13 Kanban Systems

14 Kanban System Options Dev Build Pipeline Deployment Ideas Business Case Dev (12,24) Analysis (6,12) Build Ready (3,6) Committed (1,3) Dev/ Test (1,2) Demo/ Review (1,2) Deploy Ready (0,4) Deploy A Engineering Lead Time B C Cost of acquiring Options $$$ Investment cost $$$ Transaction $$$ D E Options development Investment/committed work Deployment Commitment Point #1 Commitment Point #2 X Y Z Reject Bin Blocker Bin * * * * * *

15 Deterministic vs. Probabilistic Expectation Change Requests I don t want to take the risk of being longer than 60 days. I need a precise estimate of when it will be delivered! Mean 50 days 85% at 60 days 98% at 150 days David J Anderson and Associates, Inc.

16 Demand and Capability Analysis Demand Analysis Sources of dissatisfaction (internal and external) Failure-to-Value demand ratio Hidden work types Type, frequency and size analysis Cost of delay assessment Value assessment Options development (upstream kanban) Capability Analysis Lead time analysis by type (data mine ticketing system) Probabilistic forecasting Flow Efficiency analysis

17 Understanding Process Variation (i.e., causes of delay) Common Cause Variation I wasted a day in meetings yesterday! Bring the tail in! Special Cause Variation We can t do the build today because a 3 rd party library license expired on the server Source: hhp:// Tip: one of the largest sources of variation in an IT / development process is variation in size of work items

18 Bottleneck Management Idea Analysis Design Code ~90 ~80 ~50 ~50 Error Error Error Working Code Acceptance Test System Test Unit Test ~80 ~30 ~50 Five Focusing Steps: 1. Identify - Current CCR is System Test 30 per month 2. Exploit - Testers relieved of all non-essential tasks, extra PMs assigned to complete administrative tasks, analysts assigned to future test plans 3. Subordinate - Requirements release restricted to ~30 per month 4. Elevate - Plan to recruit temporary testing staff immediately 5. Rinse-and-repeat 2013 David J Anderson and Associates, Inc.

19 Recurring Challenges in Time-boxed Development Batch Processing inherently impedes the flow of work Coupled input/output cadences are suboptimal Commonly has big coordination and transaction costs for grooming, estimating and planning ceremonies Kanban addresses these Batch commitment is speculative and therefore carries more risk Point Velocity is an unreliable basis for understanding commitment, delivery and predictability Flaw of Averages > sprint commitment Math on Point distributions in unreliable (e.g., release burndown) Point update error Does not support expedite capability (e.g., prod defects)

20 2012 Case Study: Timebox to Pull System (1) Stopped committing to speculative estimated batches (2) Stopped doing long planning/ estimating meetings (3) Started doing frequent queue replenishment and continued to snapshot points accepted Pull system introducnon Delivery Rate 140% VariaNon 45%

21 Risk-Based Work Taxonomy

22 Cost of Delay Risk-based Scheduling Heuristics impact impact Expedite critical and immediate cost of delay; can exceed other kanban limit (bumps other work) Nme Nme impact impact Fixed date cost of delay goes up significantly after deadline; Start early enough & dynamically prioritize to insure on-time delivery impact Nme Nme impact Nme Nme Standard - cost of delay is shallow but accelerates before leveling out; provide a reasonable lead-time expectation impact Nme Intangible cost of delay may be significant but is not incurred until much later; important but not urgent 2013 David J Anderson and Associates, Inc.

23 Classes of Service and Probabilistic SLAs Work Item taxonomy (demand analysis) *Abstract Value (qualitative) *Cost of Delay (qualitative) *Risk (business, technical, commitment) Process Capability by Type (last 90 days) Lead time mode, mean and cumulative probability SLA (# of days, probability) Expedite Fixed Date Standard Intangible

24 Hedging with Capacity Allocation Backlog / Options Selection (1/4) Ready Buffer (1/4) Analysis (4) Dev / Test (4) QA (4) Deploy Ready Buffer Deployed Expedite WL=1 Fixed Due Date ~ 10% A A A Standard ~70% A A A Intangible ~20% A A A

25 Hedging with Options

26 Options Development and Business Strategy Treat work items as investment options! Development of investment options upstream Coherent? Effective approach? Based on risk or value? Enough information for commitment? What is the cost? Room for improvement? Work Item Taxonomy Simplify Scale (small, medium, large) Coalesces data sets for performance analysis Reduce variation in the system by bucketing (w/outlier detection)

27 Hedging Risk with Options Backlog / Options Selection (4) Ready Buffer (4) Analysis (5) Dev / Test (5) QA (5) Deploy Ready Buffer Deploy Bugs A B H G F The discard rate can be as much as 50% MMFs K Reject D E C Options have value because the future is uncertain 0% discard rate implies there is no uncertainty about the future Discarded N Any effort spent estimating options that get discarded is pure waste! L M J Defer commitment to last responsible moment to maximize options and decrease risk!

28 Feedback Loops

29 Feedback Loops Ops Review Monthly cross-org business / engineering metric review Capability Review Coach/player, manager/contributor 1:1 system capability & improvement dialogue Daily Standup Daily standup / boardwalk Maturity levels visual environment feedback Work boards, charts, andons, signals

30 Effective Standup Practice Drive standup from the actual work on the board instead of status updates and the black art of smells Boardwalk

31 Metrics that Matter

32 Cumulative Flow Diagram

33 Lead Time Run Chart

34 Lead Time Histogram

35 Metrics that Count WIP (Flow) Lead Time / Delivery Rate (by single mode) Lead Time Histogram (by single mode) Due Date Performance Flow Efficiency *Blockers Trends and Clustering Quality Failure / Value Demand Ratio Defect Trends and Clustering

36 Lean Portfolio Management

37 Lean Portfolio Management 1. Portfolio Kanban (cards are Projects) 2. Modeling and Simulation a. Risk modeling i. Workflow and WIP limits ii. Work item taxonomy iii. Defect insertion rate iv. Added scope v. Skillsets vi. Project phases vii. b. Monte Carlo Simulation c. Probabilistic Forecasting d. Planning by Confidence Interval (iterative) i. Gets us away from deterministic planning and commitment 3. Operations Review (scaling/monitoring)

38 Portfolio Risk Management Most big decisions are risky decisions with a lot of uncertainty Should we start Project X? Should we stop Project Y? Should we hire additional resources? Software development is a high risk and creative endeavor!

39 Project Portfolio Kanban Proposed Business Case (4) Dev Implementation (6) Deploy Delivered Cash Cows 20% budget???? $$$ Growth Markets 50% budget?? $$ $$$ $$ $ Innovative/New 30% budget?? $$ $$ $$$ $$ 1. Opportunity Assessment 2. Probabilistic Forecasting / Planning 3. Staffing / Budgeting 1. Dev Complete % Green Automated Regression Suite 3. Zero Technical Debt 1. Live to Site 2. Infrastructure Monitoring Live 1. Vetted by Leadership

40 Pragmatic Engineering Risk Management 3 questions that matter Are we meeting our commitments? (Did I make a promise last week that I can t keep this week) If we needed to reduce delivery time, what skill-set should we hire? What are the top 3 risks jeopardizing delivery that we can manage & control now? Credit: Troy Magennis

41 Hedging with Simulation

42 Committing to Project Dates with Confidence Based on historical data, not speculation What is the probability of a specific project delivery date coming to fruition? Monte Carlo SimulaNon and ProbabilisNc ForecasNng

43 Probabilistic Forecasting

44 Model-Driven Improvement

45 Improvement Models Lean Startup (Validated Learning) Risk Management, Governance, Strategy, Lean Product Development, Options Theory Deming s PDCA Service Capability Improvement Plan Build Act Do Learn Measure Check

46 A3 Problem Solving Managed Improvements Background Root Cause Analysis Plan Problem Statement Countermeasures Break Down Problem Plan Do Effec1ve Confirma1on Check Target Condi1on / Goal SeCng Followup Act

47 PDCA Kanban Ready Q Plan (2) Do (2) Check (2) Act (1) Improvements E C B A A3 Form I H G F D Act Plan Do 1. Background 2. Problem Statement 3. Targeting (Goal Setting) 1. Root Cause Analysis 2. Deploy Countermeasures 1. Measurement / Confirmation 1. Follow up Check

48 In Summary Kanban is much more than a Pull System without iterations Kanban provides a decision-support and risk management framework Kanban provides a rich set of Lean heuristics to enabled an improvement culture Moving from a Batch System to a Pull System will enable significant improvement to business performance

49 Thanks!