Modern Risk Management with Kanban

Modern Risk Management with Kanban Eric Green eric@zenkata.io @zenagilist Keep Austin Agile - March 21, 2014

What do we mean by the term modern? mod ern adjective : based on or using the newest information, methods, or technology Source: Merriam Webster Delivery expectation is probabilistic, not speculative Commitment is deferred Real Option Theory Performance/capability are more effectively understood by observing work metrics, over anecdotal assessment Sustainability underpins improvement Manage work as we manage the rest of our business with a eye to Risk Management

Speculative to Probabilistic Moneyball 2011 Move away from speculative (expert) intuition (assessing capability & estimating work) if the goal is to break the mold of traditional performance in software projects Baseball isn t just numbers, it s not science. If it was anybody could do it.. But they can t b/c they don t know what we know. Adapt or die You ve got a kid in there with a degree in economics and you ve got a scout here with 30 years of experience!

Eric Green CEO, Zenkata IO 20 years in industry 16 years at AMD (US, Europe, Asia) 1 year CTO lean startup 2010 4 years activating Scrum teams with Kanban LKU Accredited Kanban Trainer Associate with David J Anderson & Associates Zenkata founded March 2013 (left ebay post in Sept) Leverage 10+ years of learning with Scrum teams around the world Explore, apply and teach cutting-edge heuristics for lean management of knowledge work and service delivery Improve people s lives through sustainable practice and improvement culture Human knowledge has evolved beyond 1 st generation Agile

Kanban Self Assessment Level Novice Informed Prac1cing Prac11oner Skilled Desc Curious with lihle knowledge AHended conference, read some web sites/blogs, watched videos 1 small team < 6 months Larger teams > 6 months Scaled implementanon, several teams, designed kanban systems > 18 months 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Novice Kanban knowledge histogram Informed PracNcing PracNNoner Skilled

What is Kanban?

Kanban A Kanban Japanese (Hiragana) meaning: Signal Card(s) Taiichi Ohno in Toyota Production System Representing a Unit of Capacity Proto-Kanban Limiting WIP by person Recurring infinite queues Personal Kanban Task level, personal productivity, family management etc. Virtual Kanban System Proper Pull System across an integrated knowledge discovery/service workflow Kanban Method (David J Anderson) Virtual Kanban Systems Evolutionary Change Management Probabilistic Forecasting and Expectation Sustainability and Improvement Culture Lean Risk Management Framework

Kaizen, not Kaikaku Kaizen Kanban is about this ConNnuous, Incremental, EvoluNonary Improvement Kaikaku TransformaNon (J- Curve Effect)

The Kanban Method or just Kanban Is A Systems Thinking approach to service delivery in creative knowledge work An Evolutionary Change Management method A Decision Support and Risk Management framework A method without methodology Is NOT A process A software development methodology A project management methodology Prescriptive

Kanban Method Principles 1. Start with what you do (understand) 2. Agree to pursue evolutionary change 3. Respect current process and roles 4. Encourage leadership at all levels

Kanban Method General Practices 1. Visualize (work, metrics, defects, risks etc.) 2. Make policies explicit (defines the system, operational and risk management practices) 3. Limit WIP (enables delivery, quality, sustainability for people) 4. Manage Flow (knowledge discovery/value stream management) 5. Implement Feedback Loops (daily, improvement, operations) 6. Improve experimentally using models (A3, 5 Whys, PDCA, Build-Measure-Learn, Cynefin, Scientific Method)

Hedging : a means of protection or defense (as against financial loss) (Merriam Webster) : a risk management strategy used in limiting or offsetting probability of loss from fluctuations in the prices of commodities, currencies, or securities. (www.businessdictionary.com) taking odds on the Pass

Kanban Systems

Kanban System Options Dev Build Pipeline Deployment Ideas Business Case Dev (12,24) Analysis (6,12) Build Ready (3,6) Committed (1,3) Dev/ Test (1,2) Demo/ Review (1,2) Deploy Ready (0,4) Deploy A Engineering Lead Time B C Cost of acquiring Options $$$ Investment cost $$$ Transaction $$$ D E Options development Investment/committed work Deployment Commitment Point #1 Commitment Point #2 X Y Z Reject Bin Blocker Bin * * * * * *

Deterministic vs. Probabilistic Expectation Change Requests I don t want to take the risk of being longer than 60 days. I need a precise estimate of when it will be delivered! Mean 50 days 85% at 60 days 98% at 150 days David J Anderson and Associates, Inc.

Demand and Capability Analysis Demand Analysis Sources of dissatisfaction (internal and external) Failure-to-Value demand ratio Hidden work types Type, frequency and size analysis Cost of delay assessment Value assessment Options development (upstream kanban) Capability Analysis Lead time analysis by type (data mine ticketing system) Probabilistic forecasting Flow Efficiency analysis

Understanding Process Variation (i.e., causes of delay) Common Cause Variation I wasted a day in meetings yesterday! Bring the tail in! Special Cause Variation We can t do the build today because a 3 rd party library license expired on the server Source: hhp://www.astm.org Tip: one of the largest sources of variation in an IT / development process is variation in size of work items

Bottleneck Management Idea Analysis Design Code ~90 ~80 ~50 ~50 Error Error Error Working Code Acceptance Test System Test Unit Test ~80 ~30 ~50 Five Focusing Steps: 1. Identify - Current CCR is System Test 30 per month 2. Exploit - Testers relieved of all non-essential tasks, extra PMs assigned to complete administrative tasks, analysts assigned to future test plans 3. Subordinate - Requirements release restricted to ~30 per month 4. Elevate - Plan to recruit temporary testing staff immediately 5. Rinse-and-repeat 2013 David J Anderson and Associates, Inc.

Recurring Challenges in Time-boxed Development Batch Processing inherently impedes the flow of work Coupled input/output cadences are suboptimal Commonly has big coordination and transaction costs for grooming, estimating and planning ceremonies Kanban addresses these Batch commitment is speculative and therefore carries more risk Point Velocity is an unreliable basis for understanding commitment, delivery and predictability Flaw of Averages > sprint commitment Math on Point distributions in unreliable (e.g., release burndown) Point update error Does not support expedite capability (e.g., prod defects)

2012 Case Study: Timebox to Pull System (1) Stopped committing to speculative estimated batches (2) Stopped doing long planning/ estimating meetings (3) Started doing frequent queue replenishment and continued to snapshot points accepted Pull system introducnon Delivery Rate 140% VariaNon 45%

Risk-Based Work Taxonomy

Cost of Delay Risk-based Scheduling Heuristics impact impact Expedite critical and immediate cost of delay; can exceed other kanban limit (bumps other work) Nme Nme impact impact Fixed date cost of delay goes up significantly after deadline; Start early enough & dynamically prioritize to insure on-time delivery impact Nme Nme impact Nme Nme Standard - cost of delay is shallow but accelerates before leveling out; provide a reasonable lead-time expectation impact Nme Intangible cost of delay may be significant but is not incurred until much later; important but not urgent 2013 David J Anderson and Associates, Inc.

Classes of Service and Probabilistic SLAs Work Item taxonomy (demand analysis) *Abstract Value (qualitative) *Cost of Delay (qualitative) *Risk (business, technical, commitment) Process Capability by Type (last 90 days) Lead time mode, mean and cumulative probability SLA (# of days, probability) Expedite Fixed Date Standard Intangible

Hedging with Capacity Allocation Backlog / Options Selection (1/4) Ready Buffer (1/4) Analysis (4) Dev / Test (4) QA (4) Deploy Ready Buffer Deployed Expedite WL=1 Fixed Due Date ~ 10% A A A Standard ~70% A A A Intangible ~20% A A A

Hedging with Options

Options Development and Business Strategy Treat work items as investment options! Development of investment options upstream Coherent? Effective approach? Based on risk or value? Enough information for commitment? What is the cost? Room for improvement? Work Item Taxonomy Simplify Scale (small, medium, large) Coalesces data sets for performance analysis Reduce variation in the system by bucketing (w/outlier detection)

Hedging Risk with Options Backlog / Options Selection (4) Ready Buffer (4) Analysis (5) Dev / Test (5) QA (5) Deploy Ready Buffer Deploy Bugs A B H G F The discard rate can be as much as 50% MMFs K Reject D E C Options have value because the future is uncertain 0% discard rate implies there is no uncertainty about the future Discarded N Any effort spent estimating options that get discarded is pure waste! L M J Defer commitment to last responsible moment to maximize options and decrease risk!

Feedback Loops

Feedback Loops Ops Review Monthly cross-org business / engineering metric review Capability Review Coach/player, manager/contributor 1:1 system capability & improvement dialogue Daily Standup Daily standup / boardwalk Maturity levels visual environment feedback Work boards, charts, andons, signals

Effective Standup Practice Drive standup from the actual work on the board instead of status updates and the black art of smells Boardwalk

Metrics that Matter

Cumulative Flow Diagram

Lead Time Run Chart

Lead Time Histogram

Metrics that Count WIP (Flow) Lead Time / Delivery Rate (by single mode) Lead Time Histogram (by single mode) Due Date Performance Flow Efficiency *Blockers Trends and Clustering Quality Failure / Value Demand Ratio Defect Trends and Clustering

Lean Portfolio Management

Lean Portfolio Management 1. Portfolio Kanban (cards are Projects) 2. Modeling and Simulation a. Risk modeling i. Workflow and WIP limits ii. Work item taxonomy iii. Defect insertion rate iv. Added scope v. Skillsets vi. Project phases vii. b. Monte Carlo Simulation c. Probabilistic Forecasting d. Planning by Confidence Interval (iterative) i. Gets us away from deterministic planning and commitment 3. Operations Review (scaling/monitoring)

Portfolio Risk Management Most big decisions are risky decisions with a lot of uncertainty Should we start Project X? Should we stop Project Y? Should we hire additional resources? Software development is a high risk and creative endeavor!

Project Portfolio Kanban Proposed Business Case (4) Dev Implementation (6) Deploy Delivered Cash Cows 20% budget???? $$$ Growth Markets 50% budget?? $$ $$$ $$ $ Innovative/New 30% budget?? $$ $$ $$$ $$ 1. Opportunity Assessment 2. Probabilistic Forecasting / Planning 3. Staffing / Budgeting 1. Dev Complete 2. 100% Green Automated Regression Suite 3. Zero Technical Debt 1. Live to Site 2. Infrastructure Monitoring Live 1. Vetted by Leadership

Pragmatic Engineering Risk Management 3 questions that matter Are we meeting our commitments? (Did I make a promise last week that I can t keep this week) If we needed to reduce delivery time, what skill-set should we hire? What are the top 3 risks jeopardizing delivery that we can manage & control now? Credit: Troy Magennis

Hedging with Simulation

Committing to Project Dates with Confidence Based on historical data, not speculation What is the probability of a specific project delivery date coming to fruition? Monte Carlo SimulaNon and ProbabilisNc ForecasNng

Probabilistic Forecasting

Model-Driven Improvement

Improvement Models Lean Startup (Validated Learning) Risk Management, Governance, Strategy, Lean Product Development, Options Theory Deming s PDCA Service Capability Improvement Plan Build Act Do Learn Measure Check

A3 Problem Solving Managed Improvements Background Root Cause Analysis Plan Problem Statement Countermeasures Break Down Problem Plan Do Effec1ve Confirma1on Check Target Condi1on / Goal SeCng Followup Act

PDCA Kanban Ready Q Plan (2) Do (2) Check (2) Act (1) Improvements E C B A A3 Form I H G F D Act Plan Do 1. Background 2. Problem Statement 3. Targeting (Goal Setting) 1. Root Cause Analysis 2. Deploy Countermeasures 1. Measurement / Confirmation 1. Follow up Check

In Summary Kanban is much more than a Pull System without iterations Kanban provides a decision-support and risk management framework Kanban provides a rich set of Lean heuristics to enabled an improvement culture Moving from a Batch System to a Pull System will enable significant improvement to business performance

Thanks!