The Risk Driver Approach to Project Schedule Risk Analysis

The Risk Driver Approach to Project Schedule Risk Analysis A Webinar presented by David T. Hulett, Ph.D. Hulett & Associates, LLC To the College of Performance Management April 18, 2013 2013 Hulett & Associates, LLC 1

Risk Drivers Method Agenda Limitations of traditional 3-point estimating Explain Risk Drivers approach, start with the risks themselves One Driver per activity Multiple Risk Drivers per Activity Correlation is modeled in the Monte Carlos simulation Assess the schedule against CPM scheduling best practice Simple space vehicle development schedule as an example Prioritize risks to schedule Mitigation of high-priority risks 2013 Hulett & Associates, LLC 2

Limitations with the Traditional 3-point Estimate of Activity Duration Typical schedule risk analysis starts with the activity that is impacted by risks, not the risks themselves Estimates the 3-point estimate for optimistic, most likely and pessimistic duration Creates a probability distribution for activity duration Performs Monte Carlo simulation Which risks cause the most overall schedule risk? These questions are typically answered by: Sensitivity to activity durations Criticality of activity durations 2013 Hulett & Associates, LLC 3

Some Problems with Traditional Approach Can tell which activities are crucial, but not directly which risks are driving Makes poor use of the Risk Register that is usually available Cannot decompose the overall schedule risk into its components BY RISK Ability to assign the risk to its specific risk drivers helps with communication of risk causes and risk mitigation 2013 Hulett & Associates, LLC 4

We Propose a Different Approach: Start with the Risks Themselves Drive the schedule risk by the risks already analyzed in the Risk Register For each risk, specify: Probability it will occur Impact on time if it does Activities it will affect Starting with the risks themselves gives us benefits Links qualitative analysis to the quantitative analysis Estimates the impact of specific risks for prioritized mitigation purposes Correlations between activities happen automatically never have to guess at these coefficients again, never get impossible matrices 2013 Hulett & Associates, LLC 5

Simple Example of Risk Register Risks Use the Risk Drivers feature in Pertmaster 8 Collect probability and impact data on risks Load the risks Assign risks to schedule activities 2013 Hulett & Associates, LLC 6

Risk Drivers Mechanics (1) The risk Driver is assigned to one or several activities, affecting their durations by a multiplicative Driver E.g., the Driver may be.90 for optimistic, 1.0 for most likely and 1.25 for pessimistic These Drivers multiply the schedule durations of the activities to which they are assigned Risks can be assigned to one or more activities Activity durations can be influenced by one or more risks 2013 Hulett & Associates, LLC 7

Risk Drivers Mechanics (2) Risk Drivers are assigned a probability of occurring on any iteration. When the risk occurs, the Driver used is chosen at random from the 3-point estimate and operates on all activities to which it is assigned When not occurring on an iteration the risk Driver takes the value 1.0, a neutral value When an activity is influenced by more than one risk, their Drivers are multiplied together, if they happen on an iteration 2013 Hulett & Associates, LLC 8

Hits Cumulative Frequency Risk Driver Probability is 100%, Driver can be + or - Entire Plan : Duration 100% 115 Here the Ranges are based on deviations + and from the Plan. Probability is 100% 400 350 300 250 200 150 95% 111 90% 109 85% 108 80% 106 75% 105 70% 104 65% 104 60% 103 55% 102 50% 101 45% 101 40% 100 35% 99 30% 99 For the examples we use an activity with 100 days in the schedule 100 25% 98 20% 97 15% 96 50 10% 95 5% 94 0 90 95 100 105 110 115 Distribution (start of interval) 2013 Hulett & Associates, LLC 9 0% 90

Hits Cumulative Frequency Risk Driver Prob. = 100%, Driver is all Overrun 350 Entire Plan : Duration 100% 130 95% 125 90% 122 Here the Plan is the Optimistic Value. Probability is 100% 300 250 200 150 85% 121 80% 119 75% 118 70% 117 65% 116 60% 115 55% 114 50% 113 45% 112 40% 111 35% 110 100 30% 109 25% 109 20% 108 50 15% 107 10% 105 5% 104 0 100 105 110 115 120 125 130 Distribution (start of interval) 0% 100, 10

Assigning a Probability Less than 100% The essence of a risk is its uncertainty in two dimensions: Uncertainty of its occurrence, specified by a probability Uncertainty of its impact, specified by a range of durations If the risk may or may not occur, we specify the probability that it will occur The risk occurs and affects the activities it is assigned to on X% of the iterations, chosen at random, the multiplicative Driver used is chosen at random from the range of data input by the user On (1 X)% of the iterations, Driver takes 1.0 value 2013 Hulett & Associates, LLC 11

Hits Cumulative Frequency Hits Cumulative Frequency Assigning a Probability Less than 100% Spike contains 40% of the probability 2000 1800 1600 1400 1200 Entire Plan : Duration 100% 130 95% 123 90% 120 85% 118 80% 116 75% 114 70% 113 65% 111 60% 110 55% 109 50% 107 3500 3000 2500 2000 Entire Plan : Duration 100% 114 95% 107 90% 104 85% 101 80% 100 75% 100 70% 100 65% 100 60% 100 55% 100 50% 100 Spike contains 70% of the probability 1000 45% 106 45% 100 800 40% 103 35% 100 1500 40% 100 35% 100 600 30% 100 30% 100 25% 100 1000 25% 100 400 20% 100 20% 100 200 15% 100 10% 100 500 15% 100 10% 99 5% 100 5% 97 0 100 105 110 115 120 125 130 Distribution (start of interval) 0% 100 0 90 95 100 105 110 115 Distribution (start of interval) 2013 Hulett & Associates, LLC 12 0% 90

Assigning More than One Risk to an Activity If more than one risk is acting on an activity, the resulting ranges are the multiplication of the percentages This is reality an activity is often affected by multiple risks Two cases are shown next: When both risks are 100% likely to occur When both risks are < 100% likely to occur In each case, the computer simulation creates the uncertainty range on an activity s duration it is not estimated 2013 Hulett & Associates, LLC 13

Parallel and Series Risks Multiplicative Used with Risk Drivers (RiskDrivers) Risk 1 1.2 Driver Risk 2 1.05 Driver Use 1.2 Driver, the largest Driver only If these two risks are parallel, they can be recovered simultaneously Risk 1 1.2 Driver Risk 2 1.05 Driver If these two risks are series, they can not be recovered simultaneously Use (1.2 x 1.05 = 1.26) Driver, multiply the two 2013 Hulett & Associates, LLC 14

Hits Cumulative Frequency Two Risks affect One Activity using Drivers that Occur 100% - placed in Series 260 240 220 Entire Plan : Duration 100% 146 95% 131 90% 127 85% 124 80% 123 Risks in series, P80 is 123 days 200 180 160 140 75% 121 70% 119 65% 118 60% 117 55% 116 50% 115 120 45% 114 100 80 60 40 20 0 100 110 120 130 140 Distribution (start of interval) 40% 112 35% 111 30% 110 25% 109 20% 108 15% 106 10% 104 5% 102 0% 92, 15

Hits Cumulative Frequency Two Risks affect One Activity using Drivers that Occur 100% - in Parallel Entire Plan : Duration 100% 130 95% 125 350 90% 122 85% 121 300 80% 119 75% 118 70% 117 250 65% 116 60% 115 Risks in parallel, P80 is 119 days 200 150 55% 114 50% 113 45% 112 40% 111 35% 111 30% 110 100 25% 109 20% 108 50 15% 107 10% 106 5% 105 0 100 105 110 115 120 125 130 Distribution (start of interval) 0% 100 2013 Hulett & Associates, LLC 16

Hits Cumulative Frequency Two Risks with Less than 100% Probability Affecting one Activity Risks in Series The spike at 100 days represents (1) the likelihood that neither risk occurs [60% x 50% = 30%] and (2) the chance that 100 days is picked when one or both occur. 1800 1600 1400 1200 1000 800 600 400 200 Entire Plan : Duration 100% 144 95% 123 90% 119 85% 116 80% 114 75% 112 70% 110 65% 108 60% 107 55% 105 50% 103 45% 101 40% 100 35% 100 30% 100 25% 100 20% 100 15% 100 10% 99 5% 96 0 90 100 110 120 130 140 Distribution (start of interval) 2013 Hulett & Associates, LLC 17 0% 90

Hits Cumulative Frequency Two Risks with Less than 100% Probability Affecting one Activity Risks in Parallel With one parallel risk s having a minimum range of 100%, it cannot be less than 100 days 2200 2000 1800 1600 1400 1200 Entire Plan : Duration 100% 130 95% 122 90% 118 85% 115 80% 113 75% 111 70% 110 65% 109 60% 107 55% 105 50% 103 1000 45% 101 40% 100 800 35% 100 30% 100 600 25% 100 20% 100 400 15% 100 200 10% 100 5% 100 0 100 105 110 115 120 125 130 Distribution (start of interval) 0% 100 2013 Hulett & Associates, LLC 18

Risk Drivers Model How Correlation Occurs Coefficients are Calculated (1) Risk Probability =.5, Range.95, 1.05, 1.15 Activity 1 Activity 2 Correlation = 100% 2013 Hulett & Associates, LLC 19

Scatter showing 100% Correlation 2013 Hulett & Associates, LLC 20

Risk Drivers Model How Correlation Occurs Coefficients are Calculated (2) Risk Probability =.25, Range.8,.95, 1.05 Risk Probability =.5, Range.95, 1.05, 1.15 Risk Probability =.45, Range 1.0, 1.10, 1.20 Activity 1 Activity 2 Correlation = 37% Correlation is modeled as it is caused in the project Correlation coefficients are generated, not guessed 2013 Hulett & Associates, LLC 21

Scatter showing 37% Correlation 2013 Hulett & Associates, LLC 22

Sensitivity to the Risk Drivers Risk 1 is more important since it affects both Activity A and Activity B 2013 Hulett & Associates, LLC 23

http://www.gao.gov/products/gao-12-120g 2013 Hulett & Associates, LLC 24

Simple 2-Stage Space Vehicle Schedule Software used: Pertmaster v. 8.7 2013 Hulett & Associates, LLC 27

Two Types of Risk Inherent variability including duration estimating error uncertainty Probability = 100% Used Quick Risk of -5% and +10%, could be reference ranges for different types of activities Could use reference ranges that would differ by type of activity Discrete risks derived from Risk Register Probability < 100% Summarized from detailed Risk Register These have a probability of occurring and an impact on specific activities if they do Parallel to their Risk Register information 2013 Hulett & Associates, LLC 28

Standard 3-point Range Representing Inherent Variability and Duration Estimating Error Inherent variability and estimating error: Optimistic - 5% Pessimistic +10% 2013 Hulett & Associates, LLC 29

Hits Cumulative Frequency Results with Inherent Variability and Duration Estimating Error Only Spacecraft for PMChallenge 2009 Entire Plan : Finish Date 100% 30 Oct 20 400 95% 27 Aug 20 90% 14 Aug 20 350 300 85% 06 Aug 20 80% 30 Jul 20 75% 24 Jul 20 70% 20 Jul 20 Deterministic: 13 APR 2020 is 1% 250 200 65% 15 Jul 20 60% 10 Jul 20 55% 06 Jul 20 50% 01 Jul 20 45% 26 Jun 20 P-80 is 30 JUL 20, about 3.5 months later than planned 150 100 40% 22 Jun 20 35% 17 Jun 20 30% 11 Jun 20 25% 08 Jun 20 20% 01 Jun 20 Spread from P-5 to P-95 is 5 MAY 20 to 27 AUG 20 for 3.7 months 15% 25 May 20 50 10% 15 May 20 5% 05 May 20 0 15 Mar 20 04 May 20 23 Jun 20 12 Aug 20 01 Oct 20 Distribution (start of interval) 0% 11 Mar 20 2013 Hulett & Associates, LLC 30

Risk Analysis on Space Vehicle Project Risk Drivers are from Risk Register Risk Min Most Likely Max Likelihood Requirements have not been decided 95% 105% 120% 70% Several alternative designs considered 95% 100% 115% 60% New designs not yet proven 90% 103% 112% 40% Fabrication requires new materials 95% 105% 115% 50% Lost know-how since last full spacecraft 100% 100% 105% 30% Funding from Congress is problematic 90% 105% 115% 70% Schedule for testing is aggressive 100% 120% 130% 100% Seven risk Drivers have been identified and quantified. Each Risk has probability assigned Five have optimistic ranges possible, two are pure threats 2013 Hulett & Associates, LLC 31

Risks Assigned to Activities (1) Risk Requirements Definition FS Preliminary Design FS Final Design FS Fabrication Test FS Engine Requirements Not Complete X Alternative Designs Possible X Designs Not Proven X New Materials in Fabrication X Lost Know-How X Funding Problematic X X X X Testing Schedule Aggressive X 2013 Hulett & Associates, LLC 32 32

Risks Assigned to Activities (2) Risk US Preliminary Design US Final Design US Fabrication US Test Integration Integration Testing Requirements Not Complete Alternative Designs Possible X Designs Not Proven X New Materials in Fabrication X Lost Know-How X X Funding Problematic X X X X X X Testing Schedule Aggressive X X 2013 Hulett & Associates, LLC 33 33

Hits Cumulative Frequency Results Adding Risk Drivers to the Background Risk Spacecraft for PMChallenge 2009 Entire Plan : Finish Date 100% 08 Mar 22 Baseline 13 APR 20 is only 3% likely 80 th percentile is 7 APR 21, 11.8 months later 550 500 450 400 350 95% 06 Aug 21 90% 16 Jun 21 85% 10 May 21 80% 07 Apr 21 75% 03 Mar 21 70% 05 Feb 21 65% 13 Jan 21 60% 22 Dec 20 55% 03 Dec 20 Spread P-5 to P-95 is 13May20 to 6 Aug 21, for ~ 15 months 300 250 200 150 50% 16 Nov 20 45% 29 Oct 20 40% 13 Oct 20 35% 28 Sep 20 30% 11 Sep 20 25% 31 Aug 20 100 20% 14 Aug 20 15% 28 Jul 20 50 10% 30 Jun 20 5% 13 May 20 0 25 Jan 20 12 Aug 20 28 Feb 21 16 Sep 21 Distribution (start of interval) 0% 12 Nov 19 2013 Hulett & Associates, LLC 34

Activity Tornado Chart from All-In Simulation Spacecraft for PMChallenge 2009 Duration Sensitivity: Entire Plan - All tasks 00025 - US Fabrication 00011 - FS Fabrication 00028 - Integration 00023 - US Final Design 00009 - FS Final Design 00021 - US Preliminary Design 00007 - FS Preliminary Design 70% 70% 81% 81% 79% 76% 75% Risky Activities: Fabrication, Integration, Final Design, Preliminary Design, Testing All are correlated with the finish date > 60% These are activities / paths, NOT RISKS 00029 - Integration Testing 63% 00026 - US Test 61% 00012 - Test FS Engine 61% 2013 Hulett & Associates, LLC 35

Risk Driver Tornado from All-In Simulation Risk Factors Driving Project Schedule 6 - Funding from Congress is problematic 4 - Fabricaton requires new materials 3 - New designs not yet proven 7 - Schedule for testing is aggressive 2 - Several alternative designs considered 5 - Lost know-how since last full spacecraft 1 - Requirements have not been decided The main RISK, however, is funding from Congress, which affected all activities in this model. This is the main risk to mitigate, if possible 8 - Cost Risk is based on immature data 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 Correlation 2013 Hulett & Associates, LLC 36

Contribution of Each Risk to the Contingency (1) Explain the Contingency to the P-80 P-80 Date Take Risks Out: All Risks In 7-Apr-21 Days Saved % of Contingency Specific Risks Taken Out in Order Funding 23-Nov-20 135 38% Testing Schedule 5-Oct-20 49 14% New Materials 3-Sep-20 32 9% Alternative Design 21-Aug-20 13 4% Requirements 14-Aug-20 7 2% New Design 6-Aug-20 8 2% Lost Know How 31-Jul-20 6 2% Uncertainty Natural Variation & Estimating Error 13-Apr-20 109 30% Total Contingency 359 100% 2013 Hulett & Associates, LLC 37

Contribution of Each Risk to the Contingency (2) 100% 90% Variation:108 Variation:6 :8 n:7 n:13 ation:32 Variation:49 Variation:134 80% Graphic of the effect of taking the risks out in order of priority at the P-80 70% 60% 50% 40% Cumulative Probability 30% Variation:50 ariation:3 4 5 3 :11 ariation:45 ia ion:1 20% 10% 19 20 Jan 20 10 Mar 20 29 Apr 20 18 Jun 20 07 Aug 20 26 Sep 20 15 Nov 20 04 Jan 21 23 Feb 21 14 Apr 21 03 Jun 21 23 Jul 21 11 Sep 21 31 Oct 21 20 Dec 21 08 Feb 22 0% 2013 Hulett & Associates, LLC 38

Mitigating the Most Important Risk Effect of Partially Mitigating the Risk with the Highest Priority Min Most Likely Max Likelihood P-80 Date Funding from Congress is problematic 90% 105% 115% 70% 7-Apr-21 Mitigation: Situate your suppliers strategically by Congressional District of the members of the Committee. Make the jobs case for continuous funding. Impact on the parameters 90% 105% 115% 30% 5-Jan-21 Mitigating the risk is estimated to reduce the probability from 70% to 30% but, if the risk happens, the impact range remains the same. This saves 92 calendar days at the P-80 target level of confidence 2013 Hulett & Associates, LLC 39

Summary (1) The focus is on the risks, not their impact Risks explain the need for a contingency Management appreciates this focus on risks Risk interviews are conducted at 5,000 foot level, where people typically think of risk Interviews go faster, stick to the substance 2013 Hulett & Associates, LLC 40

Summary (2) Use Risk Register for quantitative analysis Specific risks can be quantified and assigned to schedule activities Quantification is probability and impact A risk can affect several activities An activity can be affected by several risks Risk Drivers can be combined with other more traditional approaches such as 3-point estimates for inherent variability risk, estimating error or for probabilistic branching 2013 Hulett & Associates, LLC 41

The Risk Driver Approach to Project Schedule Risk Analysis A Webinar presented by David T. Hulett, Ph.D. Hulett & Associates, LLC To the College of Performance Management April 18, 2013 2013 Hulett & Associates, LLC 42