in Cost Risk Analysis. Javier Ordóñez, Ph.D. Director of Custom Solutions Palisade Corporation

Transcription

1 in Cost Risk Analysis Javier Ordóñez, Ph.D. Director of Custom Solutions Palisade Corporation

2 Outline Introduction Background Project Performance Record Definitions Cost Risk Analysis Correlation Schedule Integration

3 Introduction Most projects are conducted in a changing environment; this makes the schedule and cost analysis difficult in the early stages. Traditionally, cost and duration estimates are point estimates. Estimation based on the most likely values. It is necessary to study uncertainties involved in the project.

4 Project Performance Record Project Success (RMC Project Management) Only 28% of all projects succeed Time to market can be improved by 65% Projects can be completed in 50% of the time IT Projects (Chaos Report) 31% of project cancelled before completion 53% of projects will cost 189% of their original estimate Average time overrun is 222% Average project success is 16.2% (software projects)

5 Project Performance Record Infrastructure Projects (Cont) Cost Performance Overtime (Flyvbjerg et al 2003)

6 Background: PRA Adoption in Federal & State Agencies Federal Transit Administration (FTA) requires a risk assessment/mitigation study for any new transit project applying for federal funding Department of Transportation of the State of Washington (WSDOT) has a risk-based approach to validate cost estimates OMB Capital Programming Guide, 2007: Risk Adjusted Budget and Schedule (ANSI/EIA Standard 748) DoD Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide: Schedule Risk Analysis Risk Management Guide for DoD Acquisition (2003)

7 Definitions: Project Risk & Uncertainty Project risk is defined as the possibility that the outcome of an uncertain event affects negatively or positively the cost and time performance of project activities and/or their planned execution Risk = Consequence x Probability of Occurrence Uncertainty is defined as the lack of knowledge about the parameters that characterize the system

8 Project Budgeting Typically budgets are deterministic Simulation Approach Individual cost components are unimodal and skewed Common use of 3 point estimate and triangular, beta, lognormal distributions Model cost items prone to variation with suitable statistical distributions Generate random numbers hundred of times according to specified distributions and calculate total cost Total cost dist is used to calculate probability of cost overrun and to establish adequate contingencies

9 Triangular Distributions Description Used when minimum, maximum, and most likely values are known. Used when high and low thresholds are of equal distance to expected outcome. Easy to calculate and generate, but limited ability to accurately model realworld estimates. Examples Product pricing Cost to manufacture Inputs Most likely (mode) Minimum & maximum values Shift (optional) 9

10 PERT Distributions Description Alternative to Triangular Same 3 parameters, but uses smooth curve deemphasizes tails Provides most-likely case rather than extreme values Describes outlying impacts more realistically Examples Product pricing Manufacturing costs Sales volumes Raw material pricing Most likely (mode) Minimum & maximum values Shift (optional) Inputs 10

11 Triangular vs. PERT Distributions Comparison More closely resembles realistic probability distribution. Provides close fit to normal or lognormal distributions Like the Triangular distribution, emphasizes most likely value over minimum and maximum estimates. Unlike Triangular, proves a smooth curve that progressively emphasizes values around (near) the most likely, over values around edges. Can trust estimate for most likely value. Even if it not exactly accurate, will be close. Produces a curve similar to Normal, without knowing precise parameters. Triangular distributions are fine for symmetrical data PERT more accurately depicts normal distributions Use PERT when the min, max, and most likely are known 11 Key Takeaways

12 Risks Events vs. Uncertainty Probability $ o Time Uncertainty Risk Events Impacts

13 Qualitative Risk Analysis Likelihood Score Risk Register Not Likely 1 Low Likelihood 2 Likely 3 High Likely 4 Near Certainty 5 Likelihood Priorit y 1 n Descrip tion Prob of Occurrence Activities Affected Cost / Time Impact Consequence Schedule Cost Technical Score Minimal or no impact Minimal or no impact Minimal or no impact 1 Additional activities required; able to meet key dates Budget increase <1% Minor performance shortfall, same approach retained 2 Minor schedule slip; will miss need date Budget increase <5% Project critical path affected Budget increase <10% Moderate performance shortfall, but workarounds available Unacceptable, but workaround available 3 4 Cannot achieve key project milestone Budget increase >10% Unacceptable, no alternatives exist 5

14 Description Binomial Distributions RiskBinomial(n,p) = probability of achieving certain number of successes in n independent trials, where probability of success for each trial is p, and each trial has only two possible outcomes ( success or fail ) Describes the outcome of a series of trials that can only be a success or failure. As the average increases, the profile approaches the Normal distribution. Under some conditions, you can use the Normal distribution as an approximation. Inputs n = number of outcomes p = probability of each outcome s occurrence Min, max & shift (optional) Examples Heads or tails in coin tosses Occurrence of a risk event 14

15 RiskCompound Function It uses two distributions to create a single new input distribution. The first argument specifies the number of samples which will be drawn from the distribution entered in the second argument. For example, the function: RiskCompound(RiskPoisson(5),RiskLognorm(10000,10000)) It would be used in the insurance industry where the frequency or number of claims is described by RiskPoisson(5) and the severity of each claim is given by RiskLognorm(10000,10000). 15

16 Contingency calculation w/o PRA The percentage figure is, most likely, arbitrarily arrived at and not appropriate for the specific project. There is a tendency to double count risks because some estimators are inclined to include contingencies in their best estimate. A percentage addition still results in a single-figure prediction of estimated cost, implying a degree of certainty that is simply not justified. The percentage added indicates the potential for detrimental or downside risk; it does not indicate any potential for cost reduction and may therefore hide poor management of the execution of the project. Because the percentage allows for all risk in terms of a cost contingency, it tends to direct attention away from time, performance, and quality risks. It does not encourage creativity in estimating practice, allowing it to become routine and mundane, which can propagate oversights.

17 Interdependence Full dependence model directly in Excel Partial dependence smoothing methods Aggregate calculations Correlation 17

18 Correlation and Interdependence Variables move together Positive vs. Inverse relationship Predictive sampling (magnitude) Correlation coefficient Calculating rho r Methodology (rank vs. data) Impact Comparing effect on m vs. s 18

19 Correlation Characteristics Variables must relate to each other in some manner Correlations are often calculated from actual historical data Correlation coefficients range between -1 and 1 0 = no relationship -1 = complete inverse correlation 1 = complete positive correlation Variables without correlation create non-realistic situations 19

20 Correlation Concept Measures the degree of association between 2 variables y r = -1 y r = -.8 x y r = 0 x y r = 1 y r =.8 x x 20 x

21 Cost Correlation Issues If correlation is ignored the total cost variance is underestimated Data limitations during planning stages of most engineering projects Correlation between variables makes use of historical data or subjective estimation from experts Relationship between variables are shaped by many uncontrollable factors, and are best at subjective estimates based on experience and judgment PDF that cost estimator specifies is the marginal distribution of that cost item; if cost items are correlated, the joint density function of the cost items needs to be calculated

22 11/1 11/14 11/27 12/11 12/24 1/7 1/20 2/2 2/16 3/1 Prob Value <= Value on X-Axis Correlation Effects Correlated vs. Independent Case S-Curve for Correlated and Not Correlated Durations Not Correlated Project Cost Date Correlated

23 Correlation Description Specifies the relationship between two or more input variables For analysis and risk assessment it is important to account for correlation between variables Examples Material Load vs. Fatigue Pricing vs. Volume Supply vs. Demand Weather vs. Claims Losses Competitive Entry vs. Sales 23

24 Correlation Calculation Apply the =Rank() function to historical data Correlate the ranked data, using =Correl() Add the resulting coefficient in the Correlation Matrix Situations with positive correlation have wider spread of outcomes (variance or standard deviation) than 0 or negative correlations If correlation exists but is not modeled, risk will be underestimated The effect in the tails of the distributions is disproportionate to the effect on the spread or standard deviation 24

25 Cost Schedule Integration Project cost and schedule estimates are often disconnected. i.e.: If schedule is too optimistic the cost is underestimated When the risk of schedule is disregarded in estimating cost risk, cost risk is underestimated Costo Cost Periodo Report Date Actual Time Tiempo

26 Cost Methodology Use WBS Non-biased quantification of cost elements: local uncertainty Model correlation or avoid stochastic dependencies by applying generic risks Include internal and external risks, and fixed and variable Traditional Modified for integrated analysis

27 Integrated Risk Analysis Results

28 Questions?