Palisade Risk Conference, 2014

Transcription

1 Advanced Risk Management to Improve Cost and Schedule Performance on EPC Projects Risk-Management Best Practices from Nuclear Experience Palisade Risk Conference, 2014 Sola Talabi PhD MBA MSc BSc RMP Project Risk Management Consultant, Pittsburgh Technical ( Risk Management Professor, Carnegie Mellon University 1

2 Agenda 1. My background and how I got into risk management 2. Case studies: 1. Awareness Risk identification issues and how we can improve 2. Assessment Risk assessment issues and how we can improve 3. Overall Recommendations 2

3 My Nuclear and Risk Management Background 1. Mechanical Engineer years at Westinghouse 3. Risk manager for New build opportunities 4. Established risk management organization for Westinghouse 5. Risk Management Professor, Carnegie Mellon University 6. Risk Management Consultant, Pittsburgh Technical 3

4 What is Risk Management? Practice that supports creating Awareness, Assessment and Action on issues of uncertainty that affect planned project performance Awareness: Identification of uncertainties and risks Assessment: Quantification and prioritization of risks Action: Developing and promoting risk response plans Awareness Uncertainties associated with Planned Project Performance Includes the people, processes and tools to support risk management Action Assessment

5 Case Study: Reduction of nuclear power project cost through risk management Identification of the role of risk identification, assessment and action in prevention of cost overruns 5

6 Case Study: Reduction of nuclear power project cost through risk management Background: Historically high cost of nuclear power plants: Ranks in top cost quartile for generation options Problem: Loss of nuclear construction infrastructure from 30 year new-build hiatus NSSS firm had concerns with cost overruns on 3 nuclear power projects in China, South Carolina and Georgia at $30billion Solution Establish a dedicated risk organization Improve risk awareness, assessment and action to prevent overruns 6 Source US-DOE (EIA, 2013)

7 Summary of Problem Statement: Nuclear power cost competitiveness is aggravated by poor historical cost and schedule performance Realized Cost 3 x Estimated Cost 7 Nuclear power plant construction cost comparison of planned to actual costs, for plants with construction start dates from 1970 to 1984, source: (EIA, 1986)

8 Understanding the Role of Risk Management in Cost Overruns: An empirical evaluation of risk management practices in large energy infrastructure firms Determine the relationship between the identification and occurrence of risks. Evaluate the accuracy of cost uncertainty estimates: Accuracy in predicting the impact of postulated risks Provides insights into the adequacy of contingency budgets Determine the importance of the following: Identification of risks Probability (likelihood) of event occurrence Estimated impacts 8

9 Steam Generator Replacement (SGR) Project Assessment Overview SGs are single largest nuclear components in PWRs by weight and size Full work scope cost is approximately $150million Typically replaced at least once during service life of plant due to degradation SGR projects are representative of large nuclear power EPC work scopes, and include: Engineering and design Licensing Procurement Construction Installation Startup and Testing Assessment included 15 projects over 12 years Source: Areva (

10 Results: An association exists between the rate of risk identification and risk occurrence on SGR projects Technical risks occurred on 70% of projects Technical risks identified on 50% of projects 10 Takeaway: It is possible to quantitatively demonstrate the benefit of risk identification in our organizations!

11 Average actual cost given risk event ($1,000) Actual Probability Results: Estimation errors exist with likelihood and impact estimation y = 2.80x y = -1.0x x Average estimated cost given risk event occurs ($1,000) Estimated Probability Takeaway: When performing risk assessments, we can de-bias our experts by showing them their historic propensity for errors!

12 Summary of findings: Issues with project risk identification Negative correlation exists between risk identification and risk occurrence: Emphasizes the importance of risk identification Lack of proactive risk identification causes projects to be managed as a series of surprises Note there are two possible explanations for this trend Increased awareness of risks, generates action that prevents the risks from occurring? Over-identification of risks, that may not be applicable, and hence do not occur. Cognitive and heuristic bias exists in risk identification Availability bias: High identification in areas of competency low occurrence, low identification in areas of low competency with high occurrence Optimism? Motivational? 12

13 Summary of findings: Issues with project risk assessment Evidence of underestimation of costs demonstrated in: Probability Impact Cognitive and heuristic biases exist in risk assessment: Anchoring Overconfidence Optimism 13

14 Improving Identification and Assessment of Risks in Power Plant EPC Projects Assessment of a Modified Delphi Method for Project Risk Assessment Using expert judgment of the likelihood of risks occurring to evaluate an enhanced Delphi-Method that incorporates historical frequency of occurrence in expert assessments 14

15 Elicitation of expert prior assessments and updates of risks based on historical frequency of occurrence Survey designed to assess specific risks Surveyed 10 Project Managers from an Nuclear Safety Steam Supply (NSSS) firm. Assessed estimates of the likelihood of risk events occurring Initial Updated based on historical data Updated based on group feedback 15

16 Likelihood of risk driver occurring What is the likelihood that: Risk X will occur? (Risk Driver 1) I : Initial likelihood of driver occurrence I + H: Update based on feedback of historical frequency I + H + G: Update after feedback on group distribution 16

17 17 Results: Expert Estimates for Initial, Historical and Group Feedback for Risk Driver #1

18 Likelihood of risk occurring Likelihood of risk occurring Sample Results 18

19 Case Study: Investigate the role of risk management in construction cost estimation errors Inclusion of risk management in cost estimation process 19

20 Case Study: Investigate the role of risk management in construction cost estimation errors Statistical analysis of historical nuclear power construction data including: Estimated and realized costs Estimated and realized lead-times (schedule) Plant size Constructor experience Sample of plants in dataset: 67 non-turnkey US projects, hence overruns reported by utility Start dates between , completion by 1986 Source: An Analysis of Nuclear Power Plant Construction Costs (EIA, report in1986) Main Takeaway: Include risk management in estimation process 20

21 21 Nuclear power construction costs were expected to decrease because of learning and economies of scale

22 What role did inaccurate risk assessment and cost estimation play in historical cost overruns? 22 Takeaway: A relationship existed between cost and lead-times that was not reflected in estimates Need to include correlations in models

23 Results: Identified errors in economic forecast models for nuclear power plant construction Assumptions were made of cost reduction due to increased experience No evidence of association between experience and historical cost/schedule overruns 23

24 Results: Identified errors in economic forecast models for nuclear power plant construction If there are economies of scale, then larger plants should have lower per kw construction costs Negative correlation However, a review of historical projects shows: Cost increases as size increases 24 Diseconomies of scale exists because of complexity

25 Case Study: Risk Communication and Escalation Dual use of risk registers for project management and description of financial exposure 25

26 Case Study on Risk Communication and Escalation: Dual use of risk registers for project management and description of financial exposure Background: History of cost overruns: Prior NSSS firm s power plants cost 3 times the planned cost External audits required: Major accounting firm was hired to provide external assessment of risk exposure and ensure adequate reserves Problem: Accounting firm had no history to assess expected exposure, no plant construction in 30 years, no frequency data Solution Project risk registers were designed to serve 2 purposes: Manage project risks used by PMs Quantitatively describe the organization s financial risk used by Finance Historical cost overruns in power plant construction 26

27 Case Study on Risk Communication and Escalation: Dual use of risk registers for project management and description of financial exposure Primary use: Manage project risks used by PMs Secondary use: Quantitatively describe the organization s financial risk used by Finance Communication for Awareness Communication for Assessment Finance View P-Value: Probability that reserves will cover exposure, e.g. P-50 is 50% chance Proj. Mgmt View

28 Risk Communication and Escalation Exposure Change Criteria Level of Reporting / Escalation No change and within tolerance None up to 5% change or below tolerance Project Director >5%< 10% change Project Director and Senior Vice President >10% change Project Director, Senior Vice President, CEO 28

29 29 Case Study: Organizational Structure for Effective Risk Management

30 Background: Case Study on Organizational Structure for Effective Risk Management First of a kind dedicated risk management organization established: Sole purpose to identify and manage risks on $30billion projects Problem: Risk function required autonomy and high-level of objectivity Risk professionals may need to challenge biases of senior level individuals Solution Project risk engineers reported directly to Risk Director, but were matrixed to, and collocated with project. Risk Management Organizational Structure 30

31 Good News: We ve Changed the Trend!!!: Improved Cost and Schedule Performance on Current Projects with Advanced Risk Management Current projects with advanced risk management Prior projects without advanced risk management 31

32 General Risk Improvement Recommendations: Recommendation 1: Establish Risk Management Charter and Organization Practice that supports creating Awareness, Assessment and Action on issues of uncertainty that affect planned project performance Awareness: Identification of uncertainties and risks Assessment: Quantification and prioritization of risks Action: Developing and promoting risk response plans Includes the people, processes and tools to support risk management Awareness Uncertainties associated with Planned Project Performance Action Assessment We need a simple and consistent message that can stand the test of time

33 Recommendation 2: Establish Risk Management Maturity Model Establish your risk management growth plan Measure where you are yearly! Modify your programs based on industry benchmarks and best-practices What gets measured gets improved! 1 Ad-Hoc, Reporting * Pockets of risk management competence appear in the organization * Risk management is applied as needed to specific areas 2 Managed & Corrective * Organizational risk management in place * Risk governance program adopted * Resources aligned * Roles defined * Training performed * Processes and procedures developed * Performance objectives aligned with risk management 3 Qualitative & Preventive * Identifying and preventing potential problems before they occur * Objectives are qualitatively predictable * Risk handling activities are now planned and invoked as needed to mitigate adverse impacts on achieving objectives 4 Quantitative & Predictive * The organization can predict with quantified confidence * Objectives are quantitatively predictable * Risk Management is directly aligned to the organization strategic plan * The organization spends more time on opportunity management because threats are predicted and managed 5 Performance Based * Organization demonstrates continuous improvement * Continuous workforce innovation, and improvement is self-sustaining with completely empowered workgroups

34 Recommendation 3: Develop Standard Risk Registers Repository of all known risks that ensures risk knowledge is effectively shared across applicable projects and is made available for future projects. Knowledge database Advanced starting point for risk management through a list of prior risks that occurred and the actions taken to address those risks. Project managers would not have to reinvent the wheel in identification of risks or in the development of response actions. Active Projects Standard Project Risk Register Document Storage System Corrective Actions database

35 Recommendation 4: Use Best-Practice Tools Example Includes Quantitative Schedule Risk Associated Risk

36 Requirement 5: Description of Performance Uncertainties Relative to Thresholds Stakeholders should understand the relationship between cost and schedule uncertainty and the allocated reserves For probabilistically described cost or schedule uncertainties, there is a discreet percentile confidence associated with a specific cost overrun or schedule delay. This should be compared against the float (schedule) or cost (management reserve) P-Value summary of project health

37 Future Work: Risk Management Standard of Practice Risk Standard Lifecycle Program Components 1. Risk Management Charter 2. Risk Management Maturity Model 3. Risk Management Program Governance 4. Risk Management Information System 5. Standard Risk Management Tools and Techniques 6. Risk Quality Program 7. Integrate organizational learning with risk management 8. Method for description of uncertainties associated with project performance 9. Provide framework to support decisionmaking by describing uncertainties relative to performance thresholds 10. Effective knowledge transfer across project phases Practical Manual / Guideline Risk Management Standard of Practice Vendors Utilities Constructors 37

38 Summary Large projects are prone to severe cost and schedule overruns An absence of dedicated project risk management causes cost / schedule overruns Dedicated and properly applied risk management can improve project cost and schedule delivery certainty Let s stop managing projects as a series of surprises! Contact: Sola Talabi PhD MBA MSc BSc RMP Risk Management Consultant, Pittsburgh Technical Risk Management Professor Carnegie Mellon University sola@cmu.edu