Using Earned Value Management to Manage Risk & Monitor Contract Performance

Transcription

1

2 Using Earned Value Management to Manage Risk & Monitor Contract Performance Breakout Session # B14 Stephen J. Yuter, CPCM, CFCM, PMP, Fellow Acting Director & Head of the Contracting Activity DHHS/Indian Health Service Date: July 28, 2014 Time: 2:30 p.m. 3:45 p.m.

3 OVERVIEW Definition & Principles of EVM EVM Process & Key Questions Traditional Management Systems vs. EVM Performance Measurements FAR & Agency EVM Requirements Contractor s Requirements and Validation Process for EVM Contracting Officer s Responsibilities for EVM 2 2

4 WHY IS THERE A NEED FOR EVM? Need for accurate and consistent status information Numerous complex (and interrelated) projects 70% of projects are over budget and behind schedule 52% of all projects finish at 189% of their initial budget 3 3

5 WHAT IS EVM? A Set of Business Practices integrate scope, schedule, and cost objectives establish a baseline plan for accomplishment of project or program objectives employ earned value techniques for performance management Compares where the project is now with: (1) previous work accomplished and (2) where the project was planned to be Early warning system to detect deficient or endangered progress 4 4

6 EVM DEFINED Earned value is the value of the work that has been completed it represents the amount of the overall project budget that has been earned based on the percentage of the work that has been accomplished EVM enables both the government and the contractor to have clear visibility into technical, cost and schedule planning, performance, and progress on their contracts EVM requires that each element of work be budgeted as each element is completed with clearly defined deliverables, its budgeted costs are earned by placing a cost value on status, a project's performance can be evaluated and communicated 5 5

7 ANSI/EIA-748 STANDARDS Prescribes 32 criteria in five categories: Organization - assign responsibilities for all WBS elements Planning, Scheduling, and Budgeting - measure performance against the integrated performance baseline Accounting Considerations - record project costs as consumed or incurred Analysis and Management Reports - analyze actual performance against the authorized baseline Revisions and Data Maintenance - provide timely approval/rejection of all change requests 6 6

8 Approaches to EVM: ANSI/EIA-748 COMPLIANCE All 32 ANSI Standard criteria apply (OMB full compliance) Criteria are scalable and can be adapted, based on a project s size, complexity, risk, etc. ( Tailored EVM) Selected criteria are applied and additional ones can be added, based on a project s size, complexity, risk, etc. (EVM Lite ) 7 7

9 BRIEF HISTORY OF EVM 1960s 1970s 1980s DoD adopted Cost/Schedule Control Systems Criteria (C/SCSC) as an objective measure of progress Continued use in DoD as a means to offset cost/schedule risk in cost-plus contracts Policy moved Earned Value into all Federal agencies OMB Circular A OMB began enforcement in all civilian agencies 8 8

10 BASIC PRINCIPLES OF EVM Proper use of EVM can provide early warning (forecasting) of impending problems, leaving ample time for corrective action EVM DOES NOT determine a course of action; instead, it simply isolates the cause of project problems or confirms that tasks are progressing as planned EVM is normally not appropriate for contracts that are not performancebased (e.g. FFP or T&M) or where the nature of the work is not measurable (e.g. level of effort, and/or in steady state operations and support); optimal for services contracts providing development & integration Applying EVM principles properly can provide for better control of program/project performance, along with greater accountability and a reduction in risk 9 9

11 BASIC PRINCIPLES OF EVM (CONT D.) Break down the work scope into discrete, measurable elements and assign responsibility Plan and integrate the scope, schedule, and cost into a timephased plan and control changes to the plan Provides the ability to examine detailed schedule information, critical program and technical milestones, and cost data Objectively assess progress and accomplishments Use actual costs, use information to manage Analyze variances from the plan 10 10

12 KEY ATTRIBUTES OF EVM 11 11

13 THE EARNED VALUE PROCESS Define the Work Plan the Work Work the Plan Collect Results Change Control Measure Performance External Changes Analyze Deviations Take Corrective Action 12

14 WHY USE EARNED VALUE MANAGEMENT? Ensures a clear definition of work prior to beginning that work Presents a logical plan for accomplishing the work Provides an objective measure of accomplishments Early and accurate identification of trends and problems Accurate picture of contract status - cost, schedule, and technical Basis for course correction Supports mutual goals of contractor and customer - bring project in on schedule and cost 13 13

15 EVM MEASURES PROGRESS Progress = movement towards a goal to measure progress, there must be a standard against which the movement may be compared EVM establishes that standard as the Performance Measurement Baseline and measures progress against that baseline 14 14

16 PERFORMANCE MEASUREMENT BASELINE Fundamental to earned value management Includes the undistributed budget, all summary level planning package budgets, and all control account budgets not include management reserve) Establishes the contract level timed phased baseline against which contract level earned value metrics are computed Establishes the performance targets for the program 15 15

17 WHAT DO WE MEASURE PROGRESS AGAINST? Performance measurement baseline: integrates three key measurements budget is spread over... time, to accomplish the scope of work against which progress can be measured compares the PLANNED amount of work with what has actually been COMPLETED, to determine if COST, SCHEDULE, and WORK ACCOMPLISHED are progressing as planned work is earned or credited as it is completed (expressed in dollar or hours); the value earned is the WBS budgeted cost of the activity completed to date 16 16

18 KEY QUESTIONS EVM ANSWERS We analyze past performance to help us control the future PAST PRESENT FUTURE Are we on schedule? Are we on cost? What are the significant variances? Why do we have variances? Who is responsible? What is the trend to date? When will we finish? What will it cost at the end? How can we control the trend? The Two Key Questions 1. Did we get what we wanted for what we spent? 2. At the end of the project, is it likely that the cost will be less than or equal to the original estimate? 17 17

19 WAYS OF EARNING VALUE Should be quantitative and discrete whenever possible Discrete = tangible end product e.g. delivery of a specification, vendor parts contract awarded, foundation completed Should be integrated with success criteria or technical measures e.g., successful completion of clean-up, a specific test, reliability growth curve 18 18

20 TRADITIONAL MANAGEMENT SYSTEMS In these systems, you budget work and then record actual expenditures. Example: I budget 5 widgets at 100 hrs. per widget. At the end of the month 400 hrs. had been expended. But what does this mean? Is this really the true status of work? What did I accomplish? GREAT! I'm 100hrs under budget Budget vs. Actual Variance = 19 19

21 MANAGEMENT SYSTEMS MEASUREMENT OF EXPENDITURES Well, I ve spent 400 hrs, does that mean I ve accomplished 400 hrs of work? Actual Cost is not an indication of work progress, only an indication of hours/money spent

22 MANAGEMENT USING EARNED VALUE Earned Value is an objective measure of how much work has been accomplished Example: I plan to build 5 widgets this month Each widget should take 100hrs I will measure Earned Value based on # widgets completed At Month End Budget Plan 500 Earned Value 300 (3 Widgets * 100 hrs.) Actual

23 EARNED VALUE ADDS A NEW DIMENSION TO TRADITIONAL TRACKING SYSTEMS Budget Plan 500hrs Earned Value 300hrs Cost Variance Schedule Variance (200) (100) Actual 400hrs I better figure out what is going on. I ve got 200 hours worth of work to catch up on, and I ve already overspent by 100 hours

24 USING PERFORMANCE DATA FOR DECISION-MAKING Behind Schedule How critical is schedule? Can I afford to work overtime to recover? Can I do tasks concurrently? Are there technical innovations which could speed up the process? Am I gold plating instead of just meeting requirements? Should I do a schedule risk assessment to project impact to program? Over Cost Can I reschedule tasks? (time phasing) Is there a less costly facility I can use? Are there tasks which can be deleted? Should the element be added to my risk management profile? 23 23

25 FIVE BASIC PERFORMANCE QUESTIONS & ANSWERS IN EVM Question Answer Acronym How much work did we plan to do? How much work got done? How much did the completed work cost? What was the total job supposed to cost? What do we now expect the total job to cost? Budgeted Cost for Work Scheduled Budgeted Cost for Work Performed Actual Cost of Work Performed Budget at Completion Estimate At Completion BCWS = PV BCWP = EV ACWP = AC BAC EAC 24 24

26 VARIANCE CALCULATIONS IN EVM Schedule Variance (SV) = EV PV the difference between Earned Value (EV) and Planned Value (PV) Schedule Performance Index (SPI) = EV/PV SPI<1 means project is behind schedule Cost Variance (CV) = EV - AC the difference between EV and Actual Cost (AC) Cost Performance Index (CPI) = EV/AC CPI<1 means project is over budget 25 25

27 MAKING PROJECTIONS WITH EVM Cost Schedule Index (CSI) = CPI x SPI the further CSI is from 1.0, the less likely project recovery becomes Once a project is 10% complete, the overrun at completion will not be less than the current overrun Once a project is 20% complete, the CPI does not vary from its current value by more than 10% The CPI and SPI are statistically accurate indicators of final cost results 26 26

28 CONTRACTOR S VARIANCE ANALYSIS Should address significant variances: Separate discussion of cost and schedule variances Clear description of reason for variance Quantity variances (e.g., price vs. usage) Be specific, not general Include corrective action Technical, schedule, and cost impacts Impact to estimate at completion Should be analyzed by the individual controlling the work 27 27

29 USING EVM PERFORMANCE DATA TO VALIDATE ESTIMATES New estimates at completion take on more meaning when you employ performance information To recover from a CPI of.5, the project would have to work at an efficiency > 1 The Estimate At Completion (EAC) can be predicted by the performance index: EAC = BAC = $20,000 = $40,000 CPI.5 The overrun will never be less than it is at the 15% complete point! 28 28

30 THE NEED FOR ACCURATE EACs New EACs take on more meaning when you employ performance information the EAC can be predicted by the performance index analysis of EAC realism what are the drivers? What can we do about them? Variance at Completion vs. Contractor Loss Positive VAC: EAC < BAC underrun contractor gain Negative VAC: EAC > BAC share area contractor partial loss EAC > ceiling overrun contractor loss (100%) Government develops top-level EAC for comparison: Government should limit progress payments if EAC is greater than contract ceiling Government needs accurate forecast of fund requirements 29 29

31 EARNED VALUE PROBLEM INDICATORS GOAL: to verify that effective variance analysis processes are applied to identify, correct, and report problems Potential Problem Indicators: Zero variances Monthly trends turning negative or downward Schedule variances generally indicate cost will follow Actuals > Latest Revised Estimates (LRE) BCWP increases with no increase in ACWP Negative data elements 30 30

32 STATUTES RELATED TO EVM The requirement for EVM as it is practiced today can be traced to three public laws: Government Performance and Results Act of 1993 (GPRA) Establish performance standards for Federal budget Federal Acquisition Streamlining Act, Title V of 1994 (FASA V) Report cost, schedule, and performance goals and evaluate progress Information Technology Management Reform Act of 1996/ Clinger-Cohen Act (ITMRA/Clinger-Cohen) Report performance information systems acquisition 31 31

33 EVM REQUIREMENTS OMB Circular A-11, Part 7 requires the use of EVM in the performance management of major investments and major systems in development, modernization, or enhancement (DME) or in mixed life-cycle "Agencies must use a performance-based acquisition management system to measure the achievement of cost, schedule, and performance goals, and achieve (on average), 90% of those goals Contractors Earned Value Management System (EVMS) must comply with the guidelines set forth in the American National Standards Institute (ANSI)/Electronic Industries Alliance (EIA)-748 Standard FAR (Pre-IRB), (Post-IRB), and (EVMS) Federal agencies requires its use on major programs (i.e. IT and non-it investments $100M or greater) and on their associated contracts with a contract price of $20M or greater; continued surveillance to ensure compliance 32 32

34 EVM REQUIREMENTS (CONT D.) Contract Total Value EVM Compliance CWBS & IMS CPR (monthly) CFSR (quarterly) $50M and greater $20M and greater but less than $50M Required Required Required Required Required Required Required Required Less than $20M Optional (at discretion of PM) Required if PM requires EVM Required if PM requires EVM Required 33 33

35 INTEGRATED BASELINE REVIEW (IBR) Determines that the Performance Measurement Baseline is responsive to the scope of work required Joint assessment of Coverage of SOW Scheduling of work Resource allocation Earned Value methodologies Technical content of PMB Realism and Risks Should be conducted pre-award but no later than 180 days post-award 34 34

36 THE CONTRACTOR S EVMS Describes system in sufficient detail to determine compliance Includes prior Certification/Acceptance If no EVMS is in place, Contractor must submit a comprehensive plan for compliance System description Necessary modifications Compliance map Process descriptions Recent evaluation or self-assessment results Identify major subcontracts for application of criteria Verify baseline integrity is maintained Contains plan and procedures for surveillance and self assessment 35 35

37 THE CONTRACTOR S EVMS (CONT D.) Surveillance Plan Methods and techniques Inclusion of new scope Customer coordination Schedule Analysis Ensure that reliable and timely cost, schedule, and technical performance information is generated Baseline Budget Control Accounts Baseline Schedule Work Measurement by Control Account Work-hours, dollars, units, etc

38 EVM VALIDATION Required for all companies with a EVMS cost-plus contract over $50M Validation is achieved by conducting a formal review of the processes defined and used by the contractor to comply with the EVMS guidelines in ANSI/EIA-748 Determines that the contractor is using the system as one of its primary program management processes has properly implemented the system on the contract is using the data from its system in reports to the government Single validation required for each company resulting in a compliant EVMS Note: an EVMS Compliance Review is the same as a Validation Review 37 37

39 EVM VALIDATION PROCESS A formal recognition of certification by an independent party that company s EVMS meets the guidelines in ANSI/EIA-748 A formal Government review conducted at a contractor s facility to assess the contractor s proposed EVMS compliance with ANSI/EIA-748 A recurring process by an independent party (normally DCMA) assessing the continuing compliance of the company s EVMS with ANSI/EIA-748 and the company s written system documentation n independent party (normally DCMA) assessing the continuing compliance of the company s EVMS with ANSI/EIA-748 and the company s written system documentation 38 38

40 Objectives: EVM VALIDATION REVIEW Evaluate management system capabilities against ANSI/EIA-748 Assess the description of the management system to determine if it adequately describes the management processes demonstrated during the review Evaluate the application of the management system on the contract being reviewed Validation Process: begins as soon as practicable following the implementation of the EVMS and IBR review system documentation reviews, data traces, and interviews with contractor personnel organized to follow the system approach taken by the contractor 39 39

41 FAILING THE VALIDATION Potential of extra withholding on payments that could be as much as 10% additional Disadvantage in bidding future EVM contracts Potential for additional oversight on existing EVM contracts Brings in question a company s ability to control and manage multi-million dollar programs Notable examples: Lockheed Martin & Bell Helicopter 40 40

42 EVMS IS IMPORTANT TO EFFECTIVE CONTRACT ADMINISTRATION EVMS gives early warning of potential problems As a Contract Specialist or Contracting Officer, you are a member of the Integrated Project Team (IPT) It is your job as the CS/CO to protect the Government s rights if performance does not show satisfactory progress! You must work with the other IPT members and the Contractor to avoid or mitigate performance problems 41 41

43 WHAT ARE THE CO S RESPONSIBILITIES REGARDING EVMS? Contracting Officer s Pre-Award Responsibilities as a Member of the IPT: Identify application of the Earned Value Management System in acquisition plans, solicitations and contracts Define reporting and electronic submission requirements Define evaluation criteria for source selections Factor EVMS into contract management planning 42 42

44 WHAT ARE THE CO S RESPONSIBILITIES REGARDING EVMS? (CONT D.) Contracting Officer s Post-Award Responsibilities as a Member of the IPT: Integrate performance monitoring with EVMS into Award/Incentive plans Monitor performance with IPT using the EVMS Act when performance is at risk (communication with contractor, show cause and cure notices, etc.) Evaluate the Contractor s EVMS implementation plan Evaluate the Contractor s EVM systems for compliance with the standard Conduct Integrated Baseline Reviews to ensure all the work scope is included in the baseline 43 43

45 EVM IMPACT ON MISSION By utilizing EVM properly, an agency can improve its ability to track and report on investment cost, schedule, and performance variances (agency projects must stay within a 10% variance) allow PMs and project team members to optimize resources and deliver capital investment projects on-time, on-budget, and within scope operate by tracking its programs health beyond pure financials ensure its customers requirements are satisfied 44 44

46 LIMITATIONS OF EVM Quantifying/measuring work progress can be difficult Time required for data measurement, input, and manipulation can be considerable EVM becomes problematic when Inadequate requirements definition Incomplete WBS Accounting systems that cannot address the WBS Lack of management attention to project progress Failure to respond to warning signs Failure to reconcile baseline to funding shortfalls (re-baseline) 45 45

47 EVM IS SUCCESSFUL When management embraces it When the program uses it to manage When all program levels understand how it works 46 46

48 SUMMARY EVM helps reduce the guesswork in: measuring performance forecasting Aids in getting beyond misleading measures of progress - provides a common denominator for work scope, schedule, and resources Reasons to use EVM: Good project management practices OMB/FAR requirement Should be incorporated into contracts for major acquisitions (cost-reimbursement) 47 47

49 EVM RESOURCES Practice Standard for Earned Value Management DoD Earned Value Management ANSI/EIA-748 Guide EIA748.pdf 48 48

50 Q & A 49 49