The Integrated Baseline Review Process

Transcription

1 The Integrated Baseline Review Process The following white paper is meant to help someone prepare for an Integrated Baseline Review (IBR). It will cover the major activities that have to be performed and explain why and how to accomplish them. White Paper Published April 2011 Page 1

2 Contents What Is an IBR?... 3 What Is the Purpose of an IBR?... 3 How to prepare for an IBR Establishing the Performance Measurement Baseline (PMB)... 4 How to develop the Integrated Master Schedule (IMS) and it's importance Required documentation to support an IBR The risk identification and mitigation items that need to be included in the IBR What is the work breakdown structure (WBS) and the WBS dictionary?... 8 What is the CAM notebook and what must be included in it?... 8 Who should be included in the IBR Team?... 9 What should be included in the IBR training?... 9 How does the mock CAM Interview support the IBR and what should be included in it? What is IBR process? What are the benefits of an IBR and how does it help your program? What are the results of an IBR and can you fail one? The following are reference site that contain additional data on IBRs Page 2

3 What Is an IBR? An IBR is a formal review conducted by your customer. If your customer is the government, then they will conduct the IBR. If your customer is a prime contractor to the government and you are a sub tier contractor then the prime contractor will conduct the IBR. In either case, the Defense Contract Management Agency will be invited. The IBR is usually held at the contractor s site and typically lasts from 2 to 5 days. Typically the IBR should be held within 6 months of contract award. This is early enough in the program life cycle to be able to make any changes in the program baseline without significantly affecting cost schedule or scope. The primary purpose of the IBR is to review the contract Performance Measurement Baseline (PMB) as awarded for cost, schedule and scope. This is usually the first and only time all members of the customer and your team are able to sit down together and discuss the "as" negotiated contract scope of work, schedule and associated costs. An IBR is a formal review conducted by the customer program manager and technical staff, jointly with their contractor counterparts, following contract award, to verify the technical content of the performance measurement baseline, and the accuracy of the related resource (budgets) and schedules. An IBR will also be performed when work on a production option of a development contract begins or, at the discretion of the program manager, when a major modification to an existing contract significantly changes the existing PMB. When major events occur within the life of a program, e.g. PDR, CDR, etc., and a significant shift in the content and/or time-phasing of the PMB occurs, the Program Manager may conduct a review of those areas affected by the change with the associated resources and schedules. The intent is for the IBR to be a continuous part of the process of program management by both the customer and the contractor. What Is the Purpose of an IBR? An IBR is a joint assessment conducted by the Customer Program Manager and the contractor to verify the realism and accuracy of the PMB. This involves verifying the technical content of the baseline and assessing the realism and accuracy of the related resources (performance budget and IMS). The IBR is unlike the Validation Review that focuses on EVMS compliance with ANSI/EIA Instead the IBR focuses on assessing the realism of the program baseline. The IBR confirms the program baseline by making sure that all work is defined, scheduled and budgeted. Any completed work is reviewed to insure that the performance taken was collected and measured correctly. That the methods used to determine performance were consistent and used objective metrics. The latest revised estimate to complete (ETC) is reviewed for correctness. A review of the documented ETC process is made to insure that there are established processes in place. In addition, the thresholds for performing ETCs are reviewed to determine when and where ETCs are performed. How often are program ETCs done, at what point does a CAM perform an ETC on his effort. These thresholds can be time related, schedule cost index or as a plus or minus percent change of the EAC. Page 3

4 All program related cost, schedule, technical, resource, etc. risks are identified and their mitigations are reviewed. All Earned Value Management System (EVMS) documentation will be reviewed for its accuracy and compliance with ANSI 748. The use and understanding of all existing EVMS documentation will be reviewed during the CAM interview process. How to prepare for an IBR. Before an IBR can take place, a PMB must be in place to be able to measure performance. At the start of a program the PMB is usually the negotiated contract value less withholds for profit and management reserve. The Integrated Master Schedule (IMS) is the center piece of an Earned Value Management System. Without a correctly linked detailed schedule, accurate performance cannot be taken. The IBR Team will review the IMS and any supporting schedules in detail for correct logic, and to insure that all contract deliverables are included. Documentation is critical because the reviewers can only follow the contractor s documentation during the IBR. At a minimum, there must be an Earned Value System Description which describes the EV System in general. Supporting the System Description will be desktop procedures which provide detail descriptions of all key EVMS processes to include program startup, monthly actions, ETCs, and change activities. All Program risks must be identified and mitigation plans must be done prior to the IBR and resulting documentation must be made available. The WBS must be broken down to the lowest level required to support the control accounts and work packages. A CAM Notebook with all necessary supporting documentation must be developed. The composition of the in-house team that will support the review must be determined. A mock CAM interview must be planned. The interview will be done by individuals from your company who will act as if they were the actual government interviewers. The mock interviews are to prepare the CAMs for the real interviews. Establishing the Performance Measurement Baseline (PMB). In order to establish the PMB we begin with the total negotiated contract price. We then remove all fee and/or profit to get to the Contract Budget Base (CBB). If there is any Authorized unpriced work, it is added in. Page 4

5 The result is the Negotiated Contract Cost (NCC). The estimated cost negotiated in a cost-plusfixed-fee contract or the negotiated contract target cost in either a fixed-price-incentive contract or a cost-plus-incentive-fee contract. Management Reserve (MR) is then removed. MR is the portion of the total budget withheld for management control purposes, rather than designated for the accomplishment of a specific task or a set of tasks. It is held and applied through a disciplined process to any additional work that is to be accomplished within the authorized work scope of the contract or applied to accommodate rate changes for future work. It may not be used to offset or minimize existing cost variances. The result is the Performance Measurement Baseline (PMB) which is the time-phased budget plan for accomplishing work, against which contract performance is measured. It includes the budgets assigned to scheduled control accounts and the applicable indirect budgets. For future effort, not planned at the control account level, the PMB also includes budgets assigned to higher level CWBS elements, and to undistributed budgets. It does not include management reserve. Undistributed Budget (UB) is a temporary holding account for budget for authorized work that has not yet been planned in detail at the control account or summary level planning package level. The PMB is then further reduced by splitting out distributed and undistributed budget. Distributed budget is for Control Accounts (CA), (formerly called Cost Account) which are a management control point at which budgets (resource plans) and actual costs are accumulated and compared to earned value for management control purposes. A control account is a natural management point for planning and control since it represents the work assigned to one responsible organizational element (or integrated work team) for a single program WBS element. Summary Level Planning Package (SLPP) is an aggregation of work for far-term efforts, not able to be identified at the control account level, which can be assigned to reporting level WBS elements (and is therefore not undistributed budget ). Work Package is a natural subdivision of control accounts. A work package is simply a task/activity or grouping of work. A work package is the point at which work is planned, progress is measured, and earned value is computed. It can be translated into different terms in different companies and functions. It can be a design job, a tool design package, a build-to-package, a shop order, a part number, a purchase order or any other definable task/activity at whatever level control is normal for program management within the company. A Planning Package is a holding account (within a control account) for budget, for future work that is not yet practicable to plan at the work package level. The planning package budget is timephased in accordance with known schedule requirements (due dates) for resource planning and the plans are refined as detail requirements become clearer and the time to begin work draws nearer. A company may elect to break the work assigned to a control account into smaller groupings of tasks/activities, i.e., multiple planning packages, for internal planning and control reasons. Discrete Effort are work packages and planning packages (or lower level tasks/activities) that are related to the completion of specific end products or services and can be directly planned and measured. Apportioned Effort (AE) are a method of planning and measuring the earned value for effort that is both (a) related in direct proportion to measured effort and (b) by itself is not readily measurable or broken into discrete work packages. The budget for AE is time-phased to the base accounts and the earned value is directly proportional to performance on the base account. The normal method for planning and statusing is to apply a percentage against the base account. Page 5

6 Level of Effort (LOE) is effort of a general or supportive nature which does not produce definite end products and cannot be practically measured by discrete earned value techniques. Earned value is measured by the passage of time. How to develop the Integrated Master Schedule (IMS) and it's importance. The IMS is a networked description of tasks with defined interdependencies capable of determining the critical path. The IMS typically starts with an Integrated Master Plan (IMP). The IMS is an extension of the information contained within the IMP or high-level program plan, reflecting the events, significant accomplishments, and criteria identified in the IMP. The IMS should describe a realistic and supportable schedule consistent with the IMP. In other words the IMS is intended to show how and when the IMP is accomplished. The IMS should consist of Master and Summary schedules and related subordinate schedules that provide a logical sequence, at a minimum, from the Master to the detailed work package and planning package levels. The critical path is a sequence of discrete tasks/activities in the network that has the longest total duration through the contract or project. Discrete tasks/activities along the critical path have the least amount of float/slack. Your companies EVMS requires the integration of cost and schedule. The EVMS can only be as good as your IMS because of this integration; the IMS becomes the lynchpin of the EVMS. Required documentation to support an IBR. The following is a short list of the documentation required to support an IBR. The final list is unique to the products and services that the actual program is performing. The Negotiated Contract specifies many EVMS related items. Among these are the specific SDRLs and their frequency and tailoring. A Statement of Work (SOW) is a document that captures the work activities, deliverables and timeline that a vendor will execute against, in the performance of work for a customer. Detailed requirements are usually specified in a Statement of Work, along with many other terms and conditions. The System Description describes how EVMS will be implemented. It is at a level that conceptually describes the EVMS but does not get as detailed as the Desktop Procedures. Desktop Procedures explain how the items in the System Description will be implemented at the program level. These documents are software specific as opposed to the System Description which is generic in nature. Relevant Directives are issued by the Program or upper level management to clarify aspects of the EVMS. The Contract Work Breakdown Structure (CWBS) is the lowest level at which cost and scheduled data is collected. Page 6

7 The WBS Dictionary is a written description of the task represented by each CWBS element within the context of the SOW reference, Contract Line Item Number (CLIN), specification number, etc., associated with the WBS element. The Responsibility Assignment Matrix (RAM) correlates the work required by a Contract Work Breakdown Structure (CWBS) element to the functional organization responsible for accomplishing the assigned tasks. The RAM is created by intersecting the CWBS with the program Organizational Breakdown Structure (OBS). This intersection identifies the Control Account. Organizational Breakdown Structure (OBS) is a functionally-oriented breakdown of the contractor s organization established to perform the work on a specific contract. Any CAM specific items that reflect how and what each CAM is doing to support the program. The risk identification and mitigation items that need to be included in the IBR. Risk identification and assessment are a critical focus and result of the IBR. Once identified, risks generally may be categorized into one of five areas: technical, schedule, cost, resource, and management processes. Each risk area should be evaluated and documented using the evaluation criteria established in IBR preparation. The following are brief discussions of each of the types of risk. Technical Risk - The ability of the project s technical plan to achieve the objectives of the scope of work. Technical risk includes the effects of available technology, software development capability, design maturity, etc. Schedule Risk - The adequacy of the time allocated for performing the defined tasks to successfully achieve the project schedule objectives. Schedule risk includes the effects on the schedule of the interdependency of scheduled activities to achieve project milestones and support the PMs ability to identify and maintain the critical path. Cost Risk - The ability of the PMB to successfully execute the project and attain cost objectives, recognizing the relationship between budget, resources, funding, schedule, and scope of work. The quality of the estimates affects the cost risk, which includes the assumptions used for both estimates and resource allocation on the budgets for work items. Resource Risk - The availability of personnel, facilities, and equipment, when required, to perform the defined tasks needed to execute the program successfully. Resource risk includes the effect of external factors such as loss of availability to competing programs or unexpected downtime that could preclude or otherwise limit the availability of the resources needed to complete planned work. Management Processes Risk - The degree to which the management processes provide effective and integrated technical/schedule/cost planning and baseline change control. Management processes risk includes the ability to establish and maintain valid, accurate, and timely performance data, including data from subcontractors, for early visibility into risks. Risk Mitigation Actions documents and classifies risks associated with the PMB. The PMs should document risks from the IBR in risk management planning. Each risk addressed in risk management planning should be classified as to their probability of occurrence, consequences, handling, and identification of the individuals responsible for the actions for mitigation. Page 7

8 What is the work breakdown structure (WBS) and the WBS dictionary? Mil Standard 881C is approved for use by all Departments and Agencies of the Department of Defense for WBS development. This standard addresses mandatory procedures for most programs. It also provides guidance to industry in extending contract work breakdown structures to lower levels for detailed reporting. A Work Breakdown Structure (WBS) provides a consistent and visible framework for defense materiel items and contracts within a program. The standard offers uniformity in definition and consistency of approach for developing the top levels of the WBS. The benefit of uniformity in the generation of work breakdown structures and their application to management practices will be realized in improved communication throughout the acquisition process. The WBS Dictionary is a written description of the task represented by each CWBS element within the context of the SOW reference, Contract Line Item Number (CLIN), specification number, etc., associated with the WBS element. The Program Manager and the CAMs are responsible for ensuring the adequacy and accuracy of the WBS task descriptions contained in the Dictionary. This includes comparing the contract statement of work with the WBS Dictionary to ensure traceability. All contract work must be included and described in the WBS Dictionary. This document will be a critical reference for any contract scope discussions. As part of the program s WBS Dictionary a SOW/WBS cross reference matrix should be developed. This matrix shows where the program SOW fits into the program WBS. In addition, when completed, the matrix insures that every item in the SOW has a location within the WBS. What is the CAM notebook and what must be included in it? CAMs are required to maintain current, accurate notebooks of a variety of management data. This data is listed below: The SOW that the CAM is responsible for to perform must be included so that there is no misunderstanding or confusion as to the nature of the CAM's work scope. The SOW/WBS matrix which shows where in the WBS the CAM is doing the actual work. A copy of the WBS Dictionary which shows what work is being done at each WBS level. An EOC that shows the hours and dollars that the CAM has to perform the work assigned. An EOC breakdown for each WBS that the CAM is responsible for. All related schedules that the CAM uses to perform his job. The CAM should insure that only schedules that are used by the program and which also support the IMS are used. The use of any unofficial schedule will result in a significant unfavorable write up and a Deficiency Action report. Work Authorization Document (WAD) which is a document produced by the contractor and used internally to authorize and detail what each CAM is expected to do on the program. Included in the WAD are details concerning the task work to be performed, a detailed list of task outputs, CDRL items, and other task-specific deliverables. It also includes the associated Page 8

9 budget for the task and authorizing signatures that confirm that the staff member is authorized to work on the task. It should not include the number of hours that may be charged. The Government IBR team evaluates the WAD to ensure that the work documented is truly representative of the work to be performed, that the proper signatures are present, and that the deliverables listed are complete and accurate. Control Account Plan (CAP) documents the underlying work packages and provides the CAM with a source to evaluate his or her BCWS, ACWP, and BCWP (Earned Value) within each of the CAM s Control Accounts. Baseline Change Requests (BCRs) are used to authorize any changes, once the Performance Measurement Baseline (PMB) is frozen or established. Any cost and/or schedule changes that are processed through formal change control procedures usually referred to as BCRs. These authorized changes must be incorporated into the PMB in a timely manner and reflected in both budgets and schedules. Any supporting data used by the CAM. In some cases, desktop metrics are used by the CAM to analyze and take performance, this and any other supplemental data used by the CAM to perform their responsibilities should be maintained in the CAM Notebook. Working in conjunction with this hardcopy notebook and the artifacts enclosed, a Virtual/Intranet Portal based CAM notebook is often established. This virtual notebook is CAM specific and contains electronic copies of EVMS artifacts of a recurring nature that would make hardcopy retention very tedious with massive amounts of paper products. The virtual notebook also insures that all data used is the most current and up to date. Who should be included in the IBR Team? Participants should be identified based on their programmatic or technical expertise, as required for the review. Disciplines include program management, business management, subcontract management, and technical management (e.g., system engineering, software engineering, manufacturing, integration and test engineering, and integrated logistics support). When appropriate, the team should include subcontractor personnel. The resulting size and composition of the team should reflect the PMs objectives, expectations, and risk assumptions. What should be included in the IBR training? Training is essential to ensure that the IBR team is ready for review and the resulting data analysis. The Program Manager should conduct joint training in which all members of the IBR team participate. The training should provide enough information so that the team can mutually understand the cost, schedule, technical, and management processes used on the project. The essential elements of training include the following: PMs Expectations (Program Manager, Executive Staff) IBR objectives IBR approach and expectations Risk identification and documentation Management Processes (Business Management, Contracts, Cost/Financial & Schedule Analysis, Schedulers) Baseline maintenance Risk management Page 9

10 Business processes (including EVM) Project Management Aspects (CAMs) Statement of work/statement of objectives Work breakdown structure dictionary/matrix Work authorization document Control account plans Terms and acronyms Funding Budget and schedule baselines Subcontractor management Management reserve Mock interview How does the mock CAM Interview support the IBR and what should be included in it? The mock CAM interview is a one on one interview given to each CAM selected to be a part of the actual IBR review. This interview should be done before the actual IBR to give each CAM enough time to incorporate any deficiencies found in the mock interview process. It is recommended that the mock interviewer be someone that the CAM is not familiar with, a complete outsider is best. The mock CAM interview is meant to prepare the CAM for the actual IBR interview that will be done at the time of IBR by representatives of the customers IBR team. The interview should be the last portion of the CAM training prior to the actual IBR. The interviewer should ask how the CAM manages this work on a daily basis and should try to draw the CAM out and have them explain how they manage their work. If allowed by the customer s IBR team, the CAM should prepare a set of briefing charts that cover the items listed below. By using this briefing at the start of the CAM interview the CAM can answer most of the questions that will be asked and answer them up front without waiting for the interviewers to ask them. This allows the CAM to control the start of the interview process and get his/her feet on the ground before taking on ad hoc questions from the interviewers. The questions and statements below are provided to ensure that you cover the areas needed to assess the adequacy of the baseline. -Ask about the CAMs Scope Of Work and Work Authorization Process. -Discuss how the CAM was provided the scope of work. - The CAM should be able to show a Statement of Work (SOW) paragraph, Contract Work Breakdown Structure (CWBS) narrative, and some form of work authorization document for each control account. -Have the CAM discuss how the scope of work is related to the control account plan(s). - The SOW defines scope of work. The CWBS and the CWBS Dictionary (if applicable) defines the scope of work for each WBS element. - The Work/Budget Authorization document(s) should show resources (in hours or dollars), period of performance, and a short narrative describing the scope of work. -Discuss how the CAM ensures that all elements of the authorized work scope are planned into control accounts. - Control Account Evaluation: Scheduling; Resource Allocation and Time Phasing; Earned Value Methodology; Baseline Management. -Describe the resource development process: How did the CAM come to agreement on the scope, hours, dollars, and schedule? - Program Office, CAM, Functional manager joint agreement -Discuss the schedule milestones that were used for applicable control account planning. The manager should discuss: Page 10

11 How their manning levels support schedule milestones (vertical integration). How their schedule is impacted by or impacts other organizations (horizontal relationships). How they support and are constrained by the horizontal and vertical schedule relationships Intermediate schedule constraints Relationship to Integrated Product Development Management Logical relationships of work packages/control accounts to milestones Level of Effort (LOE) tasks support a schedule, the manager should be able to show you the activity they support on the intermediate or master schedule. -Discuss with the CAM how individual work packages are time phased to achieve the control account and intermediate schedules. -Have the CAM describe any management challenges to meeting the cost, schedule and/or technical aspects of the plan? Identify any areas of risk or possible cost avoidance within the defined scope of work and see if they have been quantified in terms of potential cost or schedule impact to the contract. -Discuss how the CAM chose earned value methods for tracking progress. Are the milestones used for measuring accomplishment (earned value) logical and objective? Level of Effort (LOE) versus Discrete Discuss with the CAM the possibility of converting LOE activity into discretely measured work packages. Talk about the work content of the LOE and try to identify events that are measurable. -Have the Reviewer Ask Some Questions Not Covered by the CAM s Briefing What is IBR process? The Baseline Discussion with the CAM is the key event in the IBR. During this period the two technical team members (customer and contractor) should cover the key aspects of the contractor s planning of the contract scope of work. The discussion should center on the following items. The technical content of the control accounts/work packages relative to the manager s authorized work scope to ensure that all of the authorized work is planned. The integration of the work schedule within the control account with the IMP/IMS requirements. This ensures that working level plans will support contractual requirements. The application of resources (labor, materials, subcontractors, etc.) to the scheduled work. Sufficient resources should be authorized to provide the manager the opportunity to accomplish the plan. The identification, categorization and quantification of any risk elements contained within the plan for the control account. The identification, categorization and quantification of any cost avoidance opportunities within the plan for the control account. Risks and concerns Out Brief Results with List of Action Items Page 11

12 What are the benefits of an IBR and how does it help your program? The IBR is a tool that should be used as necessary throughout the life of the contract. Key benefits of the IBR are: Joint understanding of program risks Management insight into the planning assumptions and the resource constraints of the baseline Comparison of expectations so that any differences can be addressed early in the planning phase Correction of baseline planning errors and omissions In-depth understanding of developing variances and improved early warning of significant variances Targeting of resources to address challenges and mitigate risks Mutual commitment by the joint team to manage to the baseline Programs are more executable. What are the results of an IBR and can you fail one? At the end of the IBR, both the customer and the contractor should agree on a plan to track and close all action items, ensuring that an individual has been assigned the responsibility to resolve each action item. All risk evaluations should be summarized, analyzed, and briefed to senior management within the company and to the customer senior management. Any newly identified risk that is significant enough for risk management and mitigation should be added to the formal risk management plan. No formal IBR report is required for external distribution however the customer should write a memo for the record and attach all documentation for the official program files. There is no pass or fail to an IBR however, the measure of a successful IBR is when both customer and contractor program managers can answer this question with confidence and know where and which risks lay ahead. The team s assessment of the BCWP measurement technique should be documented and evaluated. Additionally, the IBR team should assess the MR with respect to program risk that is unaccounted for in the PMB. To complete the IBR in a reasonable time frame, anything that does not support the intent of the IBR should be moved outside the review. Any system deficiencies shall be recorded on a Corrective Action Request (CAR). Page 12

13 The following are reference site that contain additional data on IBRs. List IBR Related Data Defense Acquisition Guidebook For the IBR (Main Page) The Program Managers Guide to the Integrated Baseline Review Process IBR Sample Out Brief Integrated Baseline Review (IBR) Toolkit l pdf A guide for implementing project performance measurement baselines IBR Team Handbook IBR Team Handbook Program Managers guide to the IBR PDFs30/program_managers_guide.pdf Center for Earned Value Management (CEVM) IBR Toolkit Page 13

14 pdf EVM Glossary The IBR: Your Insurance Policy for a Sound Baseline SMC IBR Handbook Page 14