AzTech International goaztech.com

Transcription

1 Integrated Baseline Review (IBR) Training: Exploding the IBR Big Bang Theory Presented by: Luis C Contreras, Principal Consultant June 2014 AzTech International goaztech.com 1 Our Background For over 20 years, AzTech has worked with Government & Contractors on major EVMS reviews Implementing EVMS Building & assessing Integrated Master Schedules Conducting Schedule Risk Assessments (SRAs) Conducting Integrated Baseline Reviews (IBRs) Conducting EVMS Compliance Reviews Performing EVMS Independent Assessments & Self-Assessments Government Navy/NAVAIR Air Force DCMA DOE Prime Contractors Sub-Contractors 2 1

2 Agenda What is an IBR? IBR Big Bang Theory IBR Timeline Pre-Award Post-Award Phase I - Documentation Quality and Integration Review Phase II - Control Account Level Discussions IBR Exit Briefing / Follow-up Training, Supplier IBRs, & Rolling Waves Sample IBR Templates Benefits of a Streamlined IBR Process Q&A 3 A bad review is like baking a cake with all the best ingredients and having someone sit on it. Danielle Steel (Famous Earned Value Romance Novelist) WHAT IS AN IBR? 4 2

3 What is an IBR? Seeks mutual understanding between government / customer and contractor / subcontractor of the Performance Measurement Baseline (PMB) of these 5 risk areas in executing the PMB: Bottom Line Up Front (BLUF): An IBR answers: An IBR Answers: Do we have an achievable, meaningful, realistic baseline and what are the risks to successful execution? 5 IBR Assesses Risks By answering these Simple questions: Technical - Is the technical scope of the work fully included and consistent with authorizing documents? Schedule - Are key schedule milestones identified and does the schedule reflect a logical flow? Resources Are resources involving cost budgets, facilities, skilled staff adequate and available for performing assigned tasks? Costs - Are tasks well planned and can they be measured objectively relative to technical progress? Management - Are management processes in place and in use? Credit: GAO Cost Estimating Guide 6 3

4 Who s involved? Government & Contractor Program Managers: own the IBR IBR Integrators: Points of Contact for IBR, assign actions, run meetings, manage data calls, coordinate CAM discussions Contractor & Government Core Teams: Integrators, Technical (Engineering, Manufacturing), Finance / EVM, and Scheduling Artifact (Data) Experts: Subject Matter Experts (SMEs) on artifacts being evaluated CAM Discussion Leaders: Lead CAM Interviews Control Account Managers: Interviewed to assess their portions of the PMB DCMA: frequently involved on behalf of government customer 8 How is an IBR like the Big Bang? 9 4

5 When does an IBR occur? Traditional IBR Usually within 6 months after contract award Or after a major planning (or replanning) exercise 10 Minimizing the Bang! Traditional: IBRs back-loaded with big IBR finale Inconsistent pre-ibr documentation review Hard to eat the entire elephant during single event Action items typically fixed post-ibr,extends review period Unpleasant surprises anyone? Contract Award IBR Event Big Bang Theory Streamlined: Progressive IBR process with objectives: Providing consistent IBR methodology Driving up-front program integration Providing incremental assessment of readiness leading up to IBR closure Fostering collaborative Government and Contractor process Incremental Review and Discussions IBR Exit Brief Contract Award

6 Top Level View of Process Artifacts Integration Integration Meetings / Workshops (Technical, Schedule, Resource, Cost and Mgmt Processes) CAM Disc Exit Brief IBR Report Ongoing Actions and inputs to the Risk Register (Data Cleanup; Issue, Risk & Opportunity Discovery) 12 Notional IBR Timeline IBR Planning (Phase I Planning) Integration Meeting Program Documentation Quality/Integration Review Period Mo 1 Mo 2 Mo 3 Clean-up Actions Mo 4 IBR Phase I: Program Documentation Quality & Integration Review System Requirements Review Integrated Risk Assessment EVM / IBR Training Schedule Risk Assessment IBR Exit Brief IBR Phase II: Control Account Level Discussions Data Review Mo 3 Mo 4 Control Account Discussions Supplier IBRs

7 Government Prime KTR INT 07 - Integrated Baseline Review Integrated Baseline Review (IBR) Five Steps Contract Award PAC PMA Ownership of the Performance Management Framework Prior to the end of the first Prior to the end of the full accounting period third/forth full accounting period * 1 2 Within 90 days of No later than the end of the 180 Calendar Days from Subcontract Authority to fifth full month accounting Contract Award Proceed period Need for IBR Joint Government & Prime Systems Training Prime Contractor presents their Management Control Processes Baseline development processes TPM/EV methods Supplier/Subcontract management Risk process Joint setup of Business Rhythm timing & content Prime Management System Assessment (MSA) Ensure the Prime level PMB includes entire technical scope & is consistent with contract schedule requirements Demonstrate that, adequate management processes are being employed, adequate resources assigned, and effective EV methods are being used to accurately status contract cost, schedule, and technical performance Kick off Step 4 SRA Process Subcontractor IBRs Prime Contractor conducts IBRs on Subs requiring to EVM report data Subs requiring to EVM flow down requirement (with Government participation) Schedule Risk Assessment Prime Contractor participates in a Government conducted SRA Total Contract IBR Prime Contractor & Government jointly assess the PMB, including all subcontracted effort Risk Assessment IBR is a recurring process triggered by significant contract options, major modifications, or major replans. Credit: NAVAIR AIR-4.2 Cost Department * Flexible depending on whether there was a TD phase 14 IBR Post-Award Steps Form Govt IBR Team Govt IBR Team Orientation Initial Supplier Interface Meeting Form Joint IBR Team Joint IBR Team Training (Process) Define Artifact Quality & Integration Traces Contractor BMO Discussion IPT # 1 Artifact Quality & Traces IBR Exit Briefing Phase II Status Review CAM Discussion Session # 1 CAM Discussion Session # 2 Joint IBR Team Training (Discussion) IBR Data Readiness Review IPT # 2 Artifact Quality & Traces IPT # 3 Artifact Quality & Traces CAM Improvement Efforts CAM Discussion Session # n IPT #N Artifact Quality & Traces Data Improvement Efforts 15 7

8 Organizing for the IBR Small Core Team concept Continuity for next four months Program Control focused: Integrator, Engineering, Finance Roles and Responsibilities IBR Integrator and core team Artifact experts DCMA Organization Either by IPT or risk area IPT or risk topic area organization IBR Team decides Training, set expectations Introduction for overall IBR process Phase I Artifact quality and integration assessment Phase II CAM discussions 16 Quick Quiz True or False: It is common for organizations to fail their first IBR? 17 8

9 Torture the data, and it will confess to anything. Ronald Coase, Economics, Nobel Prize Laureate PHASE 1: ARTIFACT QUALITY & INTEGRATION TRACES Question? 18 Phase I: Artifact Quality Evaluation Emphasis on improving document quality Ensure all risks are identified PMB is completely documented Jointly define standards for evaluation CDRL, DID compliance Clarity, completeness, industry standards Program requirements Evaluation Joint evaluation against standards established Scoring / Action Items Adequate quality is recorded Inadequate (action required to achieve adequacy) Track evolution of artifact improvement 19 9

10 Phase I: Integration Traces Ensure integration of documents that comprise the PMB are accurately and completely flowed to execution level Jointly define proof statements Narrative description of integration points between documents Evaluation Joint evaluation against standards established Scoring / Action Items Adequate trace is recorded Inadequate action required to achieve adequacy Track evolution of integration trace improvement 20 Sample Integration Traces SOW Statement of Work WBS = Work Breakdown Structure SRA = Schedule Risk Assessment CBB Contract Budget Base EAC = Estimate at Completion IMP Integrated Master Plan RAM Resp Assignment Matrix RMP = Risk Mgt Process BCR Baseline Change Request VAR = Variance Analysis Report BOE = Basis of Estimate CAP = Control Account Plan WAD = Work Authorization Document CPR = Contract Performance report MR = Management Reserve 21 10

11 Quick Quiz If you could pick one more artifact from the last slide, which would it be? 22 To put it bluntly, I seem to have a whole superstructure with no foundation. But I'm working on the foundation. Marilyn Monroe (Infamous EV Analyst & Scheduler) INTEGRATED MASTER SCHEDULE (IMS) 23 11

12 Tasks INT 07 - Integrated Baseline Review Baseline Development EVMS 32 Guideline Compliance Identify Project Objectives 1 Statement of Work (SOW) 5 Define Detailed Work Control Account (1) Define WBS (17) Summarize direct costs into WBS WP WP WP Adopt Work Breakdown Structure 2 WBS Dictionary Construct the IMS Network Logic Assumptions (2) Define Project Organization (4) Identify O/H Mgmt (18)Summarize direct costs into OBS Establish Organizational Breakdown Structure 3 Responsibility Assignment Matrix Define Magnitude of Effort & 7 Basis of Estimate 4 Create Schedule Structure Integrated Master Plan (IMP) Assign Resources and Duration 8 (6)Schedule the authorized work (7) Identify physical products/milestones/tech performance goals to measure progress BOMs / Resource Agreements ) Establish a time-phased budget baseline 9) Establish budgets for authorized work (5) Integrate WBS/OBS 10) Identify SOW in discrete work packages to create CAs 11) Sum all WPs & PPs to = CA budget 12) Identify & control LOE 13) Establish overhead budgets 14) Identify MR & UB Adjust Plan to Fit Project Objectives Calculate Dates, Critical Path and Costs Required RISK Management /5 Plan 1/10 1/1 1/1 1/4 1/5 1/7 1/12 1/13 1/16 1/4 1/5 1/14 1/15 1/18 1/5 1/8 1/5 1/8 1/9 1/10 1/5 1/9 1/10 Credit: NAVAIR IPM190 IBR Prep Workshop Performance Measurement Baseline (PMB) Control Account Plans Work Authorization Documents 3 15 (3) Integrate Processes (15) Reconcile target Cost 24 Management System Integration EVMS Guideline Compliance ) Incorporate authorized changes timely 29) Reconcile current to prior budgets 30) Control Retroactive changes 31) Prevent unauthorized revisions to budget 32) Document changes to PMB 22) Generate SV &CV monthly 23) Identify significant differences between plan & actual performance 24) Identify budgeted & applied indirect costs 25) Summarize data elements & variances through WBS & OBS 26) Implement Corrective action 27) Develop estimate of cost at completion 20) Identify unit/equivalent/lot costs 21) Full accountability of material costs (16) Record direct costs (19) Record all indirect costs Credit: NAVAIR IPM190 IBR Prep Workshop 25 12

13 GAO Schedule Assessment Guide: Schedule Best Practices Source: GAO Schedule Assessment Guide Exposure Draft May NDIA PASEG: Generally Accepted Scheduling Principles (GASP) Valid Schedules Source: NDIA Planning & Schedule Excellence Guide (PASEG) 27 13

14 NDIA PASEG: Generally Accepted Scheduling Principles (GASP) Effective Schedules Source: NDIA Planning & Schedule Excellence Guide (PASEG) 28 Generally Accepted Scheduling Principles (GASP) Alignment to: ANSI, GAO, DCMA, & NAVAIR Generally Accepted Scheduling Principles (GASP) ANSI/EIA-748-B Earned Value Management Systems (EVMS) Guidelines US U.S. Government Accountability Office (GAO) 10 Scheduling Best Practices Defense Contract Management Agency (DCMA) IMS 14 Point Assessment Naval Center for Earned Value Management (CEVM) Program Schedule Assessment (PSA) Complete Traceable 1 - WBS 2 - OBS 7 - Interim Milestones 10 - Work Packages & Planning Packages 12 - LOE 3 - Integration 5 - RAM / Control Accounts 6 - Vertical & Horizontal Integration 1. Does the schedule reflect the work to be 1. Capturing all activities. 8. High Duration Tasks with Duration > 44 done? working days (Goal < 5%) 4. Establishing the duration of all activities. 6. Are duration estimates meaningful? 1. Logic Tasks without predecessors and / or 2. Sequencing all activities. 3. Is work sequenced logically? successors (Goal < 5%) 5. Integrating schedule activities horizontally and vertically. 4. Relationship Types Finish-to-Start (FS) 4. Are interdependencies planned in a logical should be the most common (Goal > 90%) manner? 6 - Vertical & Horizontal Integration Transparent 10-Work Package Characteristics, Objective Techniques 2. Sequencing all activities. 2. Leads (negative lag) Distorts the critical path (Goal 0%) 5. Are constraints, leads, and lags justified? 3. Lags (positive lag) Excessive size and / or usage distorts the critical path (Goal < 5%) Valid 9. Invalid Dates Tasks with invalid forecast (incomplete tasks to the left of Time Now) or actual start/finish (completed tasks in the future) dates (Goal 0%) Statused 6 - IMS Provides Current Status for All Authorized Work 9. Updating the schedule using logic and durations to determine dates. 11. Missed Tasks Tasks not completed or forecasted to complete as planned (Goal < 5%) 10. Does the schedule provide logical status and forecasts of completion dates for all authorized work? 14. Baseline Execution Index (BEI) Tasks completed as a ratio to those tasks that should have been completed to date according to the baseline plan (Goal > 0.95) Predictive 6 - IMS provides forecast of completion dates for all authorized work 6. Establishing the critical path for all activities. 7. Identifying float between activities. 6. High Float Tasks with Total Float > 44 working days (Goal < 5%) 9. Are float times reasonable? 7. Negative Float Tasks with Total Float < 0 working days (Goal 0%) 12. Critical Path Test Tests the logic in the critical path (Goal Pass ) 8. Does the critical path make sense; does the 13. Critical Path Length Index (CPLI) scheduling software calculate it? Measures the realism of the critical path (Goal > 0.95) Usable 6 - IMS Effective Planning, Statusing, & Forecasting for Achieving Requirements & Measuring Performance 8. Conducting a schedule risk analysis. 5. Hard Constraints Tasks not designated as 5. Are constraints, leads, and lags justified? As Soon As Possible (ASAP), Start No Earlier Than (SNET), or Finish No Earlier Than (FNET) 2. Are critical target dates identified; are they (Goal < 5%) being used to plan the work? 7. Are resource estimates reasonable; are key Effective Resourced 9 - Control Account Level Budgets & Detail 10 - Control Account Plans (CAPs) 3. Assigning resources to all activities. 10. Resources If loaded, tasks without dollars or hours assigned (Goal 0%) resources available to support the plan? 11. Can the current program schedule be accomplished at an acceptable risk level? 8 - Baselined Performance Measurement Baseline (PMB) Controlled Documented processes for tracing authorized changes & controlling retroactive changes 10. Maintaining a baseline schedule 29 14

15 Generally Accepted Scheduling Principles (GASP) Alignment to: ANSI, GAO, DCMA, & NAVAIR Generally Accepted Scheduling Principles (GASP) ANSI/EIA-748-B Earned Value Management Systems (EVMS) Guidelines US U.S. Government Accountability Office (GAO) 10 Scheduling Best Practices Defense Contract Management Agency (DCMA) IMS 14 Point Assessment Naval Center for Earned Value Management (CEVM) Program Schedule Assessment (PSA) Complete 1 - WBS 2 - OBS 7 - Interim Milestones 10 - Work Packages & Planning Packages 12 - LOE 1. Capturing all activities. 4. Establishing the duration of all activities. 8. High Duration Tasks with Duration > 44 working days (Goal < 5%) 1. Does the schedule reflect the work to be done? 6. Are duration estimates meaningful? Traceable 3 - Integration 5 - RAM / Control Accounts 6 - Vertical & Horizontal Integration 1. Logic Tasks without predecessors and / or 2. Sequencing all activities. 3. Is work sequenced logically? successors (Goal < 5%) 5. Integrating schedule activities horizontally and vertically. 4. Relationship Types Finish-to-Start (FS) 4. Are interdependencies planned in a logical should be the most common (Goal > 90%) manner? 2. Leads (negative lag) Distorts the critical 6 - Vertical & Horizontal Integration path (Goal 0%) Transparent 10-Work Package Characteristics, Objective 2. Sequencing all activities. 5. Are constraints, leads, and lags justified? Techniques 3. Lags (positive lag) Excessive size and / or usage distorts the critical path (Goal < 5%) 30 Generally Accepted Scheduling Principles (GASP) Alignment to: ANSI, GAO, DCMA, & Naval Center for EVM Generally Accepted Scheduling Principles (GASP) ANSI/EIA-748-B Earned Value Management Systems (EVMS) Guidelines US U.S. Government Accountability Office (GAO) 10 Scheduling Best Practices Defense Contract Management Agency (DCMA) IMS 14 Point Assessment Naval Center for Earned Value Management (CEVM) Program Schedule Assessment (PSA) 9. Invalid Dates Tasks with invalid forecast (incomplete tasks to the left of Time Now) or actual start/finish (completed tasks in the future) dates (Goal 0%) Statused 6 - IMS Provides Current Status for All Authorized Work 9. Updating the schedule using logic and durations to determine dates. 11. Missed Tasks Tasks not completed or forecasted to complete as planned (Goal < 5%) 10. Does the schedule provide logical status and forecasts of completion dates for all authorized work? 14. Baseline Execution Index (BEI) Tasks completed as a ratio to those tasks that should have been completed to date according to the baseline plan (Goal > 0.95) 6. High Float Tasks with Total Float > Establishing the critical path for all activities. working days (Goal < 5%) 7. Negative Float Tasks with Total Float < 0 9. Are float times reasonable? Predictive 6 - IMS provides forecast of completion dates for all authorized work 7. Identifying float between activities. working days (Goal 0%) 12. Critical Path Test Tests the logic in the critical path (Goal Pass ) 13. Critical Path Length Index (CPLI) Measures the realism of the critical path 8. Does the critical path make sense; does the scheduling software calculate it? (Goal > 0.95) 31 15

16 Generally Accepted Scheduling Principles (GASP) Alignment to: ANSI, GAO, DCMA, & Naval Center for EVM Generally Accepted Scheduling Principles (GASP) ANSI/EIA-748-B Earned Value Management Systems (EVMS) Guidelines US U.S. Government Accountability Office (GAO) 10 Scheduling Best Practices Defense Contract Management Agency (DCMA) IMS 14 Point Assessment Naval Center for Earned Value Management (CEVM) Program Schedule Assessment (PSA) Usable 6 - IMS Effective Planning, Statusing, & Forecasting for Achieving Requirements & Measuring Performance 8. Conducting a schedule risk analysis. 5. Hard Constraints Tasks not designated as 5. Are constraints, leads, and lags justified? As Soon As Possible (ASAP), Start No Earlier Than (SNET), or Finish No Earlier Than (FNET) 2. Are critical target dates identified; are they (Goal < 5%) being used to plan the work? Resourced 9 - Control Account Level Budgets & Detail 10 - Control Account Plans (CAPs) 3. Assigning resources to all activities. 10. Resources If loaded, tasks without dollars or hours assigned (Goal 0%) 7. Are resource estimates reasonable; are key resources available to support the plan? 11. Can the current program schedule be accomplished at an acceptable risk level? Controlled 8 - Baselined Performance Measurement Baseline (PMB) Documented processes for tracing authorized changes & controlling retroactive changes 10. Maintaining a baseline schedule 32 Integrated Master Schedule (IMS) Networked, detailed tasks to assure successful program execution Traceable to SOW, WBS, IMP (if required), Control Accounts, Work Packages Facilitates effective planning, statusing, and forecasting Significant interdependencies to determine duration & critical path Foundation for Performance Measurement Baseline (PMB) Updated at least monthly with latest forecast and revisions Credit: NAVAIR IPM190 IBR Prep Workshop

17 Living at risk is jumping off the cliff and building your wings on the way down. Ray Bradbury SCHEDULE RISK ASSESSMENT (SRA) Question? 34 Schedule Risk Assessment (SRA) (c) Credit: AzTech NDIA International, PASEG LLC

18 Schedule Risk Assessment (SRA) Credit: NDIA PASEG 36 Schedule Risk Assessment (SRA) Credit: NDIA PASEG 37 18

19 Quick Quiz If the SRA results tell us there is a 10% chance of finishing by the contract finish date, what are our options? 38 Transition from Phase I to Phase II Readiness Review validates that PMB documentation is adequate for PMO and CAM discussions Review of artifacts evaluated and traces performed Review of outstanding action items Team recommendation to IBR leadership that Phase II can begin 39 19

20 Readiness Review Is program ready to move forward to Phase II (CAM discussions)? Document quality Integration traces Action Items Transition to Phase II PMO discussions CAM discussions Phase I and Phase II overlapping is possible Parts of Phase II can proceed before Phase I is completed 40 There's only one interview technique that matters... Do your homework so you can listen to the answers and react to them and ask follow-ups. Do your homework, prepare. Jim Lehrer PHASE 2: DISCUSSIONS (OR INTERVIEWS?) 41 20

21 Phase II: PMO/Business Office Discussions Provides background information so CAM discussions are focused on specific technical issues Explain program unique procedures Work authorization process and common documentation CBB Log and the MR/UB guidance Development and maintenance of CAM EVM information Introduction to the IMS including integration of suppliers / contractor data Subcontracts /supplier management Role of Program Control and CAM PMB maintenance 42 Phase II: CAM Discussions Ensure work is achievable as planned at the control account level CA Selection Moderate to high risk Critical Path Goal ~ 80% of PMB Methods Individual discussions Round table discussions from single IPT / risk area Combination of methods Scoring / Action Items 3 ratings: high risk, medium risk, low risk By risk areas: technical, schedule, cost, resource, management processes Summarized to program level 43 21

22 Overall IBR Scoring 56 Training, Supplier IBRs, & Rolling Waves IBR Training EVMS, Phase I training, Phase II training Subcontractor and Supplier IBRs Sequencing and incorporating into single IBR (if possible) Differences in award dates Firm-Fixed Price (FFP) Suppliers Subcontractor schedule & integration into IMS still required Rolling Wave Planning Converting Planning Packages into Work Packages Follow-on mini IBRs? 57 22

23 Process Document Templates Provide consistent approach Increase efficiency of process Sample IBR schedule Artifact list Data call request Trace Narratives Readiness Review brief CAM questions CAM scoring criteria IBR briefing Memo for Record 58 Sample IBR Risk Assessment Checklist (DoD) Credit: ACC.DAU.MIL 59 23

24 Sample IBR Program Summary Results Credit: GAO Cost Estimating & Assessment Guide 60 Sample Program Risk Analysis Credit: Naval Center for EVM (CEVM) Analysis Toolkit 61 24

25 References PARCA GAO DAU DCMA NASA Navy CEVM NAVAIR Air Force Credit: Performance Assessments and Root Cause Analyses (PARCA) 62 The pessimist complains about the wind; the optimist expects it to change; the realist adjusts the sails. William Arthur Ward CONCLUSIONS 63 25

26 Benefits of Streamlined IBR Process Common understanding of IBR process and expectations Early identification of problems to employ risk mitigation Promotes joint ownership of program goals Enables comparison of IBRs across company or portfolio of programs 64 Streamlined IBR Process Thank you! Luis C Contreras, Luis@goAzTech.com Q&A 65 26