How to Pass an Estimating System Review. February 27, 2013

Transcription

1 How to Pass an Estimating System Review February 27, 2013

2 ANNOUNCEMENTS

3 Mr. James J. Gallagher Partner, McKenna Long & Aldridge LLP Ms. Jami Levy Pricing Manager, Northrop Grumman Corporation Mr. David E. Trotta Estimating Manager, ViaSat Inc.

4 The DOD Business Systems Rule February 27, 2013 Jay Gallagher McKenna Long & Aldridge LLP

5 Overview of Final Rule Applicable through clauses in DOD, CAS-covered contracts awarded after May 2011 Imposes numerous system requirements, including SOX like internal controls procedures Approval by ACO/DACO/CACO based upon DCAA opinion Applies to the contractor, meaning the legal entity Rule citations Interim rule (76 Fed. Reg. 28,856 (May 11, 2011)) Final rule (77 Fed. Reg. 11,355 (Feb. 24, 2012))

6 Overview of Final Rule (cont.) Requires existence of six approved systems Accounting (includes internal controls and billing) Purchasing Estimating Materials Management and Accounting Government Property Earned Value Management

7 Overview of System Approval Fundamentals

8 System Approval Fundamentals: No Significant Deficiency Approval requires that no significant deficiency exists Affirmative finding What if no finding? Initial assessment by DCAA Each system is to be approved by ACO/DACO/CACO Must rely on DCAA input unless basis exists to reject Rejection requires board of review Potential for rubber stamping DCAA assessment

9 No Significant Deficiency (cont.) The DFARS Rule defines a significant deficiency as: A shortcoming in the system that materially affects the ability of officials of [DOD] to rely upon information produced by the system that is needed for management purposes The DFARS Rule does not define material Focus is on DOD s perspective SOX/GAAP standards unlikely determinative Comments to Final Rule suggest any failure to comply with a system criterion is material and a significant deficiency

10 No Significant Deficiency (cont.) DCAA guidance Significant deficiency results from a material weakness Material weakness means A deficiency or combination of deficiencies, in internal control over compliance such that there is a reasonable possibility that a material noncompliance with a compliance requirement... will not be prevented, or detected and corrected on a timely basis Material noncompliance means a material noncompliance with DFARS criteria

11 No Significant Deficiency (cont.) DCAA guidance (cont.) Material A reasonable possibility that noncompliance with DFARS criteria will result in material noncompliance with... applicable government contract laws and regulations, thus materially affecting reliability of the data produced by the system Is not financial reporting standard Monetary impact not required

12 No Significant Deficiency (cont.) DCMA guidance Consistent with DCAA guidance Material weaknesses include impacts on any of the following Summary Information the government relies upon to fund contracts Information the government relies upon to price contracts Judgmental While circular in concept, the fundamentals are A noncompliance with a DFARS criterion That is a material noncompliance with law, regulation or contract That impacts the reliability of contractor data

13 No Significant Deficiency (cont.) DCAA s low materiality threshold, combined with a vague deficiency standard, means that the burden will be on the Contractor to show a deficiency is not significant Based on the facts and circumstances; e.g., System design issue Final control point vs. Initial entry subject to final control Compliance issue Number of occurrences (e.g., bad statistical sample) Amount involved Based on system criteria in the relevant system clauses in covered contracts

14 Approval Fundamentals: Key System Criteria Electronic systems that have Strong internal controls Function with minimum errors Are integrated, reconcilable and auditable Robust written policies and procedures that clearly describe the relevant systems and compliance requirements Robust practices for ensuring compliance Training Internal compliance reviews Internal controls (e.g., approval levels) Appropriate and timely resolution of deviations

15 Accounting System Includes cost accounting, cost allowability, billing and ethics Selected areas of focus CAS compliance for contract element cost allocations (among CLINs, sub-clins or units, etc.); relates to EVMS Requirements Indirect cost allocations Consistent practices Reflective of beneficial/causal relationship Cost allowability, including costs of subcontracts, bid and proposal, professional and legal services, travel and insurance

16 Accounting System (cont.) Selected areas (cont.) Internal controls Automated and integrated systems with built-in segregation of employee responsibilities Internal audit activity Functionally and organizationally independent Has requisite specialized knowledge to review government contract compliance Periodically performed System output can be and is reconciled Financial ledgers vs. contract ledgers Sub-ledgers vs. general ledger

17 Accounting System (cont.) Selected areas (cont.) Internal controls (cont.) Error correction Timely Accurate Ethics practices Billings Timely investigation of issues Documented resolution and corrective action Appropriate disclosures and follow-up Errors minimized Reconciled to ledgers

18 Purchasing System Selected areas of focus Integration of estimating and purchasing systems Enforcing TINA requirements Timely and adequate cost and price analysis Contemporaneous and written justification for lack of competition Authorized personnel make purchases based on authorized requisitions Subcontracts executed before any work is performed Update policies and procedure routinely and immediately following new requirements added to FAR/DFARS Training

19 Estimating System New requirement (DFARS (d)(1), d(3), (g)) Disclose system in writing to ACO Comply with the system as disclosed Timely disclosure to ACO of significant changes Significant deficiencies can prompt withholds in accordance with DFARS , when included in the contract Estimating system integration with Accounting system Purchasing system Budgeting process

20 Estimating System (cont.) Selected areas of focus Internal audits of estimating system Too infrequent Lack adequate scope Inadequately documented corrective action Independent management review Insufficient management oversight concerning high-risk proposals Unclear lines of authority for proposal approval Lack of documentation of review/approval process

21 Estimating System (cont.) Selected areas (cont.) Continuous system maintenance/improvement Use internal audits to identify weaknesses and implement corrective action(s); document improvements Comprehensive training plan Timely policy revisions/distribution of updates Sharp focus on proposal format and content Comply with FAR , Table 15-2 (when TINA applies)

22 Government Property System Selected areas of focus Failure to properly tag and track government property Failure to appreciate the breadth of what is government property under cost-type and T&M contracts Use of government property for commercial operations without Government approvals Comingling of contractor and government property; capital type projects that integrate government funding susceptible to challenge

23 Government Property System (cont.) Selected areas (cont.) Government property terms and conditions are not flowed down to subcontractors Lack of audits/due diligence to ensure government property systems are functioning Assumption/reliance upon accounting/purchasing systems to track government property Can result in difficult process to identify government property when questions arise during audits or closeout

24 Government Property System (cont.) Additional risks of noncompliance Cost disallowances because unable to substantiate that property was actually purchased in the quantities for which reimbursement is sought Contractor bears the risk of loss resulting from willful misconduct or lack of good faith by a contractor s managerial personnel

25 MMAS Selected areas of focus Lack of controls to timely identify unauthorized transfers/loans of materials Inadequate inventory controls to track removal of materials from inventory and use on contract Failure to ensure that loaned materials are timely repaid to lending contract Transfers/loans from cost-type contracts to commercial or fixed price work

26 EVMS Selected areas of focus Lack of necessary integration between systems (accounting, purchasing, inventory, program management) to support EVMS Failure to update budgets and schedules as a result of changes, modifications, government delays Inadequate accounting for level of effort functions in program budgets Adjustments of incurred costs lack reconciliation to accounting records

27 Potential Issues Resist temptation to accept imposition of what the government says is a better practice when the current practice is better for your business Monitor government procurement decisions Timely file appropriate bid protests If excluded from a procurement, must file preproposal If not awarded, obtain a debriefing and ascertain the impact of unapproved system(s)

28 DCMA Implementation, Business System Review Panels Established at the DCMA HQ Level Chaired by the Director, Contract Policy Members include: HQs Legal Counsel, System Policy Owner, CBS Policy Owner Purpose: To ensure regulations/policy are followed, consistently applied, action is sufficiently documented, and proper coordination has occurred 18 Reviews as of October 2012: 8 Accounting, 3 Estimating, 2 EV, 3 Property, 2 Purchasing 12 disapproved systems 3 pending further fact finding 3 panel recommended alternative course of action

29 Jami Levy Pricing Manager, Northrop Grumman Corporation

30 data driven Estimating Key to Estimating System Health Jami Levy Northrop Grumman Aerospace Systems Pricing and Estimating Compliance Systems, Training and Tools 30

31 data driven Estimating Why Basis of Estimate (BOE) Focus? BOE - Description data driven Estimating - Definition Fundamentals of data driven Estimating data driven Estimate Examples We are the Customer Non-Proprietary BOE Training Material Available to All 31

32 Background/Objectives Background: There is a significant industry effort to increase the quality of contractor estimating processes. The industry has applied the term data driven estimating to this new method of developing and documenting our estimates/boes. data driven Estimating methods have certain characteristics that are starting to emerge as standards within our industry. Objectives: What is driving our industry toward the use of data driven estimating methods and why is this a business imperative Be able to identify the characteristics of good data driven estimating processes and how they are represented in our BOEs Provide examples of data driven versus non-data driven BOEs Raise the bar by providing access to non-proprietary BOE training 32

33 Mapping of DFARS Cost Estimating System Requirements to Major Estimating System Components 13. Provide for internal review of, and accountability for, the acceptability of the estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences. Roles 1. Establish clear responsibility for preparation, review, and approval of cost estimates and budgets. 2. Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets. 12. Require management review, including verification of the estimating and budgeting policies, procedures, and practices. Internal Controls Responsibilities 3. Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the established procedures. Estimate Approval 5. Provide for adequate supervision throughout the estimating and budgeting process. 6. Provide for consistent application of estimating and budgeting techniques. 7. Provide for detection and timely correction of errors. 8. Protect against cost duplication and omissions. 10. Require use of appropriate analytical methods. 14. Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process. 16. Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as basis to reach a fair and reasonable price. 17. Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement. Estimate Review Estimate Preparation Estimate Data Sources Training 4. Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets. 9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate. 11. Integrate information available from other management systems. 15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable. Well Supported Data Driven Estimating in Key to Estimating System Health 33

34 DFARS Cost Estimating System Requirements 4. Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets. 9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate. 11. Integrate information available from other management systems. 6. Provide for consistent application of estimating and budgeting techniques. 10. Require use of appropriate analytical methods. 15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable. Provide for adequate supervision throughout the estimating and budgeting process. Estimate Data Sources Estimate Preparation Estimate Review 34

35 Purpose of a BOE Documents and substantiates effort required to perform tasks Documents basis and assumptions Details understanding of requirements Conveys credibility by clearly defining tasks and assumptions Demonstrates that estimates are credible, defendable and realistic (historical data) Provides evidence that estimate is consistent with other proposal volumes Time-phased resources consistent with IMP/IMS Allocates resources such as skill mix appropriate for task Aligns with Work Breakdown Structure (WBS), SOW, and other product structures Meet customer funding, timing, and/or scheduling constraints 35

36 BOEs The Components A BOE consists of the following main elements: 1. Work Element ID WBS, SOW, Location, CLIN, NRE/Rec 2. Description of the products or services delivered and the tasks to be performed 3. Estimating method, rationale, assumptions, and computations used to develop the estimate Method of quoting 4. Relevant historical data and comparability of the historical data to the current program Explanation of any adjustments made to historical cost or programmatic data (i.e., use of modifiers) 5. Estimated labor and ODC resources to perform the defined tasks 6. Expenditure profile (time-phasing) of resources 7. Other elements as required by the cost proposal instructions 36

37 BOE Estimating Methods Estimates are developed using one or more of these typical estimating methods Judgment based on subject matter expert (SME) experience with similar efforts. Level-of-Effort/Support - uses resource-based estimates that are often based on a predetermined level of support for a given period of time Analogy/Comparison Method estimate based on historical frame of reference from a similar effort with adjustments for programmatic or technical differences Standard Time Estimating - Labor Standards and Realization/ Performance Factors - time necessary to complete a defined element of work following a prescribed (recognized) technique or method. Unit Method generally a simple ratio (not always statistically validated) Follow-on Based On Actuals extrapolation of actuals (CEAC, production) Parametric Cost Models use of calibrated, internally developed, or commercially available cost models Cost Estimating Relationships (CERs) - equations developed based on historical, statistically correlated relationships (See full training package for more details) 37

38 Customer Expectations Customers/DCAA/DCMA expect contractors to begin adopting improved estimating methods to support TINA-compliant proposals and other estimating products Least Preferred Judgment Most Preferred Fact Based Estimating Methods Move To Fact Based ASAP 38

39 BOE Methodologies Credibility Scale Customer Credibility High Low Labor Follow-on Projections based on Actuals Analogy/Comparison CER s Parametric Estimates / Cost Models Standards Unit Methods Level Of Effort Judgment Historical resource based estimates are easier to support than judgmental type estimates. 39

40 Customer Consequences Poor Estimating Failure to be responsive to customer estimating processes could be grounds for Non-compliance with RFP or government regulations Rejection of proposal and possibly defective pricing Holdbacks Penalties for bad BOEs include: Questioned costs DCAA/Government Flash Report Negotiation losses Negative factor in competitive evaluations Defective pricing and fraud prosecution Suspension of bidding and debarment 40

41 data driven BOEs Basic Definition Fact Based Estimating: Estimating processes/boes that provide the supporting information and rationale necessary for accurate, credible and traceable estimates Information includes: Use of defined estimating methods Historical data from past programs Use of objective scope measures and complexity factors where appropriate Rationale addresses: Complete description of the estimating approach Explanation of the applicability of the historical information Estimating process must be documented, consistent and repeatable 41

42 Fact Based BOEs The Fundamentals 1. Estimate developed using a defined and documented estimating processes 2. Methods rely on scope measures (attributes) and complexity factors (parametrics) when applicable 3. Complete estimate development process is explained and documented in the BOE Professional judgments, modifiers/adjustment factors, and historical context is adequately explained Process is consistent and repeatable 4. Estimate relies on relevant historical data and context that is accessible and verifiable Shows use of latest and most current relevant data Supports all resource estimates with verifiable data 5. Estimating processes are monitored for learning and improvement as required to support future estimates 42

43 Objective Work-Scope Measures (OSM) Scope Measures = estimating predictive measures technical measures relating to the size of the effort (attributes) Generally physical and technical measures Sized at different product levels - system, subsystem, box, etc. Example: Work Products: # drawings by type, sheet counts, # documents & sheet count, SLOC by category, # requirements, # reviews, # test & type, weight, quantities, etc. Traits/Complexity Factors: characteristics that can improve the accuracy of relationships developed for the OSMs Example: Scope Measures = Drawings Traits: (numeric and descriptive): # Drawings, Type of Drawing; Drawing Heritage; Item Next Higher Assembly/Element; Number of Sheets; # of Revisions Example: Scope Measures = Source Lines of Code (SLOC) Traits: Software Language; New SLOC; Modified SLOC; Reuse SLOC; Actual Total Delivered SLOC, Deleted SLOC, Auto Generated Code, etc 43

44 Why Use Scope Measures and Traits in BOEs? Scope Measures - help to scope the size of the effort to be estimated Objective quantitative measures of scope to be performed Directly correlate to (predictors) resource estimates for work products and tasks Understanding the size (scope) of the proposed effort is critical to developing an accurate estimate (BOE) The BOE author should explain the derivation, source, and application of the scope measures in the BOE Traits/Complexity Modifiers address unique characteristics of the scope measures that can affect required effort to produce Capture the effects of technical and programmatic differences between historical programs and the current proposed effort. The produceability can vary widely for any scope measure - traits can be used to explain the differences Sometimes requires judgment 44

45 Explaining Adjustment to a BOE In order to use an objective or subjective adjustment, an estimator must explain it thoroughly. Provided below are some examples of adjustments that may be required to improve the accuracy of the estimate. The list is not meant to be totally inclusive of adjustments an estimator has available in their tool box. Technical Complexity ~ Labor Estimate Improvement Curves ~ Labor Estimate Schedule adjustments ~ Labor Estimate Quote Decrement ~ Material Estimate Quantity Adjustment ~ Material Estimate Escalation ~ Material Estimate Exchange Rate ~ Material Estimate 45

46 Using Historical Data in a BOE Use of Historical Data Must Address: Source of data generally from the organization s certified historical data repositories Complete identification of the nature, source and details of relevant historical data Ensure compliance with A.C.T.T.R. by documenting the following: Actual historical hours Charge numbers included Technical data used Time frame of historical data Repository or source of historical data 46

47 data driven Estimating - Examples We are all confronted in our daily lives with estimates for products and services E.g. Home remodel, auto repair We evaluate the credibility of these estimates based upon the content of the estimate We need to apply this customer mentality when we review our own proposal cost volumes and BOEs Example: Everyone has dealt with an automobile repair estimate that we struggle to feel comfortable with. Evaluate the following BOEs for automobile repair service how do we as consumers know that we are getting a fair price for these services 47

48 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 48

49 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 49

50 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 50

51 data driven Estimating The Challenges Ahead Focus Improve estimating products across all estimating stakeholders (e.g. BOEs, EV, CAPE, ICE, Business Case Analysis) The debate as we evolve to our future At what level will we be generating estimates (e.g. subsystem, assembly) How will we document our estimating processes Historical databases, processes and strategies need to be flexible as we evolve toward the future improved estimating products Estimating, Cost Modeling and Engineering teams working together As the desired characteristics of these high quality estimating methods evolve our methods must evolve also. Immediate needs: Documented estimating process Good historical data Libraries of BOEs demonstrating data driven estimating approaches 51

52 Back Up 52

53 Estimating Method Analogy/Comparison Analogy/Comparison: This method is based on historical context or frame of reference from a similar effort. Compares items being estimated to items of similar configuration and known cost to produce estimate Includes use of historical data Addresses differences between historical data and proposed solution being estimated May use relevant estimating attributes and complexity or adjustment factors (modifiers) to reflect programmatic, technology, requirements or technical differences Clearly document in BOE why and how factors are used and validated Estimated Cost (New Design) = Historical Cost (Similar Design) + Complexity or Scaling Factor 53

54 Estimating Method - Judgment Judgment: Least preferred, least credible estimating method very subject to bias Generally not based upon historical data, scope measures or other sound estimating approaches Relies on SME estimate s of small work packages aggregated to complete hardware, software, etc tasks State author s experience from which to make an estimate Programs worked; Years in industry Decompose effort into well defined tasks and subtasks Disclose if historical methods were investigated and not used, and why they were not used Rarely are we asked to quote something that does not have some similar historical likeness! 54

55 Estimating Method - LOE Level of Effort - Use when customer specifies effort to be provide over a stated period of time on work that can be stated only in general terms Requirement for specific skills/talents that tend to be headcount driven Used when product cannot be well defined Contract is for delivery of hours not product. Payment based on effort expended not results achieved. Restricted LOE Contracts SOW that specifically calls out LOE by customer What is NOT LOE Level loading of labor not specifically directed by customer Staffing a position (i.e. Program Manager) as 155 hrs/month during period of performance on a completion type contract 55

56 David E. Trotta Estimating Manager, ViaSat, Inc.

57 Estimating Systems Overview and Audit Scope February 27, 2013 Presented by David E. Trotta 57

58

59 Agenda Estimating System Overview and Requirements What to Expect in an Audit Steps to Compliance Questions

60

61 Estimating Overview Estimating System Need to disclose system in writing Comply with the system as disclosed You must make timely disclosure to ACO of significant changes Significant deficiencies can prompt withholds in accordance with DFARS Integrate with other systems (e.g., Accounting System) Meet requirements of law and regulation

62 Cost Estimating System Requirements DFARS Estimating system means the Contractor's policies, procedures, and practices for budgeting and planning controls, and generating estimates of costs and other data included in proposals submitted to customers in the expectation of receiving contract awards. Estimating system includes the Contractor's (1) Organizational structure; (2) Established lines of authority, duties, and responsibilities; (3) Internal controls and managerial reviews; (4) Flow of work, coordination, and communication; and (5) Budgeting, planning, estimating methods, techniques, accumulation of historical costs, and other analyses used to generate cost estimates.

63 DFARS Requirement Requires all DoD contractors to have acceptable estimating systems in writing, provides guidelines concerning the characteristics of an adequate estimating system Creates the requirement of the contracting officer to determine adequacy of the cost estimating system

64 Acceptable estimating system means... system that complies with the system criteria and provides for a system that (1) Is maintained, reliable, and consistently applied; (2) Produces verifiable, supportable, documented, and timely cost estimates that are an acceptable basis for negotiation of fair and reasonable prices; (3) Is consistent with and integrated with the Contractor s related management systems; and (4) Is subject to applicable financial control systems.

65 Significant Estimating System Deficiency means Significant deficiency means a shortcoming in the system that materially affects the ability of officials of the Department of Defense to rely upon data and information produced by the system that is needed for management purposes. ( Cost Estimating System Requirements)

66 Estimating System Is different from other system because of it s unique nature Comprised of policies and procedures as well as input from other systems A fair amount of judgment can be involved on any particular estimate; what data from what systems is relevant Often times changes to the solution being proposed does not easily tie back to prior programs or organization changes complicates how past data is relevant

67 Changes in the Business Systems Rules - From Should to Shall Cost Estimating System Requirements DFARS (4) The Contractor s estimating system shall provide for the use of appropriate source data, utilize sound estimating techniques and good judgment, maintain a consistent approach, and adhere to established policies and procedures. An acceptable estimating system shall accomplish the following functions: (i) Establish clear responsibility for preparation, review, and approval of cost estimates and budgets; (ii) Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets; (iii) Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the Contractor's established procedures; (iv) Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets; (v) Provide for adequate supervision throughout the estimating and budgeting process; (vi) Provide for consistent application of estimating and budgeting techniques; (vii) Provide for detection and timely correction of errors; (viii) Protect against cost duplication and omissions; (ix) Provide for the use of historical experience, including historical vendor pricing information, where appropriate;

68 From Should to Shall (continued) Cost Estimating System Requirements DFARS (continued) (x) Require use of appropriate analytical methods; (xi) Integrate information available from other management systems; (xii) Require management review, including verification of the company's estimating and budgeting policies, procedures, and practices; (xiii) Provide for internal review of, and accountability for, the acceptability of the estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences; (xiv) Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process; (xv) Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable; (xvi) Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as a basis to reach a fair and reasonable price; and (xvii) Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement.

69 Penalties associated with Significant Estimating System Deficiencies Contracts subject to withholds are contracts subject to CAS coverage that contain DFARS and the applicable clause Subject to limitation, ACO has discretion to identify contracts from which to withhold payments Amount 5% for one disapproved system or up to 10% for two or more disapproved systems, of all billings under covered contracts Pre-existing withholds should preclude or reduce further withholds for system disapproval

70 Payments subject to withhold Interim payments under Cost-reimbursement contracts; Incentive type contracts; Time and Material contracts; Laborhour contracts; Progress payments; Performance based payments Once contracting officer is notified of deficiencies correction CO will discontinue withhold if corrections are sufficient Has 90 days to make this determination or future withhold percentage is reduced by at least 50%

71

72 Pre Audit Process (Auditor s Perspective) Auditor will notify contractor of planned audit of the Estimating System Auditor will review prior work done by auditors and any flash estimating system reports Auditor will contact DCMA about any known concerns Auditor will request a listing of proposals for the last 12 months Summarized by customer, contract type and dollar amount

73 Entrance Conference will be Conducted Contractor will be asked To provide a system orientation or briefing Outline any changes to the Estimating System since the last examination Discuss the contractor s risk assessment process Discuss the contractor s monitoring process Discuss any prior weaknesses and actions or improvements steps taken to address these weaknesses Auditor will prepare an internal risk assessment

74 Control Environment/Information & Communications Auditor will evaluate the internal controls this includes looking at ICAPS, integrity and ethical values, competence, Board/Audit committee participation, management philosophy, authority and structure, HR policies and Financial capability Access overall accounting controls, IT systems, files, any recent related changes and query contractor staff on estimating system and process

75 Contractor Monitoring Process / IA /System Description Audit will then look at the contractor s ongoing monitoring process, including internal audit, any external reviews and controls in place Including recommendation and follow up as well as review frequency Review of Estimating System Description Organization charts, written policy, procedures and directives, flowcharts, lines of authority, review and approval, written instructions and integration with other Management Systems

76 Training Determine if regular training is required for those responsible for preparing estimates Includes training of the use of statistical aids and advanced estimating techniques Training covers estimating system policies and procedures Auditor will ask to examine records of training

77 Cost Estimating Development Are estimates developed using appropriate methods, compliance with CAS 401 and 402 Use of historical data, escalation methods, use of Forward Pricing factors, formula pricing methods, catalog pricing, format of proposal (Table 15-2),Certification of Cost or Pricing Data and Subcontractor policies

78 Cost Estimating Development (continued) Selective evaluation of recent pricing proposals for adherence to procedures, Determine if the following characteristics are in place Adequate supervision, management reviews both interim and final reviews that cover soundness of estimates and adherence to policy

79 Audit Conclusion Auditor does an assessment of the contractor Makes determination of adequacy Prepares a draft audit report Conducts an exit conference Finalize the audit report incorporating the contactor s response

80

81 Steps to Compliance Start now, don t wait until you are being notified of an Estimating System Audit Evaluate your Estimating System to ensure required changes are included Consider 3 rd party / consultant Ensure you have your management s support Review Guidance Provided DCAA Audit Program is posted on DCAA website under Standard Audit Programs Additional guidance is provided in DCAA s Contract Audit Manual (CAM)

82 Steps to Compliance (continued) Document your Estimating Policies and Procedures Update or create an Estimating System Manual Ensure the controls are in place but make sure you have the flexibility to respond to all proposals in a compliant manner When a deviation is required to comply with solicitation requirements document them in your proposal and review process

83 Steps to Compliance (continued) Training this will greatly reduce the risk of problems and is an audit hot button Build your controls and system around your business, there is no one size fits all Services are very different from products Larger businesses may have different policies and procedures for different business segments Auditor may not be familiar with your business, don t assume DCAA staff currently reviewing your business will conduct the audit Spend the time necessary to brief and explain your unique issues and approaches

84 Steps to Compliance (continued) During the Audit engage with your auditors Be responsive to issue Be proactive, if there is problem start mitigating it now Demonstrate mitigation steps before the final audit report is issued, request briefing and draft report before report is released If issues can t be resolved with auditor, contractor should provide the auditor with contractors written position to be included in the report

85 Conclusion Be proactive and get your management s support The proposal you are doing now will likely be looked at during the Estimating System Audit so start now Establish a Training and Communication Plan Any single failure can result in a inadequacy determination so don t create a hurdle you can t clear

86

87 For more information about our organizations visit us on our websites