How DCMA Helps To Ensure Good Measurements

Transcription

1 How DCMA Helps To Ensure Good Measurements Speaker/Author: Robert Field Defense Contract Management Agency 605 Stewart Avenue Garden City, New York Phone: (516) ; Fax: (516) Abstract The Defense Contract Management Agency (DCMA) is responsible for helping to ensure that the products and services provided by industry on U.S. Department of Defense and NASA contracts comply with business and technical requirements. Where contract requirements include standards such as ANSI/NCSL Z540.1, ISO/IEC and ISO 9001, DCMA performs technical surveillance of contractor and subcontractor operations. Compliance to contract requirements is important to our customers. This paper discusses the DCMA business, technical and customer focused policies, and standard practices that contribute to measurement integrity. It describes how DCMA uses risk-based surveillance techniques including process audit, system evaluation and analysis of second and third party data to perform its mission. Finally, issues of compliance to standards are presented. 1. Introduction Measurement integrity is essential to the design, development and production of products that meet customer requirements. Almost every product acquired by the U.S. Department of Defense (DoD) in direct support of its mission has characteristics that need to be measured to ensure fitness for use. The Defense Contract Management Agency (DCMA) is a contract management organization within DoD that serves as an honest broker to help DoD buying offices, other government agency customers, and contractors successfully deliver products that meet contract requirements and customer needs. DCMA has 65 major office locations and more than 10,500 civilian, active duty and reserve personnel working in the continental U.S. and around the world. It manages a contract business base of 325,000 contracts valued at over $852 billion with 25,000 domestic and foreign contractors. In many cases, DCMA has technical and other personnel permanently located on-site at contractor facilities to perform its mission that includes assuring products are accurately measured. 2. Contract Management 2.1 Contract management of DoD contracts is the core business of DCMA. The core business is established by the authority of codified uniform policies, known as the Federal Acquisition Regulation (FAR), for acquisition of supplies and services by U.S. Government Executive Branch agencies. In accordance with the FAR, and when assigned contract responsibility by its

2 customers, DCMA has broad business and technical responsibilities to ensure contract compliance. Technical responsibilities include Ensuring contractor compliance with contractual quality assurance requirements, Performing surveillance to assess compliance with contractual terms for schedule, cost, and technical performance in the areas of design, development, and production, and Evaluating for adequacy and performing surveillance of contractor engineering efforts and management systems that relate to design, development, production, engineering changes, subcontractors, tests, management of engineering resources, reliability and maintainability, data control systems, configuration management, and independent research and development. Regarding quality assurance specifically, DCMA is required to ensure that contractor quality assurance requirements are met; develop and apply efficient procedures for performing government contract quality assurance actions; perform all actions necessary to verify whether products conform to contract quality requirements; and report any defects observed in design or technical requirements [1&2]. 2.2 Included in these responsibilities is the requirement for assuring accurate measurements needed to produce products acceptable to customers, and where contracts include standards such as ANSI/NCSL Z540-1, ISO/IEC and ISO 9001 assuring that management systems are established and used to control measurement processes. Processes evaluated at calibration laboratories include the quality system, competency of personnel, measurement traceability, calibration methods, subcontracting for calibration services and records. Processes associated with the control of measuring and test equipment used to verify the acceptability of deliverable products are also evaluated. These include the calibration system, measurement standards, calibration intervals, calibration procedures, records, and storage and handling of equipment. 3. Policies and Practices 3.1 Applying DoD guidelines, DCMA uses an integrated risk management approach to evaluate contractor performance risks and to determine and execute actions to mitigate those risks for customers [3&4]. This approach includes risk planning, risk assessment, risk handling, risk monitoring and risk documentation. The approach focuses on important customer requirements such as contractor management systems and key business and technical processes that are essential to successful performance on a contract. 3.2 Risk planning involves the review of new and modified contracts, as well as other customer guidance, to gain an understanding of customer needs and expectations. Based on an understanding of customer needs, contractor management systems and key processes are identified for risk management. Contractors are designated as strategic, critical or routine based on the scope and importance of contract requirements as they relate to national security and tactical mission performance. For instance, a contractor may be designated as strategic when it is supporting a major military aircraft or ship program, or designated as critical if contracts involve

3 hazardous materials or flight critical products, or designated routine when products are noncomplex and of a non-critical nature. 3.3 Risk assessment is performed on all identified systems and key processes. Each system and process is assigned a risk rating of high, moderate or low. Assessments are based on both the consequence of product failure and the likelihood that a defect resulting in a failure will be produced. The consequence of failure is associated with impact to the customer, and the likelihood of failure to contactor past performance and process maturity. Figure 1 is used as the primary assessment tool. After all identified systems and key processes are assessed and rated, the contractor is assigned an overall risk rating for performance, cost and schedule. Figure 1. Risk Assessment Tool. 3.4 Contractors perform risk handling. The role of DCMA in risk handling is one of influence rather than control. Using our risk assessments, we work in a cooperative way to convince contractors to take actions to mitigate/manage risks on their systems and processes by either making changes to avoid the risks or controlling the losses associated with the risks. This is especially important where risks are deemed to be high or moderate. 3.5 Just as all identified systems and key processes are assessed for risk, they are all monitored for risk. Risk monitoring involves a continuing set of activities performed by DCMA functional specialists. Activities may include process audits, system evaluations, and review and analysis of

4 data. Where a contractor system or process is considered to be low risk, DCMA evaluation relies primarily on existing second and third party data. Where risk is considered moderate or high, DCMA performs its own hands-on evaluation to supplement and/or validate available data. Examples of these activities as they relate to measurement processes are discussed below. 3.6 DCMA prepares and retains records of its risk management efforts in a web based risk assessment and management program known as RAMP, which includes risk planning, assessment, handling and monitoring information. RAMP information concerning a contractor is shared with that contractor and any disagreements are resolved. Supporting data such as system and process evaluations are documented and maintained in separate files. 4. Evaluation of Systems and Processes 4.1 Management Systems - DCMA performs system evaluations when contracts contain higherlevel quality system requirements such as ISO 9001, Quality Management Systems Requirements [1&4]. Included as part of an evaluation is an audit of the requirement to control monitoring and measuring devices. An ISO 9001, clause 7.6 audit checklist is used to determine if appropriate processes are in place and are being used to ensure measurement integrity, and that measurement devices are capable of ensuring that product requirements are met. In addition, if a contract requirement such as ANSI/NCSL Z540.1 is invoked, the system evaluation includes more detailed evaluation of that requirement. If available, second party audit data and third party accreditation data are also used in the evaluation process. 4.2 Key Processes - Risk monitoring of contractor s product verification processes is a primary focus of the DCMA government quality assurance surveillance program. Usually, at an individual contractor facility one or more processes are monitored depending on the criticality and complexity of the products being produced. In the case of non-critical non-complex products the final inspection process is normally the sole process monitored. For critical and complex products multiple verification processes are checked. They may include receiving inspection, various inprocess inspection and testing points, first article inspection, non-destructive testing, qualification testing, and final inspection and test. In each case, and for each process monitored, the use of appropriate and properly controlled measuring devices is verified. Some manufacturing processes are also monitored to ensure that devices used to measure temperature, pressure, weight, et cetera, are accurate to maintain process control. Examples of these manufacturing processes are soldering, welding, plating and painting. 5. Technical Competency - In DCMA, surveillance of measurement control processes is primarily the responsibility of the quality assurance specialist. At a minimum, quality assurance specialists are certified to both Defense Acquisition Workforce Improvement Act (DAWIA) Level II in the Manufacturing, Production, and Quality Assurance career field and one or more product commodities, such as aircraft, electronics, mechanical, chemical or ammunition. Commodity certification requirements are in accordance with the DCMA Quality Assurance Technical Development Program. In addition, specialists who evaluate contractor quality systems must meet ISO , Guidelines for Auditing Quality Systems - Part II: Qualification Criteria for Quality Systems Auditors. Some specialists have industry experience and other specialized training in measurement control systems and processes.

5 6. Examples of Issues of Compliance - Where issues of compliance are identified, they are documented and corrective action is requested from contractors. The types of issues most commonly found include: Calibration intervals exceeded Lack of understanding of measurement control requirements (especially at small contractors) Record keeping errors and omissions Use of uncontrolled devices in engineering laboratories Use of personal (uncontrolled) mechanical measurement devices Subcontracted services non-compliant Uncertainty analysis not performed Out-of-service devices not properly controlled 7. Summary - DCMA performs surveillance using a risk-based management approach. Both management systems and specific measurement and manufacturing processes are evaluated to ensure the integrity of measurements made on products delivered to DoD and other customers. Compliance issues are identified and followed until they are corrected. 8. References 1. FAR Part 46, Quality Assurance 2. FAR SubPart 42.3, Contract Administration Office Functions 3. Risk Management Guide For DoD Acquisition, June DCMA Directive 1- Contract Management