PROFICIENCY TESTING:

Transcription

1 BELAC Rev PROFICIENCY TESTING: GUIDELINES ON THE LEVEL OF PARTICIPATION AND EVALUATION OF PERFORMANCES IN PROFICIENCY TESTING ACTIVITIES IN THE CONTEXT OF ACCREDITATION ASSESSEMENTS The only valid versions of the documents of the BELAC management system are those available from the internet website. English translation for information only French and Dutch versions remain the authoritative documents Applicable from: BELAC Rev / 11

2 BELAC Rev / 11

3 DOCUMENT HISTORY Revised and date of approval Reason for revision Impact of revision 0 CC CC written procedure with end date First document Replacement of reference to Guide ISO 43 by the standard NBN EN ISO/IEC 17043:2010 Extension of application area to all types of laboratories and inspection bodies performing tests. Revision of guidelines concerning frequency of participation. Addition of the concept «measurement audit» as part of the accreditation assessment. Not applicable Full document Section 1 Section Section BELAC Rev / 11

4 TABLE OF CONTENTS 1. OBJECTIVES AND REFERENCES TO NORMATIVE DOCUMENTS RECIPIENTS DEFINITIONS Proficiency Testing Interlaboratory comparisons Measurement Audit /Bilateral proficiency testing GENERAL POLICY AND PRACTICAL ASPECTS REGARDING PARTICIPATION IN PROFICIENCY TESTING General Terms for participation in proficiency testing Proficiency testing selection: general provisions Proficiency testing selection: additional provisions Responsibilities of the laboratory concerning the participation in proficiency testing Documentation Frequency of participations Statistical evaluation of the results Corrective actions BELAC responsibilities in the evaluation by BELAC of the results of participation in proficiency testing Evaluation during accreditation-assessments Communication of results of proficiency testing to BELAC Taking into account the proficiency testing results in a certain technical sector Sanctions within the context of result evaluation of proficiency testing BELAC Rev / 11

5 PROFICIENCY TESTING: GUIDELINES ON THE LEVEL OF PARTICIPATION AND EVALUATION OF PERFORMANCES IN PROFICIENCY TESTING ACTIVITIES IN THE CONTEXT OF ACCREDITATION ASSESSMENTS 1. OBJECTIVES AND REFERENCES TO NORMATIVE DOCUMENTS The standards NBN EN ISO/IEC and NBN EN ISO require that laboratories seek confirmation for confidence in their results and organize quality control procedures for monitoring the validity of performed tests. This monitoring includes the participation in interlaboratory comparisons or proficiency testing programs. In this document, the term laboratory covers testing laboratories, calibration laboratories and medical laboratories. The purpose of this document is to identify the BELAC requirements concerning the participation of laboratories in proficiency testing. The document further determines the criteria for evaluation of performance in the context of an accreditation process. Principles and requirements described within this document also apply to inspection bodies performing themselves critical tests and calibrations activities (cfr. BELAC clause 1.4). This document refers to and is in compliance with: - standard NBN EN ISO/IEC 17011; - standard NBN EN ISO/IEC 17025; - standard NBN EN ISO/IEC 17020; - standard NBN EN ISO/IEC 17043; - relevant EA en ILAC guidelines (cfr and 2. RECIPIENTS With follow up of modifications: Members of the Accreditation Board Members of the Coordination Commission The accreditation secretariat Assessors / experts from the sector laboratories and inspection Accredited laboratories and inspection bodies Without follow up of modifications: Laboratories and inspection bodies having submitted a request to accreditation Any external request BELAC Rev / 11

6 3. DEFINITIONS Note: Direct quotations from standards and other normative references are written in italic Proficiency Testing Evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons. NOTE 1 For the purposes of this International Standard, the term proficiency testing is taken in its widest sense and includes, but is not limited to: a) quantitative scheme where the objective is to quantify one or more measurands of the proficiency test item; b) qualitative scheme where the objective is to identify or describe one or more characteristics of the proficiency test item; c) sequential scheme where one or more proficiency test items are distributed sequentially for testing or measurement and returned to the proficiency testing provider at intervals; d) simultaneous scheme where proficiency test items are distributed for concurrent testing or measurement within a defined time period; e) single occasion exercise where proficiency test items are provided on a single occasion; f) continuous scheme where proficiency test items are provided at regular intervals; g) sampling where samples are taken for subsequent analysis; and h) data transformation and interpretation where sets of data or other information are furnished and the information is processed to provide an interpretation (or other outcome). NOTE 2 Some providers of proficiency testing in the medical area use the term External Quality Assessment (EQA) for their proficiency testing schemes, or for their broader programmes, or both. (NBN EN ISO/IEC clause 3.7) 3.2. Interlaboratory comparisons Organization, performance and evaluation of measurements or tests on the same or similar items by two or more laboratories in accordance with predetermined conditions (NBN EN ISO/IEC clause 3.4) 3.3. Measurement Audit /Bilateral proficiency testing Laboratories receive a test item with accurately determined characteristics, which is to be tested in the context of an accreditation process. The test item is given either by the assessor or provided by a third party. BELAC Rev / 11

7 4. GENERAL POLICY AND PRACTICAL ASPECTS REGARDING PARTICIPATION IN PROFICIENCY TESTING General - Proficiency testing is a means by which laboratories can prove their technical competence and the level of operation of their management system. Proficiency testing is in most cases a part of the validation procedure. Accreditation bodies use the evaluation of the performance level of laboratories during assessments as an essential element that in the context of the mutual recognitions, can be used for the acceptance of results on national and international level. Additionally, it can provide an indication for effectiveness and efficiency of assessors and in general, confirm the value of the granted accreditation. - Analyses of the performance in proficiency testing shall be used as a tool to improve the level of operation of the laboratory management system. It is not intended to be used as a tool to confirm or not the accreditation status. However, repeated or major failures during the performance of proficiency testing should result in sanctions and this to ensure the credibility of the accreditation concept. - Improvement and enforcement of laboratory quality can also be accomplished by regular participation in interlaboratory comparisons used for different purposes (e.g. method validation or characterization of reference materials). These interlaboratory comparisons should be adapted to be able to service the purpose of proficiency testing. The results obtained by interlaboratory comparisons should be interpreted with great caution. - In specific sectors where no proficiency testing is available, the laboratory should use alternative resources to prove its technical competence. These include o The regular use of (certified) reference materials; o Comparison of results obtained by independent techniques; o Comparison of the results of 2 or more laboratories on their own initiative. The general requirements concerning the policy on participation in proficiency testing also apply to laboratories using alternative means Terms for participation in proficiency testing Proficiency testing selection: general provisions - A high level of technical competence is required from the organizer of proficiency testing. This competence is necessary for the planning, preparation, performance, interpretation and documentation of the results of these tests. BELAC Rev / 11

8 - The proficiency testing organizer can be selected by the laboratories. This organizer is considered to be a service supplier (ISO clause 4.6). The selection should be based on the evaluation of the suitability of proposed services regarding the needs. Previous experience of the laboratory with the organizer of proficiency testing can serve as the basis for the selection. In general, the evaluation should include the following elements: Competence of the proficiency testing organizer: this can be proven by accreditation according to the standard NBN EN ISO/IEC If the laboratory relies on an organizer that is not accredited, the laboratory is required to perform and document an evaluation. Material/matrix type: should, as far as possible, correspond with the materials/matrixes examined in the laboratory; Parameters to be measured: should include as many parameters as possible as actually determined by the laboratory; Range of measurement: should be as close as possible to the actual situation in the laboratory; Frequency: should meet the laboratories needs to ensure that participation in proficiency testing is an efficient means to evaluate quality management. Other means can also be used, by the laboratory, to ensure the quality of their results; Statistical result analysis: the used protocol should be clearly defined. - In many cases it will be difficult if not impossible to select a proficiency testing that covers all of the above mentioned criteria, and each test/calibration or group of test/calibrations covered by the accreditation. A compromise will be necessary and it is the laboratory s task to justify its choices while ensuring adequate coverage of activities covered by accreditation (price / efficiency ratio). - BELAC recommends laboratories to consult the EPTIS database (European PT Information System to, whenever necessary, identify the appropriate proficiency testing for their activities Proficiency testing selection: additional provisions - BELAC can require laboratories to participate in specific proficiency testing, on their own costs, in case elements indicate that there is need to do so; ; in this case BELAC will preferentially select an organizer whose management conforms to the requirements of the standard NBN EN ISO/IEC Arguments that might lead to such initiatives are: the absence of proficiency testing in a certain sector or special problems discovered during assessment of a laboratory in a specific sector. - In some sectors, participation in specific proficiency testing is mandatory in the context of the contract with the customer, and in particular in the case accreditation is a prerequisite for agreement by a regulatory authority. - In the context of the international mutual recognitions by EA and ILAC, international proficiency testing programs are organized. The purpose is to ensure mutual trust on an international level. The laboratories invited by BELAC to participate in these tests are BELAC Rev / 11

9 obliged to participate, but they cannot demand any compensation for the provided services Responsibilities of the laboratory concerning the participation in proficiency testing Documentation - The policy of the laboratory concerning the participation in proficiency testing shall be documented in the quality manual or in other documents pertaining to the management system of the laboratory. Will be especially considered: planning, performance, evaluation of the results, corrective actions and archiving. For selection of proficiency testing organizers: see The laboratories shall moreover maintain an updated synthesis of the results of proficiency testing in which they participated, or in case of lack of proficiency testing, of other initiatives taken to ensure the validity of the test results. This shall be the case for each test/calibration or group of tests/calibrations covered by accreditation. This document shall be available to BELAC in preparation of each assessment Frequency of participations - An accredited laboratory shall take part in proficiency testing on a regular testing and define its frequency of participation based on a risk assessment. This should take into account the following: Number of performed tests, calibrations or measurements; The possibility to use the results of a specific proficiency testing for assessment of performance of a group of tests based on similar measuring techniques and/or concerning a group of products (equal level of technical competence); Number of involved operators and frequency of turnover in technical staff; The education level and general experience of the personnel; The availability or not of source of traceability (national standards, certified reference material); The complexity of the used measurement technique; The level of criticality of the result, taking into account its final use; The use of alternative techniques for the evaluation of the results in addition to the participation in proficiency testing. A re-evaluation of the choice of participation frequency shall be performed in case of important changes in laboratory organization. - BELAC can, for a specific sector, impose a higher participation frequency or specific implementation provisions. - An accreditation for one or several tests/calibrations can only be provided if the performance level of the laboratory can be proven by its results of proficiency testing or, in the absence of these proficiency testing, by other sources as mentioned above. - If proficiency testing is available, the laboratory should justify why it did not participate. BELAC Rev / 11

10 Statistical evaluation of the results - The statistical result evaluation is a key element to estimate the laboratories performance. - Investigation concerning consistency between performance (in terms of trueness and precision) and the uncertainty reported by the laboratory is of great importance to decide if corrective actions have to be taken and to determine the characteristics of these actions Corrective actions - In the case of unsatisfactory results the laboratory should investigate the causes and possible consequences and if necessary take appropriate actions. These actions should involve feedback to analysis results or calibration certificates that could be influenced and which have already been send to customers. The actions should also comprise some technical corrective and preventive measures that guaranty that the quality of the future test results will satisfy the costumers demands and expectations. - As long as the laboratory cannot prove its test/calibration competence, either via proficiency testing or any other source available to the laboratory, it is not allowed to issue results covered by accreditation. If this situation continues, the laboratory is obliged to inform BELAC of this situation BELAC responsibilities in the evaluation by BELAC of the results of participation in proficiency testing Evaluation during accreditation assessments - During each assessment the BELAC assessors shall check the performance in proficiency testing. They have to evaluate efficiency of corrective actions and preventive measures. The assessor specifically focus on following items: - the choice of proficiency testing and participation frequency; - the quality of the proficiency testing organizer; - the examination of raw data; - the performance conditions should match with those used for the routine-analyses; - the analysis of the result by the laboratory; - the consistency with the uncertainty of measurements normally reported; - the extent of the observed deviation in relation to the customers expectations; - the corrective actions and measures undertaken concerning analysis reports that were already sent to customers when proficiency testing outcome resulted in modifications of the test procedure; - the complexity of the test. - The findings shall be mentioned in the assessment report. - Where appropriate and in particular in calibration activities, the assessor can ask for a bilateral test/calibration (measurement audit) being performed as part of the assessment activities.. BELAC Rev / 11

11 Communication of results of proficiency testing to BELAC. The Accreditation secretariat investigates the results of the proficiency testing in which BELAC accredited laboratories participated in case they are presented to BELAC for example by competent authorities or international organizations. In case of unsatisfactory performance, BELAC contacts the concerned laboratory and the competent technical assessors. An additional assessment will be organized if necessary Taking into account the proficiency testing results in a certain technical sector Repeated and collective unsatisfactory performance of BELAC accredited laboratories in a certain sector should result in a thorough discussion in the Accreditation Bureau with respect to the need for general corrective measures; this can include additional training of BELAC assessors active in this specified sector Sanctions within the context of result evaluation of proficiency testing An unsatisfactory performance without effective corrective actions or repeated (consecutive or not) bad performances in proficiency testing, will result in a sanction. This will normally be a temporary suspension of the concerned test/calibration from the accreditation scope. When, for a particular laboratory, problems arise in multiple sectors the complete accreditations status can be questioned. An inappropriate management of the participation in proficiency testing is indeed a proof of an ineffective quality management system, in particular as far as the control of test results is concerned, which is the main aim of accreditation. Before deciding to reduce the scope or to withdraw the accreditation status, the Accreditation Bureau, as decision-making body, will contact the involved assessor and the concerned laboratory will be given the opportunity to present its arguments.. Focus will be on aspects described in BELAC Rev / 11