Inter-Laboratory Comparisons (ILC) 16 June 05 ILC-1 Slide 1 Overview Definitions Types of ILCs Why to participate How ILC are organised Assignment of values & evaluation ILC organisers Corrective Action after participation 16 June 05 ILC-1 Slide 2 1
Definitions Inter-Laboratory Comparison -ILC Organisation, performance and evaluation of tests on the same or similar test items by two or more laboratories in accordance with predetermined conditions (Laboratory) Proficiency Testing - PT Determination of laboratory testing performance by means of inter-laboratory test comparisons [ISO/IEC Guide 43:1997] 16 June 05 ILC-1 Slide 3 Goals of an ILC ILC to demonstrate competence and establish degree of equivalence between results of the participating laboratories ILC used to assign certified values to RMs ILC to standardise/improve a method (determine repeatability, reproducibility,...) ILC as a training exercise to improve skills 16 June 05 ILC-1 Slide 4 2
ILC for Validation of measurement procedure Objective: determine procedure repeatability s r and reproducibility (between lab) s R Evaluation using ANOVA (Analysis of Variance) Check for Outliers (before averaging/concluding) Cochran test for variance outliers, Grubbs test for average outliers [ISO5725-2] 16 June 05 ILC-1 Slide 5 ILC for certification of (C)RM Objective: determine the certified value and it s uncertainty for Reference Materials Uncertainty estimation, u char [ISO-GUM] Technical Discussion Meeting [ISO Guide 35] See Module CRM 16 June 05 ILC-1 Slide 6 3
ILC for performance evaluation of laboratories (PT) Objective: determine the performance of laboratories Evaluation Parameters: Assigned value Performance indicators Evaluation of single performance Evaluation of combined performance with composite scores [ISO Guide 43 & ISO/DIS 13528] 16 June 05 ILC-1 Slide 7 Organising ILC (1) I - Design Establish objectives/ purpose Selection of organiser Selection of sample/matrix & measurand/analyte Selection material provider Preparation of Test material Test of Homogeneity and stability Determination of assigned/reference value Selection of participants 16 June 05 ILC-1 Slide 8 4
Organising PT/ILC (2) II - Execution Distribution of test samples to participants Analysis by participants (measurand quantification) Reporting by participants to Organiser III - Evaluation Evaluation of results Reporting by Organiser to participants (feedback) Draw Conclusions corrective action 16 June 05 ILC-1 Slide 9 Performance Evaluation Criteria are set by the... Organiser of the PT/ ILC Accreditation body Regulator Participating laboratories themselves 16 June 05 ILC-1 Slide 10 5
How to obtain assigned values? Nominal value By preparation (gravimetric/volumetric) Value derived from all participants results a sub-set (after outlier rejection) Reference Value independent from participant results, with demonstrated metrological quality traceability and small uncertainty link to international measurement infrastructure 16 June 05 ILC-1 Slide 11 IMEP-13 (directive 94/62/EC) Trace Elements in polyethylene 0.55 0. Pb 10 c mmol kg -1 0.45 0. 0.35 0. 0.25 0. 0.15 Experienced laboratory (self-declaration) Less experienced laboratory (self-declaration) Non experienced laboratory (self-declaration) No statement by laboratory 0-10 - - - - -60-70 0.10-80 0.05-90 0.00-100 Results from all participants. 16 June 05 ILC-1 Slide 12 6
IMEP-9 : Trace Elements in Water 122 117 Cd values above % 112 107 102 97 92 10 c nmol L -1 87 82 77 72 0-10 ± 10% 67-62 57-52 47 42 6 values below -% Results from all laboratories. 3 'less than' values - - externally set deviation unit : set by legislation 98/83/EC 16 June 05 ILC-1 Slide 13 Performance Indicators Percent Difference x lab - X X ref ref 100 Z-scores; zeta score; z = zeta = x lab x - X s lab u - X + ref ref 2 2 lab u ref [ISO/IEC Guide 43-1:1997 A.2.1.4 & ISO/DIS 13528 02] 16 June 05 ILC-1 Slide 14 7
Evaluation of Performance Common examples of application scores are: Performance Satisfactory Questionable Unsatisfactory z score z 2 > 2 but 3 > 3 zeta score 2 > 2 but 3 > 3 16 June 05 ILC-1 Slide 15 Corrective Action after participation : 1) blunder (measurement system out of control, calculation error) 2) Measurement model is not correct : the mathematical description of reality is not complete enough, (e.g. bias not taken into account: digestion? extraction?) 3) Underestimated uncertainty of an influencing input quantity 4) Combination of 2) and 3) Unsatisfactory performance? Spot the mistake & implement Corrective Action 16 June 05 ILC-1 Slide 16 8
c c Who organises ILC/PT? CCQM (www.bipm.fr) IMEP by IRMM (www.imep.ws) external reference value, linked to international measurement capability support to EA (European Cooperation Accreditation) on issues related to EU directives, crossing borders of sectors & geographic regions FAPAS (www.fapas.com) AFSSA (www.afssa.fr) EA (www.european-accreditation.org) Community Reference Laboratories (CRLs), for National Reference Laboratories (NRLs) Other check www.eptis.bam.de (European Information System on PT Shemes) 16 June 05 ILC-1 Slide 17 water batch EUROMET project 528 (Cd) 60.39 55.39 Y. AREGBE 04 Cd in water EUROMET 528 RMOs IMEP-15 WMO/GAW nominated laboratories IMEP-12 EA nominated labs Support to new member states European and International field laboratories 16 June 05 ILC-1 Results from all participants Slide 18 amount content (10-12 mol/g) 10-9 mol L -1 10-9 mol L -1.39 45.39.39 35.39.39 25.39.39 60 55 45 35 25 60 55 45 35 25 Cd VUV Results from all participants including "Trueness % of parametric value" from water directive 98/83/EC IMEP- 12: Trace elements in Water Certified value :.78 10-9 ± 0.82 10-9 mol L -1 [U =k u c (k =2)] Cd IMEP- 15 : Trace elements in Water Certified value :.78 10-9 ± 0.82 10-9 mol L -1 [U =k u c (k =2)] 2 values below -% ITM LGC IRMM BAM 17 values below -% 36 values above % 15 'less than' values including "Trueness % of parametric value" from water directive 98/83/EC NMi IMEP-12 reference value 4 values above % WRI 1 'less than' value Vituki 10 0-10 - - - - 10 0-10 - - - - Deviation from the certified value in % Deviation from the certified value in % 9