Interlaboratory studies

Interlaboratory studies Department of Food Chemistry and Analysis, ICT Prague Vladimír Kocourek Prague, 2012

Interlaboratory studies Various titles: Interlaboratory proficiency test or studies Interlaboratory comparisons Round test, round robin tests Test of qualification Certification study Interlaboratory validation

Interlaboratory comparison 1 2 1 2 6 P 6 P 3 3 5 4 5 4 Non-destructive tests Sample circulates between laboratories Destructive tests Parallel distribution of samples

Interlaboratory studies Importance of interlaboratory comparisons: Successful participation in an interlaboratory comparison is one of the necessary requirements of a laboratory accreditation or authorisation!!! Interlaboratory comparison: Interlaboratory comparison often consists in inter-comparison of measurement results of a laboratory and a reference (target) value or reference laboratory which is the highest authority of particular country for the measurement concerned. The goal of such interlaboratory comparison is to verify the competence of accredited or non-accredited laboratories, including verification of the reported measurement uncertainties, whenever possible.

Interlaboratory studies Before you get started you should ask and answer the following questions: What am I testing? Why do I test? How do I test? How will the workflow be organized? What happens with the results? What am I testing? What is the commodity/matrix I have to test? What is the analyte / substance I have to test for?

Interlaboratory studies Why do I test? How do I test? What kind of result is needed? quantitative semiquantitative qualitative exact concentration information approximative concentration levels only YES/NO answer at a certain level

TYPES: Proficiency testing Validation of methods Materials certification Assessing: Laboratory performance Analytical methods performance Reference quantity values Number of Labs. not-specified 8 and more (min.5) not-specified Experience of Labs. Various Only experienced Only experienced Analytical method Various Strictly specified Various Influence on lab. confidence Significant Low Low Tracebility Not required Required Extremely important Target quantity X,, (z) r, R X Protocols: ISO 17043:2010 ISO 5725-2 ISO Guide 35:2006 34:2000

Reference materials o o o Certified In-house Proficiency Scheme samples Usage for: Analytical quality control Method validation Calibration Material or substance one or more of whose property values are sufficiently homogenous and well established to be used for calibration of an apparatus or assessment of a measurement methods - ISO Guide No. 30 1992

CRM certified reference materials -controls or standards used to check the quality and traceability of products - a reference standard for a unit of measurement is an artifact that embodies the quantity of interest in a way that ties its value to the reference base for calibration. - the most of analytical instrumentation is comparative, it requires a sample of known composition (reference material) for accurate calibration. - reference materials are produced under stringent manufacturing procedures and differ from laboratory reagents in their certification and the traceability of the data provided. - Quality management systems involving laboratory accreditation under national and international accreditation/certification standards such as ISO 9000 and ISO 17025 require the use of Reference Materials. - Whilst Certified Reference Materials are preferred, their availability is limited. The available Reference Materials generally differ only in the detail provided on the certificate.

Preparation of reference materials Material is prepared with desired characteristic Homogeneity Stability Certified through a round robin study to establish Reference (consensus) value μ uncertainty of values u or U or confidence interval (95 %) Usage of CRM s improve the reliability lab by means of verifying the accuracy and precision of the lab/method Many matrix/analytes combinations available - does not cover every combination Expensive and limited supply Not relevant where either matrix or analytes are Caution must be exercised when interpreting the results Poor availability pesticide and mycotoxins

Reference materials commercially available Properties: o Stable o Homogenous - to allow sub-sampling without bias o Large amounts allow use over a long period o Over 100 producers of CRM s worldwide o LGC/Promochem in the UK o BCR/IRRM at the EU level o Coordinated exercise to prepare CRM s o o o Certified In-house Proficiency Scheme samples

Inter-lab validation of method - AOAC

Purpose of proficiency tests The primary purpose of proficiency testing is to help laboratories detect and cure any unacceptably large inaccuracy in their reported results. THE INTERNATIONAL HARMONIZED PROTOCOL FOR THE PROFICIENCY TESTING OF ANALYTICAL CHEMISTRY LABORATORIES (IUPAC Technical Report). MICHAEL THOMPSON, STEPHEN L. R. ELLISON, AND ROGER WOOD: Pure Appl. Chem., Vol. 78, No. 1, pp. 145 196, 2006.

Purpose of proficiency tests evaluation of the performance of laboratories for specific tests and monitoring laboratory performance, identification of problems in laboratories and initiation of action for improvement of which, establishment of the effectiveness and comparability of test methods, provision of conffidence to laboratory customers and lab management, education of participating laboratories based on the outcomes of comparisons, validation of uncertainty claims, external quality control activities required by accreditation bodies and/or authorities, other benefits?

Proficiency testing harmonogram Preparation & verification of testing material Report distribution identification of nonconformities (by labs) Distribution of testing material to participants Assessment & Reporting of results Treatment of results Analysis of testing material by participants

PT providers basic tasks:

PT providers basic tasks:

PT providers

PT providers

Proficiency Testing for Food Chemistry Organization

Assessments of results z-score Individual value for each result: z x Xˆ p x = value (concentration) in testing material reported by laboratory X = assigned value (considered as true) p = target value of standard deviation (reproducibility)

Assessments of results

Assessments of z-score z 2 Satisfactory 2 < z 3 Questionable (95 %) z > 3 Unsatisfactory (99 %) z-scores 4 3 2 1 0-1 -2-3 -4 Unsatisfactory FP! Questionable FP? GOOD! GOOD! Questionable FN? Unsatisfactory FN!

Proficiency Testing for Food Chemistry FAPAS - Food Analysis Performance Assessment Scheme Provider: FERA, UK (http://www.fapas.com/): Nutritional components Food ingredients Natural food contaminants Organic & inorganic contaminants Pesticides and Veterinary drug residues Food additives Migration from food packaging

FAPAS proficiency tests FAPAS - Food Analysis Performance Assessment Scheme, www.fapas.com Report: - Assigned value - Results of all laboratories - Z-score of all laboratories - Used methods

FAPAS proficiency tests

PT results evaluation - statistics 1. Assessment of assigned value (X ref ) known (reference) value addition of std., CRM, median of all results mean of results mean value of reference (expert) laboratories mean value of experienced laboratories using standard testing method (ISO, EN, DIN )

PT results evaluation - statistics Histogram with two appex, i.e. low frequencies inside: Data population is non-homogeneous i.e. affected mostly by different analytical methods with different bias. 10 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9

Assessment: various metods Estimation of the asigned value (X ref ) method bias

PT results evaluation - statistics Assessment of assigned value (X ref ) Mean = 0,116 Median = 0,160 Fig. 1 - MTZL V/6 z-score ARSENIC ( Xref = 0.116 mg/kg) Fig.. 1 - MTZL V/6 z-score ARSENIC (Xref = 0.160 mg/kg) 4 4 2 2 0,227 z-score 0 0,116 z-score 0 0,160-2 -2 0,093-4 7 2 8 13 11 14 5 9 1 12 10 4 3 6 Laboratory No. -4 7 2 8 13 11 14 5 9 1 12 10 4 3 6 Laboratory No.

Uncertainty of assigned value - possibilities 1a. X ref is established of median of all results, when each of laboratory is giving its own uncertainty (ISO 13528:2005) p number of expert laboratories u i standard deviation of i-laboratory 1b. X ref is established as mean of values, uncrtainties are not taken into consideration u X rob n (FAPAS, harmonized protocol)

Uncertainty of assigned value When the number of participants is smaller than about 15, even the statistical uncertainty on the consensus (as the standard deviation) will be undesirably high. (THOMPSON M., ELLISON S.L.R., WOOD R.: Pure Appl. Chem., Vol. 78, No. 1, 2006, p. 145) ISO 13528:2005 Guidelines for limiting the standard uncertainty of the assigned value: In case: ux 0. 3 p ( p assigned value for PT) Than is uncertainty of results small and cannot be taken into acconsideration.

General criteria precision and trueness AOAC/FAO/IAEA/IUPAC: Guidelines for single-laboratory validation of analytical methods for tracelevel concentrations of organic chemicals. Report of the AOAC/FAO/IAEA/IUPAC Expert Consultation on Single-laboratory validation of Analytical Methods for Trace-level

Evaluation of results in PT Examples of target values (as RSD %): Analyte Conc. level (%) Source Ochratoxin A 3 g/kg 38 Horwitz Vitamin B 2 24 mg/kg 20 Standard (ISO) Nitrates 150 mg/kg 7,5 Horwitz Cyclamate 195 mg/kg 3,1 Standard (ISO) ph 3.25 1,5 Standard (CSN) ethanol 11.48 % 0,87 Standard (CSN)

Evaluation of results in PT Individual z-score z <-2;+2> z x p Xˆ Do not set up any social ladder of laboratories based on z-scores within -1; +1 Combined z-score: weighted sum 1) assign the index ω lzl to each z-score for individual analytes in this way: 2) Calculate WSZ as the average of all ωlzl values : z <-2;+2>

Evaluation of results in PT Combined z-score RSZ i z i / n Rescaled Sum of Z-scores, sensitive to systematic errors / trends SSZ i 2 zi Sum of Squared Z-scores, sensitive to random errors and outliers

Participation of official control laboratories Proficiency tests organized by Community Reference Laboratories - EUPT (European Commission s Proficiency Testing Program ) ICT Prague 3rd best results in Europe

Evaluation of results in PT uncertainty?

Fit For Purpose assessment of uncertainty E zl n u x x x U 2 2 x L x Xˆ Xˆ 2 ux Xˆ U 2 X z L -score specific to fit individual laboratory criteria (e.g. L = 20% as RSD) z L <-2;+2> zeta-score: z-score reflecting differences related to standard uncertainty reported by individual laboratory zeta <-2;+2> E n -score: zeta-score calculated using expanded uncertainty E n <-1;+1>

FAPAS Series 19, Round 37 Matrix: lemons Analyt: fenitrothion z-skore: -0.3 Lab. code: 034

FAPAS Series 19, Round 35 Matrix: lettuce Analyte: lambda-cyhalotrin z-skore: -2,1 Eluční profil pesticidů na GPC (PL gel [60 x 0,2 cm, 10 um]) jímaná frakce 14.5-31 ml 70% 50% relativní odezva 30% 10% 0-10% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 frakce [ml] difenylamin metalaxyl fenitrothion-npd cyhalothrin-lambda koextrakty Lab. code: 075

FAPAS Series 19, Round 32 Baby Food test identification of pesticides on trace levels MRL = 10 g/kg Matrix: carrot puree Lab. code: 003

EU PT 07: Incurred residues of pesticides in grapes homogenate (organized by CRL for pesticides) Laboratories assessment: 1. Individual z-skore (z-skore + FFP skore) 2. Number of analytes in methods 3. Number of good identified analytes (TP) 4. Number of not-good identified analytes (FP) 5. Weigh sum z-skore

Documents: THE INTERNATIONAL HARMONIZED PROTOCOL FOR THE PROFICIENCY TESTING OF ANALYTICAL CHEMISTRY LABORATORIES (IUPAC Technical Report). MICHAEL THOMPSON, STEPHEN L. R. ELLISON, AND ROGER WOOD: Pure Appl. Chem., Vol. 78, No. 1, pp. 145 196, 2006. ISO/IEC 17043:2010 General requirements for preparation and distribution suggestion, selection of materials, methods and procedures, Performance and evaluation, reports, communication with participants, conffidence, security, Requirements for management, Statistical methods

ISO/IEC 17043:2010

Thank you for your attention vladimir.kocourek@vscht.cz