Importance of Proficiency Testing/Interlaboratory Comparison

Transcription

1 Importance of Proficiency Testing/Interlaboratory Comparison Dr. Erik J.M. Konings, Nestlé Research Centre, Lausanne, Switzerland International Workshop and Training Program on Good Food Laboratory Practices to be held on November 15-19, 2016 in Mumbai. Organized By Food Safety and Standard Authority of India, Government of India & International Life Sciences Institute-India & Export Inspection Council, Government of India

2 Agenda 1 Introduction 2 General requirements for Analytical Quality Assurance 3 Introduction to Proficiency Testing (process, statistical design, benefits) 4 Nestlé Proficiency Testing 5 AOAC INTERNATIONAL Proficiency Testing (to start as from March 2017)

3 Poor Quality data can result in substantial problems and high costs Consequences of reporting data of poor quality for the end-users? High costs involved in repeated measurements to correct poor data; Faulty decision-making; Damage to reputation and credibility; Possible loss of business; Legal and financial liability from the use of poor data.

4 The various tools helping laboratories to demonstrate the reliability of their analytical data

5 ISO/IEC 17025: General requirements for the competence of testing and calibration laboratories Chapter 5.9: Ensuring the quality of test and calibration results

6 What is Proficiency Testing? ISO/IEC 17043:2010 General requirements for Proficiency Testing Proficiency testing schemes (PTS) are interlaboratory comparisons that are organized regularly to assess the performance of analytical laboratories and the competence of the analytical personnel.

7 Proficiency testing The Process P-test Provider Testing laboratory Prepare samples Homogeneity and Stability testing Ship samples Analyse samples Conduct Root cause analysis Prepare report Analyse results Report results

8 How often does a lab has to participate?

9 Proficiency testing The Statistical design How is the assigned value established? The assigned value is the value selected as being the best estimate of the true value for the parameter under test. The method used to determine the assigned value may vary depending upon the particular scheme and test material. As per ISO Statistical methods for use in proficiency testing by interlaboratory comparisons, there are five ways to determine the assigned value : Formulation Certified references values Results from one laboratory Consensus from experts laboratories Consensus value from participants (e.g. median) For Qualitative tests, participant s results are compared against the intended result based on formulation. For Microbiology samples, participant s results are transformed by converting them to log10 before the statistical analyses is undertaken.

10 Proficiency testing The Statistical design How is the standard deviation for proficiency assessment (SDPA) determined? The SDPA expressed the acceptable difference between the laboratory results and the assigned value. The method used to determine the assigned value may vary depending upon the particular scheme and test material. As per ISO Statistical methods for use in proficiency testing by interlaboratory comparisons there are five ways to determine the SDPA: Prescribed value By perception From a general model From results of precision experiment From data obtained on a round of proficiency testing scheme

11 Participant s result converted in Z-score. How is the z-score calculated? z-score is a scaling that makes the errors of equal magnitude in different situations, (normalized different errors in different methods) allowing to compare the performance of the laboratories observed in different tests and with different analytes. As per ISO 13528, it is the ratio between the deviation of each individual result from the assigned value, and the standard deviation for proficiency assessment SDPA.

12 How do I interpret the z-score?

13 What to do with unsatisfactory results? ISO/IEC Management Requirements 4.9 Manage non conformities If anything in the lab is not conform to what it shall be, a procedure must be activated to: Take immediate actions (repeat analyses, contact customer if necessary, remove an instrument out of order) Evaluate the cause and take corrective actions (train people, replace out of spec instrument, etc) Evaluate consequence of the work done Verify that the corrective actions prevent similar problems

14 Example of FAPAS report If the following criteria is not met: u(xpt) < 0,3 *σpt (standard uncertainty of the assigned value lesser than 0.3 times the Standard Deviation of Proficiency Assessment) The uncertainty is not considered as negligible and needs to be included in the interpretation of the results. In this case other statistical design should be considered (e.g. use of z - score).

15 Example of FAPAS report

16 There are multiple benefits taking part to Proficiency Testing Demonstrate good performance of the laboratories Detecting method related issued Raising trust in supplier s certificates Comparing the various analytical methods Generating reference materials for methods monitoring

17 Example of Method Related Issues detected in P-test

18 Example of method performance comparison Source: LGC report

19 Nestlé Proficiency Testing Is an accredited ISO/IEC Proficiency Testing Provider, by the Swiss Accreditation Service. Comprehensive testing programs offering a wide range of analytes and matrices Work with external P-test providers to offer the most suitable P-test programme adapted to Nestlé product specifications (analyte, matrix, concentration) Participation to the Nestlé Proficiency Tests programme is mandatory for the Nestlé laboratories For parameters and/or matrices not covered by the Nestlé P-Test program, each laboratory must also participate in Proficiency test schemes provided by external organizations like FAPAS, LGC, Bipea etc. accredited as a P-test provider based on ISO/IEC 17043

20 Who are the Nestlé users of Proficiency Testing? 900 laboratories Internal laboratories External laboratories (e.g. private, governmental, supplier) Worldwide located (90 countries) Using around 600 test methods and 120 official methods Delivering 128 million of analytical results per year

21 Nestlé P-test Program - key figures 3 domains CHEMICAL NUTRIENTS CHEMICAL CONTAMINANTS (organised in collaboration with Fapas, LGC, Bipea) MICROBIOLOGY (organised in collaboration with LGC) 244 analytical determinations Fats Sugars Vitamins Minerals Proximates Fatty acids Allergens Antibiotics GMO Melamine& cyanuric acid Metallic contaminants Mycotoxins Nitrite/nitrate Pesticides Authenticity Probiotics Pathogens Hygiene indicators 42 food products Infant formula Cereals Pet food Health care product Confectionery Sweetened/Unsweetened condensed milk Bouillon Water Beverages Oil Spicy Fruit Purée Milk powder Cereals Meat Milk powder Lyophilized analytical results /year

22 How do we treat non-satisfactory P-test results at Nestlé? A root cause analysis (RCA) must be conducted after every non-satisfactory results i.e. for each unsatisfactory and questionable result. Remedial action must be taken to solve the problem The investigation must be carried out immediately after each scheme round report has been published. Repeat samples are sent to the laboratory that failed to verify effectiveness of the corrective actions Two consecutive unsatisfactory results must trigger a specific action Key learnings and P-test best practices are shared with laboratories to prevent gaps reoccurrence Example of key learning

23 The challenges from the Nestlé P-Test provider perspective Barrier to start new P-Tests Specific/exotic P-test demands Less than 8 results engaged, limitation of the statistical design Difficulties to meet criteria of homogeneity and stability check Ensure samples delivery in every countries Increasing number of documentation requested by the customs Laboratory gives special treatment to P-Test sample Testing multiple times and reporting an average Selection of senior technicians, not involved in everyday routine analysis, to perform the analysis of P-Test samples

24 AOAC Proficiency Testing AOAC INTERNATIONAL is an A2LA accredited Proficiency Testing Provider. First accredited program for analytical food laboratories in the United States, in Comprehensive testing programs offering a wide array of analytes and matrices AOAC s proficiency testing samples arrive at the laboratory ready to analyze like routine samples. The samples undergo rigorous homogeneity testing prior to shipment

25 Methods endorsed by CODEX Vitamin A ISO 20633:2015 AOAC Total nucleotides ISO 20638:2015 AOAC Pantothenic acid ISO 20639:2015 AOAC Iodine ISO IDF 234:2015 AOAC Cr, Mo, Se ISO IDF 235:2015 AOAC

26 Implementation of Infant Formula and Adult Nutritionals PT Program As of March 2017 Nutrients: Fatty acids Vitamin E Vitamin B 12 Panthothenic acid Iodine Vitamin A Myo-Inositol Nucleotides Ultra Trace Minerals (Cr, Mo, Se) Quarterly $650 per Shipment 30 days for analysis

27 Example of Method Performance Information

28 Why is participating to Proficiency Testing/Interlaboratory comparison important? Proficiency test is a key element of quality assurance to assess the proficiency of laboratories Proficiency tests are pivotal to help demonstrate the reliability of laboratory results. P-test helps to identify problems in laboratories and initiate actions for improvement P-test plays a key role as it maintains confidence in laboratories capability to perform analysis, irrespectfully whether or not accredited to ISO (also interesting for third parties, e.g. authorities, customers)

29 Acknowledgement Thanks to Laura Coisne, Nestlé P-test Manager, and the Nestlé P-test team for providing the information! CONTACT: