Importance of Proficiency Testing/Interlaboratory Comparison Dr. Erik J.M. Konings, Nestlé Research Centre, Lausanne, Switzerland International Workshop and Training Program on Good Food Laboratory Practices to be held on November 15-19, 2016 in Mumbai. Organized By Food Safety and Standard Authority of India, Government of India & International Life Sciences Institute-India & Export Inspection Council, Government of India
Agenda 1 Introduction 2 General requirements for Analytical Quality Assurance 3 Introduction to Proficiency Testing (process, statistical design, benefits) 4 Nestlé Proficiency Testing 5 AOAC INTERNATIONAL Proficiency Testing (to start as from March 2017)
Poor Quality data can result in substantial problems and high costs Consequences of reporting data of poor quality for the end-users? High costs involved in repeated measurements to correct poor data; Faulty decision-making; Damage to reputation and credibility; Possible loss of business; Legal and financial liability from the use of poor data.
The various tools helping laboratories to demonstrate the reliability of their analytical data
ISO/IEC 17025: General requirements for the competence of testing and calibration laboratories Chapter 5.9: Ensuring the quality of test and calibration results
What is Proficiency Testing? ISO/IEC 17043:2010 General requirements for Proficiency Testing Proficiency testing schemes (PTS) are interlaboratory comparisons that are organized regularly to assess the performance of analytical laboratories and the competence of the analytical personnel.
Proficiency testing The Process P-test Provider Testing laboratory Prepare samples Homogeneity and Stability testing Ship samples Analyse samples Conduct Root cause analysis Prepare report Analyse results Report results
How often does a lab has to participate?
Proficiency testing The Statistical design How is the assigned value established? The assigned value is the value selected as being the best estimate of the true value for the parameter under test. The method used to determine the assigned value may vary depending upon the particular scheme and test material. As per ISO 13528 Statistical methods for use in proficiency testing by interlaboratory comparisons, there are five ways to determine the assigned value : Formulation Certified references values Results from one laboratory Consensus from experts laboratories Consensus value from participants (e.g. median) For Qualitative tests, participant s results are compared against the intended result based on formulation. For Microbiology samples, participant s results are transformed by converting them to log10 before the statistical analyses is undertaken.
Proficiency testing The Statistical design How is the standard deviation for proficiency assessment (SDPA) determined? The SDPA expressed the acceptable difference between the laboratory results and the assigned value. The method used to determine the assigned value may vary depending upon the particular scheme and test material. As per ISO 13528 Statistical methods for use in proficiency testing by interlaboratory comparisons there are five ways to determine the SDPA: Prescribed value By perception From a general model From results of precision experiment From data obtained on a round of proficiency testing scheme
Participant s result converted in Z-score. How is the z-score calculated? z-score is a scaling that makes the errors of equal magnitude in different situations, (normalized different errors in different methods) allowing to compare the performance of the laboratories observed in different tests and with different analytes. As per ISO 13528, it is the ratio between the deviation of each individual result from the assigned value, and the standard deviation for proficiency assessment SDPA.
How do I interpret the z-score?
What to do with unsatisfactory results? ISO/IEC 17025 Management Requirements 4.9 Manage non conformities If anything in the lab is not conform to what it shall be, a procedure must be activated to: Take immediate actions (repeat analyses, contact customer if necessary, remove an instrument out of order) Evaluate the cause and take corrective actions (train people, replace out of spec instrument, etc) Evaluate consequence of the work done Verify that the corrective actions prevent similar problems
Example of FAPAS report If the following criteria is not met: u(xpt) < 0,3 *σpt (standard uncertainty of the assigned value lesser than 0.3 times the Standard Deviation of Proficiency Assessment) The uncertainty is not considered as negligible and needs to be included in the interpretation of the results. In this case other statistical design should be considered (e.g. use of z - score).
Example of FAPAS report
There are multiple benefits taking part to Proficiency Testing Demonstrate good performance of the laboratories Detecting method related issued Raising trust in supplier s certificates Comparing the various analytical methods Generating reference materials for methods monitoring
Example of Method Related Issues detected in P-test
Example of method performance comparison Source: LGC report
Nestlé Proficiency Testing Is an accredited ISO/IEC 17043 Proficiency Testing Provider, by the Swiss Accreditation Service. Comprehensive testing programs offering a wide range of analytes and matrices Work with external P-test providers to offer the most suitable P-test programme adapted to Nestlé product specifications (analyte, matrix, concentration) Participation to the Nestlé Proficiency Tests programme is mandatory for the Nestlé laboratories For parameters and/or matrices not covered by the Nestlé P-Test program, each laboratory must also participate in Proficiency test schemes provided by external organizations like FAPAS, LGC, Bipea etc. accredited as a P-test provider based on ISO/IEC 17043
Who are the Nestlé users of Proficiency Testing? 900 laboratories Internal laboratories External laboratories (e.g. private, governmental, supplier) Worldwide located (90 countries) Using around 600 test methods and 120 official methods Delivering 128 million of analytical results per year
Nestlé P-test Program - key figures 3 domains CHEMICAL NUTRIENTS CHEMICAL CONTAMINANTS (organised in collaboration with Fapas, LGC, Bipea) MICROBIOLOGY (organised in collaboration with LGC) 244 analytical determinations Fats Sugars Vitamins Minerals Proximates Fatty acids Allergens Antibiotics GMO Melamine& cyanuric acid Metallic contaminants Mycotoxins Nitrite/nitrate Pesticides Authenticity Probiotics Pathogens Hygiene indicators 42 food products Infant formula Cereals Pet food Health care product Confectionery Sweetened/Unsweetened condensed milk Bouillon Water Beverages Oil Spicy Fruit Purée Milk powder Cereals Meat Milk powder Lyophilized 41 000 analytical results /year
How do we treat non-satisfactory P-test results at Nestlé? A root cause analysis (RCA) must be conducted after every non-satisfactory results i.e. for each unsatisfactory and questionable result. Remedial action must be taken to solve the problem The investigation must be carried out immediately after each scheme round report has been published. Repeat samples are sent to the laboratory that failed to verify effectiveness of the corrective actions Two consecutive unsatisfactory results must trigger a specific action Key learnings and P-test best practices are shared with laboratories to prevent gaps reoccurrence Example of key learning
The challenges from the Nestlé P-Test provider perspective Barrier to start new P-Tests Specific/exotic P-test demands Less than 8 results engaged, limitation of the statistical design Difficulties to meet criteria of homogeneity and stability check Ensure samples delivery in every countries Increasing number of documentation requested by the customs Laboratory gives special treatment to P-Test sample Testing multiple times and reporting an average Selection of senior technicians, not involved in everyday routine analysis, to perform the analysis of P-Test samples
AOAC Proficiency Testing AOAC INTERNATIONAL is an A2LA accredited Proficiency Testing Provider. First accredited program for analytical food laboratories in the United States, in 2001. Comprehensive testing programs offering a wide array of analytes and matrices AOAC s proficiency testing samples arrive at the laboratory ready to analyze like routine samples. The samples undergo rigorous homogeneity testing prior to shipment
Methods endorsed by CODEX Vitamin A ISO 20633:2015 AOAC 2012.10 Total nucleotides ISO 20638:2015 AOAC 2011.20 Pantothenic acid ISO 20639:2015 AOAC 2012.16 Iodine ISO 20647 IDF 234:2015 AOAC 2012.15 Cr, Mo, Se ISO 20649 IDF 235:2015 AOAC 2011.19
Implementation of Infant Formula and Adult Nutritionals PT Program As of March 2017 Nutrients: Fatty acids Vitamin E Vitamin B 12 Panthothenic acid Iodine Vitamin A Myo-Inositol Nucleotides Ultra Trace Minerals (Cr, Mo, Se) Quarterly $650 per Shipment 30 days for analysis
Example of Method Performance Information
Why is participating to Proficiency Testing/Interlaboratory comparison important? Proficiency test is a key element of quality assurance to assess the proficiency of laboratories Proficiency tests are pivotal to help demonstrate the reliability of laboratory results. P-test helps to identify problems in laboratories and initiate actions for improvement P-test plays a key role as it maintains confidence in laboratories capability to perform analysis, irrespectfully whether or not accredited to ISO 17025 (also interesting for third parties, e.g. authorities, customers)
Acknowledgement Thanks to Laura Coisne, Nestlé P-test Manager, and the Nestlé P-test team for providing the information! CONTACT: Erik.Konings@rdls.nestle.com