FOOD FOR THOUGHT Topical Insights from our Subject Matter Experts UNDERSTANDING WHAT IS NEEDED TO PRODUCE QUALITY DATA

Transcription

1 FOOD FOR THOUGHT Topical Insights from our Subject Matter Experts UNDERSTANDING WHAT IS NEEDED TO PRODUCE QUALITY DATA The NFL White Paper Series Volume 7, January 2013

2 Overview and a Scenario With so many fictional crime solving dramas on television it s easy to believe that analytical results can be determined instantaneously. My personal favorite is when a scientist injects a blood sample directly into a gas chromatograph (GC), and within 30 seconds a printer spits out a list of all foreign substances in the blood. The investigator then grabs the printout and runs off to solve the crime. This type of drama has fueled the belief in an analytical black box, a magical instrument that answers all analytical questions in one quick test. As useful as that would be, no single test or instrument is capable of answering all questions. Additionally, a single injection is rarely enough to provide even one good result. To obtain accurate and precise results, the single injection of the blood sample should have been followed by a minimum of five quality control injections. Without adequate quality control data, results can be misleading or incorrect. A food research center based at a public research university in China tested several American brands of infant formula and reported that they contained the prohibited food flavoring vanillin. This report caused a public outcry and the price per share of company stock dropped immediately. Upon further investigation, the university said the test results were errant and issued an apology to the affected companies. "The test result is our mistake," the center said. At least two questions come to mind in the above example: 1) What failed? and 2) Did the university have the correct quality control systems in place? Answers to these questions and the risks they create can be directly linked to using testing laboratories that have properly managed and accredited quality programs. The International Organization for Standardization (ISO) and the Association of Analytical Communities (AOAC) set the standard for quality programs in the food industry. A laboratory accredited by ISO 17025:2005 and AOAC has to meet very specific requirements to ensure test results are accurate. These requirements may seem simple and even obvious to some, but the execution is really what defines quality data. Since the devil is in the details, let s take a closer look at the requirements needed to prevent such failures, and some of the more common areas you should understand and question when commissioning such work. Use the Correct Method Choosing the appropriate method is the first critical step in ensuring accurate and reliable results. Considering methods published by industry-recognized organizations is a good start. These methods are reviewed by knowledgeable members of the scientific community and have good supporting data to ensure a reliable outcome. Most widely used methods come from the AOAC or FDA and are often called Standard Methods. Whether performing a quantitative (a number) or qualitative (pass/fail) test, you must make sure that the method is appropriate. The following factors must be considered when choosing a method: objective of the test, type of matrix, detection limit and turnaround time needs. For example, if a client sends in a meat sample for E. coli testing and does not require USDA methodology or a specific detection limit, but needs completed results in 2 days, then it would be more appropriate to use the 3M TM Petrifilm over the Most Probable Number (MPN) method, the latter of which could take up to 7 days to get confirmed results. If you do not know what method is best, a knowledgeable lab should recommend the appropriate method to meet your specific requirements. Understanding What Is Needed To Produce Quality Data Page 2 of 9

3 Verify or Validate the Method Method verification is a process in which a lab proves they are able to perform a Standard Method correctly. Method validation is the process in which a lab proves a non-standard method, such as a method developed in-house, actually works as intended and meets the requirements of a client. Most microbiological and many food chemical tests use Standard Methods, so verification is all that is needed. Many food contaminant tests require Standard Methods to be modified in such a way that a complete validation is needed. Table 1 below, based on information from US Pharmacopeia, summarizes the parameters required to validate a method. Table 1: Method Validation Parameters Category I II III Parameter Major Quantitative Components Impurities Performance Accuracy Yes Yes * Precision Yes Yes Yes Limit of Detection No No * Limit of Quantitation No Yes * Linearity Yes Yes * Range Yes Yes * Ruggedness Yes Yes Yes * May be required, depending on the nature of the specific test. Elements of Method Verification or Validation Accuracy and Precision Accuracy is the closeness of the test results obtained by the method to the actual or true value. Precision is the degree of agreement among individual test results when the procedure is applied repeatedly to multiple samplings of a homogeneous sample. An easy way to visualize accuracy and precision is to picture targets, such as those shown below in Figure 1. The center bull s-eye equates to the true value. Points close to the center show accuracy. A tight cluster of points on the target, wherever they may be, shows precision. Ideally you want a tight cluster of points in the center, showing both accuracy and precision. Understanding What Is Needed To Produce Quality Data Page 3 of 9

4 Figure 1: The Relationship Between Accuracy and Precision Accurate (The average is accurate, not precise) Precise (Not accurate) Accurate and Precise Accuracy can be determined by comparing the results of a series of samples to the true value. The true value can be the published results of a certified reference material, calculated by consensus of multiple laboratories, or comparison of the results of fortified samples, also referred to as spiked samples, against the published method. Precision can be determined by calculating the standard deviation of a series of like samples. Limit of Detection and Limit of Quantitation Limit of Detection (LOD) is the minimum concentration of a substance that can be detected by a method. Limit of Quantitation (LOQ) is the minimum concentration of a substance that can be measured and reported with 99% confidence. The rule of thumb is the LOD is 3 times the noise (standard deviation) above the blank response, and the LOQ is 10 times the noise. LOD and LOQ are determined by calculating the standard deviation of a series of samples at the suspected LOD. Linearity Linearity compares the relationship between an instrument response and concentration of an analyte. The ideal situation is when there is a one to one linear response, producing a graph similar to that presented below in Figure 2. Understanding What Is Needed To Produce Quality Data Page 4 of 9

5 Figure 2: Example Standard Curve to Demonstrate Linearity Instrument response Standard Curve ppb To determine the linearity, a series of analytical standards at different concentrations is analyzed. Range Range is the upper and lower concentration limits of the analyte that can be determined with precision, accuracy, and linearity. Range is determined by fortifying a blank matrix with a representative analyte at the highest expected value of your samples. Ruggedness Ruggedness is the degree of reproducibility of test results obtained by the analysis of the same samples under a variety of normal test conditions, such as different analysts. Ruggedness is determined by comparing the results of a series of samples analyzed under different conditions. Use Appropriate Standards Many methods require analytical standards to properly identify the analyte and/or determine the concentration. For the best results, standards need to be purchased from certified reference material providers when possible or other reputable sources. No matter the source, all standards should have a certificate of analyses to prove authenticity. Understanding What Is Needed To Produce Quality Data Page 5 of 9

6 Standards are used in multiple ways: A standard curve, usually 3-5 different concentrations of the standard, is run at the start of a sequence to show linearity and to ensure the instrument is running properly. An Internal Calibration Verification (ICV) is a standard often of a different source or lot number. The ICV is run to prove the other standards being used have not degraded and that the instrument is running properly. A Continuing Calibration Verification (CCV) is a standard run throughout a sequence to show consistency and that the integrity of the instrument has not changed from start to finish. This standard is at a concentration of analyte most commonly found in samples. A CCV is often placed between every set of 10 samples. Participate in Proficiency Programs ISO accredited labs must participate in proficiency testing (PT) programs on a yearly basis. PT programs enable laboratories to assess their performance by comparing their data with other laboratories. Two such programs are AOAC and American Proficiency Institute (API). Perform Internal Quality Control Once the right method and standards have been chosen and validated and the lab has proven its proficiency, the next step is having systems in place to ensure that everything goes right on any given day. Whether it s over- or under-reporting moisture levels, or incorrectly reporting results for E. coli O157:H7, testing or reporting errors can jeopardize the reputation of a brand as well as the lab. Thus, labs must have programs in place to minimize such errors. Proficiency programs are great for initial and occasional checks, but there must also be internal programs for ongoing verification. This can be done through the use of Laboratory Control Samples (LCS), also referred to as Quality Control (QC) samples. LCS are samples with known true values. Running LCS with every batch of samples will demonstrate accuracy every time the test is run. The type of LCS is chosen to reflect the qualities of common client samples. Spiking of client samples may also be performed for difficult or uncommon matrices to check for sample matrix effects. There are several types of LCS that can be used in a food testing laboratory to prove accuracy: When testing for stable components of food, such as moisture or salt content, the LCS could be food samples with a known true value of the component of interest. The true values can be determined by the certificates of analysis if Standard Reference Materials (SRM) are used or by a consensus of at least three other laboratories results if general food samples are used. Understanding What Is Needed To Produce Quality Data Page 6 of 9

7 When testing for quantitative impurities or unstable food components, such as pesticide residues or vitamins, the LCS are generally fortified samples. Fortifying a sample is to add known amounts of test components to the clean sample at the beginning of the test, and then prepare the fortified sample for analysis like all other samples. The amount recovered from the fortified sample provides an indication of how effective the analytical procedure was for that particular sample type. When testing for microorganisms the LCS can be set up from working slants or lyophilized cultures. Working slants can be more difficult to maintain and come with a lower out-ofpocket cost, while lyophilized cultures are easier to maintain and come with a much greater out-of-pocket cost. o For quantitative analysis, the LCS are working slants or lyophilized cultures with a known level of growth and are processed right along client samples. o For qualitative samples the LCS are set up as positive and negative controls that are tested for the same assay as the client samples. If presumptive results are obtained with a client sample, then the LCS are also carried through the confirmation process. LCS results are plotted on control charts to demonstrate that results are within the acceptable range and prove the method s accuracy, producing a chart similar to that presented below in Figure 3. Figure 3: Example Chart Demonstrating Accuracy Relative Percent Difference for Repeated Analysis % of Expected Concentration Test Number To determine precision, duplicates are run along with the client samples. Over time the method precision and measurement of uncertainty can be calculated. Results within 3 standard deviations prove acceptable precision. Results are plotted on control charts, producing a chart similar to that presented below in Figure 4. Understanding What Is Needed To Produce Quality Data Page 7 of 9

8 Figure 4: Example Chart Demonstrating Precision Relative Percent Difference for Duplicate Analysis Relative Percent Difference Test Number Confirm Results, When Possible The last quality check is to confirm results. This is accomplished by performing a second review of all data by a qualified analyst. For certain tests when components or organisms are detected, samples may be reanalyzed and/or confirmatory tests may be performed. Looking Forward and being Prepared Quality data does not come easily or instantaneously. It takes time and dedication to get the correct answer. ISO provides the road map for laboratories to develop a comprehensive and robust quality management system. While it is not always feasible for a lab to be ISO accredited for every test they perform, having quality management systems in place throughout the lab can ensure that the lab has taken the time and is dedicated to getting the correct answer for all tests. Now is the time to review your laboratory s quality management system. Although the rule making process for the Food Safety Modernization Act (FSMA) is far from complete, the law requires certain food testing to be carried out by accredited laboratories and directs the FDA to establish a program for laboratory accreditation to ensure that U.S. food testing laboratories meet high-quality standards. It is unknown whether the FDA will accept the ISO accreditations or develop their own program, but laboratories with strong quality management systems will be one step closer to meeting this regulatory standard, and thus better prepared to protect your brand portfolio. Understanding What Is Needed To Produce Quality Data Page 8 of 9

9 About The NFL The National Food Laboratory is a food and beverage consulting and testing firm providing creative, practical and science-based solutions for the following areas: Food Safety and Quality; Product and Process Development; and Sensory and Consumer Research. We create value for our clients by enabling them to develop commercially safe, high quality and great tasting foods and beverages. For more information about The National Food Laboratory, please visit us at For more information please contact: Julie Hill, Vice President, Chemistry, at [email protected], or Shaunti Luce, Division Manager, Analytical Microbiology, at [email protected], or Bob Takens, Quality Assurance Manager, at [email protected], Understanding What Is Needed To Produce Quality Data Page 9 of 9