ICMSF Lecture on Microbiological Sampling Plans Susanne Dahms IAFP, San Diego, 2002 Client - meeting - - 1
Overview Introduction Sampling plans: Design and means to study their performance Two-class attributes plans for qualitative data Two-class and three-class attributes plans for grouped quantitative data Variables plans for quantitative data Microbiological sampling plans and Food Safety Objectives Summary Client - meeting - - 2
ICMSF Books on Microbiological Sampling Plans Microorganisms in Foods 2. Sampling for Microbiological Analysis: Principles and Specific Applications. (2nd ed. 1986, University of Toronto Press, out of print) Topics: The statistical principles underlying attributes sampling plans and their application to foods. Client - meeting - - 3
ICMSF Books on Microbiological Sampling Plans Microorganisms in Foods 7. Microbiological Testing in Food Safety Management. (2002, Kluwer Academic/Plenum Publishers) Topics: Concepts of probability and sampling Appropriate sampling plans Selection of cases and attributes plans Establishing micro criteria for lot acceptance Client - meeting - - 4
ICMSF Cases 15 cases which reflect severity of the hazard, effect of handling/preparation on the hazard, and intended population Type of hazard Utility Conditions reduce hazard Case 1 Conditions cause no change in hazard Case 2 Conditions may increase hazard Case 3 Indicator Case 4 Case 5 Case 6 Moderate Case 7 Case 8 Case 9 Serious Case 10 Case 11 Case 12 Severe Case 13 Case 14 Case 15 Client - meeting - - 5
Types of Microbiological Sampling Plans Attributes plans: Qualitative analytical results (presence/absence) or quantitative results that have been grouped (e.g. <10 cfu/g, 10 to 100 cfu/g, >100 cfu/g) Variables plans: Non-grouped quantitative analytical results Require distributional assumptions be made Client - meeting - - 6
Two-Class Attributes Sampling Plans Two-class sampling plans designed to decide on acceptance or rejection of a lot consist of n number of sample units to be chosen independently and randomly from the lot m a microbiological limit (i.e. in cfu/g); a sample is defined to be positive, if its microbial content exceeds this limit c maximum allowable number of sample units yielding a positive result (presence/absence testing) or exceeding the microbiological limit m; for pathogens c is usually set to 0 Client - meeting - - 7
OC Curve for Two-Class Plans Operation characteristics (OC) or performance for two-class sampling plans: Probability of lot acceptance calculated for possible proportions defective in lot Plot of OC curve to visualize sampling plan performance dependency on n and c Acceptance probability Proportion defective Client - meeting - - 8
Probability of Acceptance by Proportion Defective 1.0 n=5, c=0 0.8 Probability of Acceptance 0.6 0.4 P(rejection) 0.2 P(acceptance) 0.0 0.0 0.2 0.4 0.6 0.8 Proportion Defective
Probability of Acceptance by Proportion Defective 1.0 0.8 P(acceptance)=95% n=5, c=0 Probability of Acceptance 0.6 0.4 0.2 P(rejection)=95% 0.0 0.0 0.2 0.4 0.6 0.8 Proportion Defective
Probability of Acceptance by Proportion Defective 1.0 Probability of Acceptance 0.8 0.6 0.4 n=5, c=0 n=10, c=0 n=20, c=0 0.2 0.0 0.0 0.2 0.4 0.6 0.8 Proportion Defective
Three-Class Attributes Sampling Plans Three-class sampling plans consist of n number of sample units to be chosen independently and randomly from the lot m a microbiological limit that separates good quality from marginally acceptable quality M a microbiological limit above which sampling results are unacceptable or defective c maximum allowable number of sample units yielding results between m and M (marginally acceptable); the number of sample units allowed to exceed M is usually set to 0 Client - meeting - - 12
OC Function for Three-Class Plans Operation characteristics (OC) or performance for three-class plans: Probability of lot acceptance depending on two proportions marginally acceptable: between m and M defective: above M OC function plotted as a three-dimensional graph Acceptance probability Prop. marginally acceptable Proportion defective Client - meeting - - 13
OC Curve Referring to Mean Log CFU/G Alternative approach for quantitative data: Distributional assumption for sampling results e.g. log-normal with standard deviation known from previous experience Client - meeting - - 14
0.6 0.5 Frequency Distribution Describing Lot Quality mean s.d.: standard deviation (=0.8) Probability Density 0.4 0.3 0.2 s.d. s.d. 0.1 0.0 0 1 2 3 4 5 6 Log cfu/g
OC Curve Referring to Mean Log CFU/G Alternative approach for quantitative data: Distributional assumption for sampling results e.g. log-normal with standard deviation known from previous experience Determine proportions acceptable, (marginally acceptable), and defective for possible mean log cfu/g Client - meeting - - 16
0.6 Two-class sampling plan: m 0.5 Probability Density 0.4 0.3 0.2 Proportion defective 0.1 0.0 0 1 2 3 4 5 6 Log cfu/g
0.6 Three-class sampling plan: m M 0.5 Probability Density 0.4 0.3 0.2 Proportion marginally acceptable 0.1 0.0 Proportion defective 0 1 2 3 4 5 6 Log cfu/g
OC Curve Referring to Mean Log CFU/G Alternative approach for quantitative data: Distributional assumption for sampling results e.g. log-normal with standard deviation known from previous experience Determine proportions acceptable, (marginally acceptable), and defective for possible mean log cfu/g Calculate acceptance probabilities and plot against mean log cfu/g Client - meeting - - 19
Probability Density m 0.0 1.0 2.0 3.0 4.0 5.0 6.0 Log cfu/g
1.0 0.8 0.6 0.4 0.2 0.0 Proportion defective, p d m Mean Log cfu/g
Probability of acceptance p d 1.0 0.8 0.6 0.4 0.2 0.0 OC curve n = 10, c = 2 1.0 0.8 0.6 0.4 0.2 0.0 Mean log cfu/g P(accept) p d
Probability of Acceptance by Mean Log cfu/g (s.d.=0.8) 1.0 P(acceptance)=95% n=5, c=0, m=100 cfu/g 0.8 Probability of Acceptance 0.6 0.4 0.2 P(rejection)=95% 0.0-2 -1 0 1 2 3 4 Mean Log cfu/g
Probability of Acceptance by Mean Log cfu/g (s.d.=0.8) 1.0 Probability of Acceptance 0.8 0.6 0.4 n=5, c=0, m=100 cfu/g n=10, c=0, m=100 cfu/g n=20, c=0, m=100 cfu/g 0.2 0.0-2 -1 0 1 2 3 4 Mean Log cfu/g
Probability of Acceptance by Mean Log cfu/g (s.d.=0.8) 1.0 Probability of Acceptance 0.8 0.6 0.4 n=5, c=0, m=100 cfu/g n=10, c=0, m=100 cfu/g n=20, c=0, m=1 cfu/g 0.2 0.0-2 -1 0 1 2 3 4 Mean Log cfu/g
Probability of Acceptance by Mean Log cfu/g (s.d.=0.8) 1.0 Probability of Acceptance 0.8 0.6 0.4 n=5, c=0, m=1 cfu/25g n=10, c=0, m=100 cfu/g n=20, c=0, m=1 cfu/g 0.2 0.0-2 -1 0 1 2 3 4 Mean Log cfu/g
Performance of Sampling Plans Sampling plan stringency, steepness of OC curve, location of critical lot qualities (95% probability of rejection, 95% probability of acceptance) depend on Plan specifications n and c Microbiological limits m and M Client - meeting - - 27
Probability of Acceptance by Mean Log cfu/g, s.d.=0.8 1.0 Probability of Acceptance 0.8 0.6 0.4 Two-Class Plan: n=5, c=1, m=1000 cfu/g Three-Class Plan: n=5, c=1, m=1000 cfu/g, M=10 000 cfu/g 0.2 0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Mean Log cfu/g
Probability of Acceptance by Mean Log cfu/g, s.d.=1.2 1.0 Probability of Acceptance 0.8 0.6 0.4 Two-Class Plan: n=5, c=1, m=1000 cfu/g Three-Class Plan: n=5, c=1, m=1000 cfu/g, M=10 000 cfu/g 0.2 0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Mean Log cfu/g
1.0 Probability of Acceptance by Mean Log cfu/g 3-Class Plan: n=5, c=1, m=1000 cfu/g, M=10000 cfu/g Probability of Acceptance 0.8 0.6 0.4 s.d.=0.8 s.d.=0.4 s.d.=0.2 0.2 0.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Mean Log cfu/g
Performance of Sampling Plans Sampling plan stringency, steepness of OC curve, location of critical lot qualities (95% probability of rejection, 95% probability of acceptance) depend on Plan specifications n and c Microbiological limits m and M Standard deviation s.d. Difference M-m in relation to s.d. Client - meeting - - 31
ICMSF Three-Class Plans: Mean CFU/G Rejected With 95% Probability Case 4: n=5, c=3 5128 cfu/g Case 5: n=5, c=2 3311 cfu/g Case 6: n=5, c=1 1819 cfu/g Case 7: n=5, c=3 3311 cfu/g Case 8: n=5, c=1 1819 cfu/g Case 9: n=10, c=1 575 cfu/g With: m = 1000 cfu/g, M = 10 000 cfu/g, and standard deviation s.d. = 0.8 Client - meeting - - 32
ICMSF Three-Class Plans: Mean CFU/G Accepted With 95% Probability Case 4: n=5, c=3 138 cfu/g Case 5: n=5, c=2 115 cfu/g Case 6: n=5, c=1 63 cfu/g Case 7: n=5, c=3 115 cfu/g Case 8: n=5, c=1 63 cfu/g Case 9: n=10, c=1 35 cfu/g With: m=1000 cfu/g, M=10 000 cfu/g, and standard deviation s.d.=0.8 Client - meeting - - 33
ICMSF Two-Class Plans: Mean CFU/G Rejected With 95% Probability Case 10: n=5, c=0 1 cfu / 32g Case 11: n=10, c=0 1 cfu / 83g Case 12: n=20, c=0 1 cfu / 185g Case 13: n=15, c=0 1 cfu / 135g Case 14: n=30, c=0 1 cfu / 278g Case 15: n=60, c=0 1 cfu / 526g With: m = 0 cfu / 25g, and standard deviation s.d. = 0.8 Client - meeting - - 34
ICMSF Two-Class Plans: Mean CFU/G Accepted With 95% Probability Case 10: n=5, c=0 1 cfu / 1515g Case 11: n=10, c=0 1 cfu / 2439g Case 12: n=20, c=0 1 cfu / 3846g Case 13: n=15, c=0 1 cfu / 3125g Case 14: n=30, c=0 1 cfu / 4761g Case 15: n=60, c=0 1 cfu / 7142g With: m = 0 cfu / 25g, and standard deviation s.d. = 0.8 Client - meeting - - 35
Variables Sampling Plans: Design Variables Sampling Plans evaluate non-grouped quantitative analytical results (i.e. log cfu/g). Specifications: Number of sample units: n Acceptable quality limit: V (in log cfu/g) Maximum proportion above V: p 0 Probability to accept a non-conforming lot: α Client - meeting - - 36
0.6 Variables Sampling Plan (s.d.=0.8) Mean 0 = V - u 1-p0 * s.d. 0.5 V: acceptable quality limit Probability Density 0.4 0.3 0.2 0.1 p 0 : acceptable proportion defective 0.0 0 1 2 3 4 5 6 Log cfu/g
Variables Sampling Plans: Decision Rule Calculate the average of n sample results and a confidence interval to estimate true lot mean. For acceptable lots the estimate is expected to be lower than mean 0 corresponding to V and p 0. Decision rule: Rejection of lot, if average + k * s.d. > V Acceptance of lot, if average + k * s.d. <= V k is depending on n, p 0, and α. Client - meeting - - 38
Sampling Plans and FSOs: Example Food Safety Objective: 100 Listeria monocytogenes per g in cold-smoked salmon at time of consumption Cases and sampling plans: No inactivation, growth assumed not to occur case 11: n = 10 samples with c = 0 and m = 100 cfu/g No inactivation, growth assumed to occur case 12: n = 20 samples with c = 0 and m = 100 cfu/g Client - meeting - - 39 ICMSF (1994) Int. J. Food Microbiol. 22:89-96 CODEX ALIMENTARIUS COMMISSION, August 2001, CX/FH 01/6 ANNEX 3.2
Performance of Sampling Plans for Listeria Monocytogenes Assumption: standard deviation s.d. = 0.8 Case 11: n = 10 samples with c = 0 and m = 100 cfu/g Mean cfu/g rejected with 95% probability: 30 cfu/g Mean cfu/g accepted with 95% probability: 1 cfu/g Case 12: n = 20 samples with c = 0 and m = 100 cfu/g Mean cfu/g rejected with 95% probability: 13 cfu/g Mean cfu/g accepted with 95% probability: 0.5 cfu/g Client - meeting - - 40
Probability Density 0.6 0.5 0.4 0.3 0.2 0.1 Lot quality rejected with 95% probability (s.d.=0.8) Case 11: 2-class plan, n=10, c=0, m=100 cfu/g m = FSO 26% 0.0-1 0 1 2 3 4 Log cfu/g Probability Density 0.6 0.5 0.4 0.3 0.2 0.1 Case 12: 2-class plan, n=20, c=0, m=100 cfu/g m = FSO 14% 0.0-1 0 1 2 3 4 Log cfu/g
Variables Sampling Plans and FSOs Example: Listeria monocytogenes in cold-smoked salmon; FSO = 100 cfu/g at time of consumption Variables Plan: Number of sample units: n = 10 Acceptable quality limit: V = FSO = 100 cfu/g Maximum proportion above V: p 0 = 5% Probability to reject a non-conforming lot: 1-α = 95% Approach is based on the assumption that L. m. number in sample can be reliably quantified! Client - meeting - - 42
Summary (1) Though microbiological sampling plans are widely used and adopted (e.g. microbiological criteria), their implications are not fully understood: Hypothesis tested Reliability of decision Performance: steepness of OC curve / stringency, quality rejected / accepted with given, say 95%, probability Justification of number of samples, n Client - meeting - - 43
Summary (2) Based on a quantitative description of lot quality in terms of a frequency distribution for sample results even attributes plans can be used to assess mean microbiological concentrations in lots of foods. Performance depends on: Sampling plan specifications (attributes plans: n and c) Microbial limits set (attributes plans: m and M) Validity of assumptions for the frequency distribution (standard deviation s.d.) Reliability and precision of the analytical technique Client - meeting - - 44
Summary (3) Firm understanding of sampling plan performance and its statistical background is required to use them effectively, for instance, to design sampling plans that are in agreement with given Food Safety Objectives. www.icmsf.org Client - meeting - - 45