Scoring. AIB Consolidated Standards for Inspection. The scoring described in this booklet supersedes. all previous scoring methods used in all AIB

Transcription

1 AIB Consolidated Standards for Inspection Scoring The scoring described in this booklet supersedes all previous scoring methods used in all AIB Standards pictured above in all languages effective January 1, September 2010

2 Introduction AIB International introduced the revised versions of its 11 Consolidated Standards in This established a modern platform to support AIB s Inspection Program, and can be built upon to deliver continuous improvements in process and content that the evolving world food industry requires. The scoring method described in this booklet, which becomes effective January 1, 2011, is one such an improvement and was driven by independent research of customer requirements that showed auditor consistency in scoring and report writing was critical to confidence in third party audit programs. To this end, AIB developed a method to build greater consistency and reduce subjectivity in its scoring system, no matter the location or the auditor. The revised scoring method assigns point values for the severity and frequency of observations within each of the five inspection Categories. The score earned on an inspection is the true measure of achievement; however, AIB has for many years also awarded a laudatory term Superior or Excellent that described the score. After much deliberation and customer consultation, AIB will continue to use laudatory terms, but will change the method by which they are awarded. Only those facilities whose score falls within the top 25% of all scores will receive a Recognition of High Achievement Superior award effective January 1, A Recognition of Accomplishment document will be provided to all other facilities scoring between a passing score of 700 and the applicable Superior score. There will be two groups determined by type of facility inspected. A statistical analysis will be conducted annually to establish the Superior baseline score for the following year. For 2011, the two rates will be 915 and 935. The two groups are: GROUP A: 915/1000 GROUP B: 935/1000 Prerequisite and Food Safety Programs Fresh Produce and Fruit Packinghouses Beverage Facilities Fresh Cut Produce Food Distribution Centers Food Contact Packaging Manufacturing Facilities Nonfood Contact Packaging Manufacturing Facilities Agricultural Crops Dairy Facilities Retail Facilities Grain Handling AIB International believes these changes will reinforce the rigor of its Program and increase the confidence placed in it by thousands of customers and facilities worldwide. 2

3 The Inspection Scoring At-A-Glance 1The Inspector observes operations on the facility floor to determine the effectiveness of programs, and reviews written programs to determine if there are gaps. 2The Inspector determines the most significant food safety observation(s) in each of the first 4 Categories. A risk assessment of observations is made for each Category: Operational Methods and Personnel Practices Maintenance for Food Safety Cleaning Practices Integrated Pest Management Adequacy of Prerequisite and Food Safety Programs Determining Risk and Assigning Category Scores The Inspector then decides the severity of the most significant observation in each Category. Risk Assessment Description Score Range No Issues Noted No identified risk. 200 Minor Issues Noted No potential for contamination Improvement Needed A potential hazard, partial program omission, or food safety finding that is inconsistent with the Standards. If this hazard, omission, or finding is not corrected it could lead to program failure. Serious A significant food safety risk or risk of program failure. Unsatisfactory An imminent food safety hazard, program failure, or departure from the Good Manufacturing Practices. 135 Scores are based on the definitions shown above. If the most significant observation in the Category is severe, the category will be assigned a score at the bottom of the range. If the most significant observation is not as severe, it will be assigned a score at the top of the range. Multiple observations in a Category will reduce the score in 5 point increments for each additional observation. The score will not drop below the bottom of the Category score range. Evaluating Adequacy 3Before the Inspector can assign the Total Score, the Adequacy of the Food Safety Programs must be assessed. Assigning Adequacy scores involves comparing written food safety programs to the results they produce in the facility. It is impossible to receive a perfect Adequacy score unless the written programs result in perfect scores for each Category evaluated in the facility. 4Total Score After the Adequacy score is determined, it is added to the scores from the first four Categories to produce the total score. Recognition and Laudatory Terms 5Data from facilities that receive AIB inspections will be collected from October 1 through September 30 each year and a statistical analysis of the data will determine the total score range for the top 25 percent of scores. This range will be the criteria for Recognition of High Achievement-Superior for the following calendar year. Recognition of High Achievement Superior will be awarded to facilities scoring within the statistically calculated total score range. A facility cannot receive a Recognition of High Achievement Superior if there is a category score below 160, irrespective of the total score. Recognition of Accomplishment will be awarded to all facilities scoring between 700 and the required number of points to achieve a Recognition of High Achievement Superior. A facility cannot receive a Recognition of Accomplishment if there is a category score below 140, irrespective of the total score.

4 1The Inspection weakest link. Like a chain, the strength of a Food Safety Program depends on its To assess the food safety risks in a facility, an AIB Inspector conducts a thorough and fair physical inspection and concludes with a review of written programs. The Inspector notes observations based on the five Categories of The AIB International Consolidated Standards for Inspection: 1. Operational Methods and Personnel Practices 2. Maintenance for Food Safety 3. Cleaning Practices 4. Integrated Pest Management 5. Adequacy of Prerequisite and Food Safety Programs 2 Determining Risk and Assigning Category Scores The AIB Inspector will then assign a level of risk and a Category Score to the five Categories shown above. Use Table 1 as a guide. Table 1 Risk Assessment Risk Assessment Description Category Score Range No Issues Observed No identified risk. 200 Minor Issues Noted No potential for contamination Improvement Needed A potential hazard, partial program omission, or food safety finding that is inconsistent with the standards. If this hazard, omission, or finding is not corrected, it could lead to a program failure. Serious A significant food safety risk or risk of program failure Unsatisfactory An imminent food safety hazard, program failure, or departure from the Good 135 Manufacturing Practices. 3

5 The Inspector uses a three-step process to assess risk. The inspector: 1. Determines the most significant observation(s) in a Category and assigns a score range. 2. Determines the severity of the most significant observation(s) and decides whether the initial score should be at the top or bottom of the score range assigned. 3. Lowers the initial score in 5 point increments for each additional observation if the assigned score is at the top of the score range. The Inspector determines the most significant risk in a category and assigns it to the Minor Issues Noted score range. The most significant observation is not as severe, and so would be scored at the top of the Category Score range. The most significant observation is severe, and so would be scored at the bottom of the Category Score Range and cannot be further reduced. Highest Category Score Multiple occurrences of separate observations lower the score in increments of 5 until the lowest Category Score is reached. Lowest Category Score Figure 1 Example of Risk and Category Score Determination in the Minor Issues Noted Range. 4

6 Here are some guidelines: The initial score for a Category is always either at the top or the bottom of the range. A Category score can be adjusted from the top of the range, but will never go below the bottom of the range. All critical or minor findings associated with a single Standard of a Category would be grouped together as a single observation. For example, any findings (single or multiple) noted under the following Standard and related requirements would only be counted as one observation: 1.5 Pallets Findings assigned to several Standards within a Category would be considered distinct and separate observations. For example, any findings (single or multiple) noted for each of the following Standards would be counted as 2 observations: 1.1 Rejection of Shipments 1.3 Storage Conditions A single observation in a Category may be severe enough to require the category to be scored at the bottom of the score range. Severity can be due to a single significant observation, or it can be due to multiple findings establishing a pattern within a single observation. Observations of Minor Requirements are always assessed in the Minor Issues Noted score range. If the initial score is at the top of the assigned score range, each additional observation lowers the scores in 5 point increments. Possible scores are listed in Table 2. Table 2 Lowering an Initial Category Score for Multiple Observations # of Observations Category Scores for All Risk Assessments Minor Issues Improvement Noted Needed Serious Unsatisfactory * * Will be lowered an additional 5 points for additional observations. 5

7 3Evaluating the Adequacy of the Food Safety Program Evaluation of the written programs is not limited to determining if a written program and its records are in place and current. What the AIB Inspector sees in the facility determines whether or not the written Food Safety Programs actually work. A facility cannot have perfect programs if food safety observations are noted during the inspection. The Inspector reviews the observations in the facility against the written programs to determine where the gaps in the program exist and what should be done to alleviate these conditions. The score for the Adequacy Category is determined using the same method that is used for calculating the other four Category Scores. The Adequacy score, however, is also guided by four additional rules. Rules to Determine the Adequacy Score Rule 1 The Adequacy Score cannot be the highest score. How can the programs that manage outcomes in the other categories be scored higher than the categories themselves? Rule 2 The Adequacy score can be no more than one Risk Assessment Category higher than the Category with the worst finding. In other words, if the worst Risk Assessment is Serious, how could the Adequacy section be said to have only minor issues with its operation? Again, this relates to how well the program functions in a facility. See Table 3. Worst Risk Assessment Table 3 Maximum Adequacy Score Range Based on Rule 2 Score Range for Worst Risk Assessment Maximum Adequacy Score Range Minor Issues Noted Improvement Needed Serious Unsatisfactory

8 Rule 3 If the worst score is at the bottom of the score range, the Adequacy score can be no higher than the bottom category score, one level above. If observations require the score to be at the bottom of the category score range, this indicates that the related program is not effective. Table 4 Maximum Adequacy Score Based on Rule 3 Worst Risk Assessment Rule 4 A 200 may only be assigned for Adequacy if the other four category scores are all assigned a 200; i.e., the only way it can be said that the programs are working perfectly is if there are no observations to indicate otherwise. 4Total Score Score of Worst Risk Assessment at Lowest Number in the Score Range The total score is the sum of the points assigned to each Category: Operational Methods and Personnel Practices, Maintenance for Food Safety, Cleaning Practices, and Integrated Pest Management, but is not complete until aligned with the Adequacy of Prerequisite and Food Safety Programs because written programs drive the results from the other four categories. 5Recognition and Laudatory Terms Recognition is based on the total score assigned to the facility. Total scores will be collected from all facilities evaluated against the AIB Consolidated Standards for Inspection. Facilities will be according to the specific standard. Data will be collected from October 1 through September 30 of each year. A statistical analysis of the data will determine the total score range for the top 25 percent of scores. This range will be the criteria for Recognition of High Achievement Superior for the following calendar year. The awarded score range will change every year depending on the past year s performance of AIB International s varied client base. 7 Maximum Adequacy Score Minor Issues Noted Improvement Needed Serious Unsatisfactory

9 Recognition of High Achievement Superior will be awarded to facilities scoring within a statistically calculated total score range. A facility cannot receive a Recognition of High Achievement Superior if there is a category score below 160, irrespective of total score. If a facility scores within the Recognition of High Achievement Superior range, but also has a category score below 160, the facility will receive a Recognition of Accomplishment. Recognition of Accomplishment will be awarded to all facilities scoring between 700 and the required number of points to achieve a Recognition of High Achievement Superior. A facility cannot receive a Recognition of Accomplishment if there is a category score below 140, irrespective of the total score. The AIB International Recognition Document: Recognizes that on the day of the inspection, the facility achieved a certain score according to the AIB International Consolidated Standards for Inspection. Is not a certificate of compliance (such as an ISO certificate). Does not have a specific expiration date. Is labeled as announced or unannounced. Defines which areas of the facility were included in the inspection. 8

10 Sample Scoring with Explanations Category Score Range Category Operational Methods and Personnel Practices Maintenance for Food Safety Cleaning Practices Integrated Pest Management Adequacy of Prerequisite and Food Safety Programs # Minor Issues Noted Observations # Improvement Needed Observations # Serious Observations # Unsatisfactory Observations Category Score Total Score 815 Recognition of Recognition Accomplishment The Inspector noted 6 observations at the lowest risk of severity, but the Category Score does not go lower than the lowest possible score for the Minor Issues Noted Category (180). The severity of the single observation was significant so the score at the bottom of the score range (160) is assigned. Three observations are documented. There were actually 5 findings, but 3 of the findings were related to the same requirement in the Standard and were therefore grouped together as a single observation. The Serious observations that posed the most potential for contamination were at the lowest severity of risk, so the Category score begins with the first observation at 155. There were 2 additional observations, so the score was lowered by 5 points for each to 145. The Adequacy score is determined using the most constraining rules that apply: The observation with the most significant risk is in the Improvement Needed category so the score should fall in the range. The most significant observation is not severe, so the initial score is 175. There are three separate observations, so 5 points are deducted for each additional observation beyond the first (175 to 170 to 165). Rule 1: The highest score in the other 4 categories is 180, but that is outside the range so Rule 1 does not apply. Rule 2: The lowest score in the other 4 categories is 145, so the Adequacy score can be no higher than the range. Rule 3: The lowest category score (145) is not at the bottom of the range, so Rule 3 does not apply. Rule 4: The other 4 categories are not assigned a 200, so rule 4 does not apply.

11 Glossary of Terms Category The AIB International Consolidated Standards for Inspection are divided into five categories: Operational Methods and Personnel Practices, Maintenance for Food Safety, Cleaning Practices, Integrated Pest Management, and Adequacy of Prerequisite and Food Safety Programs. Category Score Range The numerical range within which a category will be scored. The five category score ranges align with the five risk assessment categories and they are: No Issues Observed (200), Minor Issues Noted ( ), Improvement Needed ( ), Serious ( ), or Unsatisfactory ( 135). Category Score The numerical score for each of the following categories: Operational Methods and Personnel Practices, Maintenance for Food Safety, Cleaning Practices, Integrated Pest Management, and Adequacy of Prerequisite and Food Safety Programs. Findings Notes made by an inspector that are indexed to a Standard or related requirement. There may be multiple findings in an observation. Initial Category Score This is the first score assigned based on severity. The total number of single and separate observations may bring the initial category score down. Multiple Observations Findings (single or multiple) noted under more than one Standard and related requirements. For example: All findings noted in 1.1 Rejection of Shipments and 1.3 Storage Conditions will be counted as 2 observations. An observation will be counted for each standard involved. Program A collection of documentation related to the management of an element in a facility that impacts food safety. Recognition The awarding of recognition based on total score. The recognition levels are: Recognition of High Achievement Superior and Recognition of Accomplishment. Previously, these levels of recognition were known as ratings and had the values of Superior, Excellent, Pass, Satisfactory and Unsatisfactory. Risk Assessment The categorization of observations in a facility into one of five categories: No Issues Observed, Minor Issues Noted, Improvement Needed, Serious, or Unsatisfactory. Severity The level of risk within a risk assessment category (i.e., how severe is an observation within the risk category of Improvement Needed?). Single Observation Findings (single or multiple) noted under a single Standard and related requirements. Example: All findings noted in Standard 1.5 Pallets or in any of its requirements ( , , ) will be evaluated as one observation. Total Score The total of all category scores. 10

12 Automatic Assessment of Unsatisfactory The conditions for an automatic unsatisfactory remain as stated in the current standard that applies. Call AIB International World Headquarters ( ) for international contact information 11