Feedback calibration training improves whisky sensory profiling AB.09 Dr Chris Findlay, Compusense, Canada Sensory Descriptive Analysis Feedback Calibration Method
Overview Sensory Descriptive Analysis Whisky Panel Y2K Feedback Calibration 2006 Whisky Panel 2012 Comparison of progress over 12 years
Sensory Descriptive Analysis 1. Screening and Selection of the Panel 2. Training and Ballot Creation Panelists are trained using reference standards All descriptive attributes are rated on unstructured 100-point line scales. Training sessions are 2 hours long and 20 sessions are typically used to train a new group of screened panelists 3. Data Collection and Analysis Ten to twelve trained panelists performed three replicate evaluations of around 10 products using a balanced William s design, for a total of at least 30 evaluations per product. Means are calculated and analysis of variance is used to determine significance Results are presented in various plots Lawless and Heymann 1998
Y2K Whisky Sensory Descriptive Analysis
Example Products Product Abbreviated Product Name Johnnie Walker Black Label Blended Scotch Whisky JWB Maker s Mark Kentucky Straight Bourbon Whisky MM Jack Daniel s Tennessee Sour Mash Whiskey Labrot & Graham Woodford Reserve - Kentucky Straight Bourbon Whiskey Four Roses Kentucky Straight Bourbon Whiskey Jameson Triple Distilled Irish Whiskey JD WR 4R JAM
Example Whisky
Sensory Descriptive Attributes and Definitions Table 1. Aroma Spikes for the Whisky Panel, July 2000 Descriptor Concentration Stock solution Recipe (into 300ml of BAS) Smokey-wood Real Hickory Smoke Flavour 140μl of flavour Smokey-charcoal 80/20 Smoke Flavour 15μl of flavour Phenolic o-cresol 0.025ppm 2.3μl of o-cresol into 100ml ethanol (40%) 0.324ml of stock solution Penolic Guicacol 0.5ppm 69μl Guiacol into 100ml of ethanol/ vodka (40%) 0.216ml of stock solution Fruity Isoamyl acetate 10.9ppm 100μl of Isoamyl acetate into 100ml of water 3.273ml of stock solution Floral Geraniol 31.2ppm 780μl of Geraniol into 100ml of water (total) 1.2ml of stock solution Sweet-caramel Caramel Natural Flavour 1.4ml of flavour (Metarom) Sweet-vanilla Vanilla Extract (Club House) 0.9ml of extract Sourness Acetic Acid 1000ppm 6ml of white vinegar (5%) Solvent Ethyl Acetate 557ppm 0.120ml of the Ethyl Acetate Woody Cedar Extract 0.6ml of extract Musty 2,4,6-Trichloroanisole 4.6ppm 115mg Trichloroanisole into 100ml ethanol 1.2ml of stock solution Spicy Eugenol 2.8ppm 78μl of Eugenol into 100ml of ethanol (40%) 1ml of stock solution Green/ Grassy Cis-3-Hexen-1-ol 100ppm 30μl of Cis-3-Hexenol Malty Massive Irish beer Nutty Peanut Extract (Metarom) 9667ppm 2.9ml of Peanut Extract Buttery Diacetyl 2.24ppm 20μl of Diacetyl into 100ml (25ml vodka + 75ml water) 3.2ml of stock solution Sulfur Canned cooked corn Based upon personal communications with John Piggott, 2000. Lee, Paterson, Piggott and Richardson, 2001
Sample Preparation All samples are evaluated at 20% ABV. Blends are prepared each day using room temperature distilled water. Samples are poured approximately 1 hour before testing; 25 ml of each sample is poured into a 6 oz blue glass copita and covered immediately with a lid. Samples are labeled with three-digit blinding codes and served at room temperature.
Sample Evaluation At the start of testing, panelists receive a warm-up sample for calibration purposes. For each evaluation, the panelists receive one lidded 6 oz blue glass copita containing 25 ml of product. Panelists evaluate each sample monadically for aroma and flavor attributes. Panelists expectorate the samples for all attributes. Panelists cleanse their palates between samples with room temperature distilled water and unsalted soda crackers. Panelists are also provided covered mugs of steaming water with which to clear their nasal passages. Panelists get a five minute break between samples. The test is run in individual computerized booths under white lights.
Means and ANOVA Results of Example Whisky p-value LSD Value JW MM JD WR FR JAM Medicinal Aroma 0.00 2.0 14.0 9.7 9.6 10.7 9.4 8.7 Phenolic Aroma 0.00 2.1 15.7 12.0 12.8 11.7 11.7 10.7 Tobacco Aroma 0.69 2.0 8.5 7.2 7.8 7.3 7.9 7.3 Cooked Cereal Aroma 0.75 1.2 5.9 5.5 5.3 6.1 5.2 5.3 Malty Aroma 0.05 1.8 7.7 7.5 8.0 9.0 7.2 7.2 Grassy Aroma 0.02 1.4 9.0 9.2 8.8 9.2 9.0 9.9 Floral Aroma 0.02 1.9 6.4 8.4 8.5 9.2 9.8 9.3 Fruity Aroma 0.01 2.2 11.8 14.6 15.5 16.3 13.5 15.2 Solvent Aroma 0.20 1.5 7.6 7.1 7.5 7.3 7.1 6.8 Vanilla Aroma 0.01 2.1 7.2 10.3 9.4 11.1 10.2 8.9 Oak Aroma 0.87 2.3 8.7 8.1 8.1 7.4 8.0 7.6 Cedar Aroma 0.27 1.8 6.4 7.5 6.7 5.8 6.4 5.5 Buttery Aroma 0.15 1.5 2.9 3.7 4.0 4.7 3.5 3.8 Nutty Aroma 0.35 1.6 8.1 9.3 9.3 10.5 9.4 9.0 Medicinal Flavor 0.00 2.2 15.3 10.4 9.2 10.7 9.9 9.4 Phenolic Flavor 0.00 2.1 17.5 12.5 13.7 13.1 13.0 11.9 Tobacco Flavor 0.19 2.0 9.8 7.9 8.0 7.5 7.9 8.0 Cooked Cereal Flavor 0.81 1.1 5.6 5.6 6.0 6.3 5.7 5.6 Malty Flavor 0.64 1.8 7.7 8.9 8.9 9.3 8.3 8.9 Grassy Flavor 0.00 1.3 8.2 9.4 9.4 10.0 9.3 9.0 Floral Flavor 0.00 1.9 5.9 9.0 7.9 9.1 10.2 8.0 Fruity Flavor 0.01 2.0 11.1 13.6 14.4 14.9 13.5 14.8 Solvent Flavor 0.07 1.3 8.0 7.3 6.7 7.8 6.5 6.4 Vanilla Flavor 0.01 1.7 7.5 9.9 9.8 10.7 9.8 10.5 Oak Flavor 0.65 2.1 10.3 9.1 8.6 9.2 8.3 8.9 Cedar Flavor 0.75 1.9 6.5 7.7 7.2 7.3 7.1 5.8 Buttery Flavor 0.53 1.4 2.6 3.7 3.8 4.1 3.8 3.3 Nutty Flavor 0.94 1.8 8.4 9.2 9.1 9.6 9.3 9.5 Sweetness 0.03 1.6 17.0 18.4 18.0 19.1 19.2 19.4 Sourness 0.12 1.1 9.6 9.7 9.0 9.9 9.2 8.3 Means are based upon 3 replications per product. All responses were collected on 100-point unstructured line scales.
Whisky Sensory Map 2.02 Sweetness Vanilla Aroma Malty Aroma WR Phenolic Flavour Medicinal Flavour JWB Medicinal_Aroma Malty_Aroma Floral_Flavour Vanilla_Flavour Fruity_Flavour -2.02 Floral_Flavor 2.02 JD Grassy_Aroma MM 4R Fruity_Aroma JAM Grassy -2.02
Improving Sensory Descriptive Analysis
Descriptive Analysis as an Analytical Tool Descriptive Analysis is an analytical method The quality of the results of DA depends on both accuracy & precision. Are the results correct and repeatable? Results depend on training assessors Experts still require panel screening and training Two basic questions about DA Can we get the panel right from the start? What is the best possible panel?
The Sensory Order of Operations What is an order of operations? In mathematics an equation is calculated using BEDMAS (Brackets, Exponents, Divide, Multiply, Add & Subtract) The Sensory Order of Operations 1. Identify the attribute (attribute standard) 2. Rank its intensity 3. Scale the intensity (calibration standard)
Attribute Classification System Attribute Identity Specific Standard Group of Attributes Verbal or Evocative A primary reference exists that defines the attribute completely A few good examples provide the definition for several related attributes No specific reference can be used, but the concept can be communicated Sugar or Salt Fruit or floral Barnyard or Diesel Scaling Difficulty Full Scaling Rankable Off/On The attribute can be measured across the full range with precision of <10% of the scale range Across the range for the product, may be detected at 2,3 or 4 levels At the level in the product it is either absent or present, but does not lend itself to scaling Sucrose in juices Bitterness in black coffee Metallic in beverages
Providing Immediate Feedback Assists Learning
Implicit Information Integration Category Learning Feedback administration significantly influences category learning Observational feedback severely impairs learning vs. feedback learning Giving selectively positive or negative feedback will impair learning vs. full feedback Delaying feedback more than a few seconds severely impairs learning Ashby, G.F. and O Brien, J.B. (2007)
The Ballot
The Response
Immediate Feedback
Setting Meaningful Attribute Targets The Feedback Calibration Method (FCM ) is based upon providing panelists with true information at the time they evaluate the attribute. If feedback is either untrue or trivial, the panelist will become confused and the desired learning will not take place. Meaningful targets for feedback depend upon; an understanding of the role of the stimulus-response curve of any attribute the effect of context the intensity of the attribute for the product category
Conclusions on Feedback Calibration Immediate feedback provides panelists with strong individualized method of learning attributes and scaling. It also facilitates integration of new panelists. Panels can develop and refine their own targets. Calibration can be achieved through specific lexicons with reproducible attribute standards. Training times can be cut in half with no penalty in panel performance. Castura, Findlay and Lesschaeve; 2005, Findlay, Castura and Lesschaeve, 2007; Findlay, Castura, Schlich and Lesschaeve, 2006
2012 Whisky Sensory Descriptive Analysis Using Feedback Calibration
FCM Sensory Descriptive Analysis 1. Recruit and screen panelists 2. Identify the key sensory attributes of the product range 3. Apply a sensory order of operations approach to attribute development and classification 4. Develop meaningful feedback targets for individualized training 5. Use Feedback Calibration sessions to train the panel 6. Set proficiency targets for panelists 7. Assess the proficiency of the panelists and panel 8. Finalize the ballot 9. Measure the attribute responses for the products 10. Analyze and interpret product results
Descriptive Analysis Methodology Panelists were trained using reference standards and were calibrated for the descriptive attributes using the Compusense Feedback Calibration Method (FCM ). All descriptive attributes were rated on unstructured 100-point line scales. Testing was conducted at the Compusense Research Facility in Guelph, July 2012. Eleven trained panelists performed three replicate evaluations of the 10 products using a balanced William s design, for a total of 33 evaluations per product.
Sample Preparation All samples were evaluated at 20% ABV. Blends were prepared each day using room temperature distilled water. Samples were poured approximately 1 hour before testing; 25 ml of each sample was poured into a 6 oz blue glass copita and covered immediately with a lid. Samples were labeled with three-digit blinding codes and served at room temperature.
Training
Sensory Descriptive Attributes and Definitions Table 2. Aroma spikes for Whisky Panel, July 2012 Method of Evaluation: Lift the sample to the nose and remove the lid. Sniff rapidly and deeply 3 times. Repeat as needed. Attribute Definition Left Anchor Right Anchor Reference Standard Medicinal Band-aid, antiseptic None Very Strong Aroxa capsule: 4-ethyl phenol Phenolic Phenolic (peaty) None Very Strong Aroxa capsule: Guaiacol Tobacco Tobacco, hay, dry grass None Very Strong Cooked Cereal Cooked cereal, cooked grains None Very Strong Aroxa capsule: Beta-cyclocitral Aroxa capsule: 3-ethyl pyridine Aroxa capsule: 2-acetyl pyridine Aroxa capsule: Methional Aroxa capsule: Isobutyraldehyde Malty Malt, malted barley None Very Strong Happy Home Malt Syrup Grassy Fresh cut grass, green leaves, cuttings, green beans, green banana peel Floral Roses, violets, lilacs None Very Strong Fruity Banana, apple, peach, pear, cherry, black currant, prunes, plums, pineapple, orange, lemon, lime None Very Strong Aroxa capsule: Cis-3-hexenol None Very Strong Aroxa capsule: Beta-damascenone Aroxa capsule: Beta-ionone Aroxa capsule: Isoamyl acetate Aroxa capsule: Ethyl hexanoate Solvent Nail polish remover, paint thinner None Very Strong Aroxa capsule: Ethyl acetate Vanilla Vanilla, vanillin None Very Strong Aroxa capsule: Vanillin Oak Oak, sawdust, papery None Very Strong Cedar Cedar None Very Strong Cedar shavings Aroxa capsule: Trans-2-nonenal Oak shavings Buttery Butter, diacetyl None Very Strong Aroxa capsule: Diacetyl Nutty Hazelnut None Very Strong Aroxa capsule: 5-methyl-2-hept-4-one http://www.aroxa.com/
Sample Evaluation At the start of testing, panelists received a warm-up sample for calibration purposes. For each evaluation, the panelists received one lidded 6 oz blue glass copita containing 25 ml of product. Panelists evaluated each sample monadically for aroma and flavor attributes. Panelists expectorated the samples for all attributes. Panelists cleansed their palates between samples with room temperature distilled water and unsalted crackers. Panelists were also provided covered mugs of steaming water with which to clear their nasal passages. Panelists were given a five minute break between samples. The test was completed in individual computerized booths under white lights.
Testing
Analysis The data was collected using Compusense at-hand software. All raw data had been reviewed. Mean scores and Fisher s LSD analysis (at 5%) were calculated in Senstools Version 3.3.1 and used to identify significant differences between the products. Generalized Procrustes Analysis (GPA) was performed in Senstools Version 3.3.1.
JAM 0.70 Whisky Sensory Map Dimensions 1 and 2 95% Confidence Intervals for All Whisky WR JD FR MM JWB -0.50 1.50-0.70
0.70 Whisky Sensory Map Dimensions 3 and 4 95% Confidence Intervals for All Whisky JAM JWB -0.50 JD 0.60 WR MM -0.70
Comparing Y2K to 2012 Table 3. Least significant difference and training time of panels after introduction of FCM training. Attributes selected are matched between both panels ATTRIBUTE Significant at p<0.05 Least Significant Difference Year 2000 Year 2012 Fruity aroma 3.2 1.7 Floral aroma 3.3 1.7 Phenolic aroma 5.4 1.8 Smoky aroma 5.1 1.5 Sweet aroma 3.1 1.7 Phenolic flavour 3.8 1.6 Smoky flavour 3.8 1.5 On unstructured line scale anchored at 0 and 100 3.96 1.64 Whisky Training Time (h) 12 6
The Effectiveness of FCM Training 1990 s 40 hours 2000 s 20 hours 2010 s FCM Introduced in 2006 6 hours
Calibrated Descriptive Analysis When using FCM training Analytical sensory profiles of products are both more accurate and precise. A library of the sensory properties of products can be created Competitor profiles are meaningful Reliable multi-attribute measures of sensory shelf life can be obtained.
References Ashby, G.F. and O Brien, J.B. (2007). The effects of positive versus negative feedback on information-integration category learning. Journal of Perception & Psychophysics 69(6): 865-878 Castura, J.C., Findlay, C.J., Lesschaeve, I. (2005) Monitoring calibration of descriptive sensory panels using distance from target measurements. Food Quality and Preference 16(8): 682-690. Findlay, C.J., Castura, J.C., Lesschaeve, I. (2007). Feedback Calibration: a training method for descriptive panels. Food Quality and Preference 18(2): 321-328 Findlay, C.J., Castura, J.C. Schlich, P., Lesschaeve, I. (2006). Use of feedback calibration to reduce the training time for wine panels. Food Quality and Preference 17(3-4): 266-276. Lawless, H. T., & Heymann, H. Sensory evaluation of food, 1998. Chapman Hall, New York. Lee, K.-Y. M., Paterson, A., Piggott, J. R. and Richardson, G. D. (2001), Origins of Flavour in Whiskies and a Revised Flavour Wheel: a Review. Jnl Institute Brewing, 107: 287 313.
Thank You compusense.com +1 519 836 9993 Worldwide 1 800 367 6666 Toll free in North America info@compusense.com