How reliable are the consumers? Comparison of sensory profiles from consumers and experts

How reliable are the consumers? Comparison of sensory profiles from consumers and experts WORCH Thierry (1) LÊ Sébastien (2) PUNTER Pieter (1) (1) OPP Product Research (2) AgroCampus Ouest mailto: thierry@opp.nl Project 8013 July 2008 Senior project manager Pieter Punter Project manager Thierry Worch

introduction in the sensory theory: experts panels are used for the products description consumers should only be used for the hedonic task they lack two essentials qualities for profiling (consensus and reproducibility) there are strong halo effects (Earthy, MacFie & Hedderley, 1997) in the sensory practice: consumers are sometimes used for both tasks it has been proven that consumers description show the required qualities (consensus and reproducibility) (Husson, Le Dien, Pagès, 2001) 8013 3

problematic How reliable are the consumers? 8013 4

presentation of the studies products: twelve luxurious women perfumes (Gazano, Ballay, Eladan & Sieffermann, 2005) Angel (Eau de Parfum) Cinéma (Eau de Parfum) Pleasures (Eau de Parfum) Aromatics Elixir (Eau de Parfum) Lolita Lempicka (Eau de Parfum) Chanel N 5 (Eau de Parfum) L Instant (Eau de Parfum) J Adore (Eau de Toilette) J Adore (Eau de Parfum) Pure Poison (Eau de Parfum) Shalimar (Eau de Toilette) Coco Mademoiselle (Eau de Parfum) 8013 5

presentation of the studies expert panel (Agrocampus Rennes) twelve persons (11 students and 1 teacher) from the Chantal Le Cozic school (esthetics and cosmetic school) focus group per group of six, with two animators generation of a list of twelve attributes Vanille, Notes Florales, Agrume, Boisé, Vert, Epicé, Capiteux, Fruité, Fraîcheur Marine, Gourmand, Oriental, Enveloppant training session for the most difficult ones the twelve products were tested two times in two one-hour sessions 8013 6

presentation of the studies consumer panel (OP&P Product Research, Utrecht) 103 naïve Dutch consumers living in the Utrecht area the same twelve perfumes were rated on 21 attributes odour intensity, freshness, jasmine, rose, camomile, fresh lemon, vanilla, mandarin/orange, anis, sweet fruit/melon, honey, caramel, spicy, woody, leather, nutty/almond, musk, animal, earthy, incense, green two products (Shalimar and Pure Poison) were duplicated the fourteen (12+2) products were tasted in two one-hour sessions (seven products in each session, presentation order was balanced) 8013 7

presentation route map the consumer and expert data are compared in three different ways 1.Univariate analysis analyses of variance correlations 2.Multivariate comparison construction of the two products spaces (PCA) comparison of the products spaces through GPA and MFA 3.Confidence ellipses graphical confidence intervals around the products averaged over the two panels graphical confidence intervals around the products defined by the different panels 8013 8

Performance of the two panels (univariate analysis) 8013 9

performance of the panels usually, the expert panels should have many qualities: discrimination: panelists should be able to detect and describe the differences existing between the products reproducibility: panelists should describe the products in the same way, when they are repeated agreement: panelists should give the same description of the products as the rest of the panel it can be measured with the correlations (usually, one panelist is compared to the mean over the rest of the panel) 8013 10

expert panel panel performance discriminate on 11 out of 12 attributes ( Agrume, pvalue=0.08) reproducible for 11 out of 12 attributes ( Notes Florales ) panellist performance (discrimination, reproducibility) panellists 1, 3 and 12 are very good panellists 8, 9 and 10 are not good in discrimination (discriminate the products on less than 6 out of 12 attributes) panellist 9 is also not good in reproducibility (reproducible on only 3 out of 12 attributes. Notes Florales, Agrume and Enveloppant ) 8013 11

expert panel (correlations) distribution of the correlations (correlation between expert i and the mean over the (n-1) others) 6 5 4 3 2 1 0-1,00-0,90-0,80-0,70-0,60-0,50-0,40-0,30-0,20-0,10 0,00 0,10 0,20 0,30 0,40 0,50 0,60 0,70 0,80 0,90 1,00 8013 12

consumer panel discrimination (on the twelve original products) the consumers discriminate the products on all attributes except camomile (pvalue = 0.62) NB: the consumers discriminate on Citrus (pvalue < 0.001) reproducibility (on the two duplicated products only) consumers are reproducible on all attributes except one ( woody ) 8013 13

consumer panel (reproducibility) Pure Poison Pure Poison 2 earthy incense green intensity 70 60 50 40 freshness jasmin rose animal musk 30 20 10 camomille fresh_lemon nutty vanilla leather citrus woody anis spicy sweet_fruit caramel honey 8013 14

consumer panel (reproducibility) Shalimar Shalimar 2 earthy incense green intensity 70 60 50 40 freshness jasmin rose animal musk 30 20 10 camomille fresh_lemon nutty vanilla leather citrus woody anis spicy caramel honey sweet_fruit 8013 15

consumer panel (correlations) distribution of the correlations (correlation between a consumer i and the mean over the (n-1) others) 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 0-1,00-0,90-0,80-0,70-0,60-0,50-0,40-0,30-0,20-0,10 0,00 0,10 0,20 0,30 0,40 0,50 0,60 0,70 0,80 0,90 1,00 8013 16

conclusions on the panel performance expert panel discriminates between the products are reproducible high correlations consumer panel discriminates between the products shows reproducibility s qualities lower but still positive correlations (consumers are untrained) Both panels show the same qualities 8013 17

Products spaces (multivariate analysis) 8013 18

methodology products spaces the products profiles (averaged over the panellists or consumers) are computed. Principal Components Analysis is then run on these product x attribute matrices comparison of the two products spaces (expert and consumer) is a multi-table problem comparison through the Procrustean analysis comparison through Multiple Factor Analysis (MFA) comparison through the confidence ellipses technique 8013 19

expert panel g p 3,5 3,0 2,5 2,0 AromaticsElixir Shalimar Dimension 2 (21.87 %) 1,5 1,0 0,5 0,0-0,5-1,0-1,5-2,0-2,5-3,0-3,5-4,0-4,5 Pleasures JAdore_EP JAdore_ET PurePoison CocoMelle LInstant Cinema Chaneln5 LolitaLempicka Angel Dimension 2 (21.87 %) 1 0 Notes.florales Vert Agrume Fraicheur.marine Fruité Boisé Epicé Oriental Capiteux Enveloppant -4-3 -2-1 0 1 2 3 4 Dimension 1 (64.22 %) Vanille -1 Gourmand -1 0 1 Dimension 1 (64.22 %) 8013 20

expert panel (conclusions) first dimension (64% of the total inertia) shows two clusters: Aromatics Elixir, Shalimar, Angel, Chanel n5 and Lolita Lempicka (characterized by Epice, Oriental, Capiteux, Enveloppant) versus Pleasures, J Adore (EP and ET) (characterized by Fraicheur Marine, Agrume, Notes Florales, Vert, Fruité) second dimension (22% of the total inertia) discriminates between Aromatics Elixir, Shalimar (characterized by Boisé) versus Lolita Lempicka (characterized by Gourmand, Vanille) 8013 21

consumer panel 6,0 5,5 5,0 4,5 4,0 3,5 3,0 LolitaLempicka g Dimension 2 (17.97 %) 2,5 2,0 1,5 1,0 0,5 0,0-0,5-1,0-1,5-2,0-2,5-3,0-3,5-4,0-4,5-5,0-5,5-6,0 JAdore_EP JAdore_ET Pleasures CocoMelle Cinema PurePoison LInstant Chaneln5 AromaticsElixir Shalimar Angel Dimension 2 (17.97 %) 1 0 citrus sweet_fruit freshness green jasmin rose fresh_lemon vanilla honey camomille caramel anis nutty animal musk incense leather woody earthy -6,5-7,0 intensity spicy -5-2.5 0 2.5 5 7.5 Dimension 1 (68.29 %) -1-1 0 1 Dimension 1 (68.29 %) 8013 22

consumer panel (conclusions) first dimension (68% of the total inertia) shows two clusters Angel, Shalimar, Aromatics Elixir (characterized by nutty, animal, musk, incense, leather, woody earthy, spicy) versus J Adore (EP and ET), Pleasures (characterized by citrus, sweet fruit, freshness, green, jasmin, rose, fresh lemon) second dimension (18% of the total inertia) discriminates between Lolita Lempicka (characterized by vanilla, honey, camomile, caramel) versus Aromatics Elixir, Shalimar (characterized by intense, spicy) 8013 23

Multivariate comparison of the two panels (GPA and MFA) 8013 24

expert vs consumer: Procrustes analysis GPA consensus space (coefficient of similarity: 0.93) Dim 2-0.3-0.2-0.1 0.0 0.1 0.2 0.3 0.4 Cinema LInstant JAdore_EP JAdore_ET CocoMelle PurePoison Pleasures LolitaLempicka Chaneln5 Angel Shalimar AromaticsElixir -0.2 0.0 0.2 0.4 8013 25 Dim 1

expert vs consumer: Multiple Factor Analysis MFA partial points representation experts consommateurs (RV coefficient: 0.87) Dim 2 (19.35 %) -2-1 0 1 2 LolitaLempicka Angel Cinema LInstant JAdore_EP JAdore_ETCocoMelle PurePoison Pleasures Chaneln5 Shalimar AromaticsElixir -2-1 0 1 2 3 I di id lf t Dim 1 (64.06 %) 8013 26

expert vs consumer: Multiple Factor Analysis g p MFA variables representation (RV coefficient: 0.87) 1 expert consumer vanilla Gourmand honey Vanille caramel anis camomille Dimension 2 (19.35 %) 0 citrus Fruité sweet_fruit freshness Fraicheur.marine green jasmin rose Agrume Notes.florales Vert fresh_lemon intensity Enveloppant animal musk woody leather incense earthy Capiteux Epicé nutty Oriental spicy Boisé -1-1 0 1 Dimension 1 (64.05 %) 8013 27

Comparison through the confidence ellipses technique (Husson, Lê & Pagès, 2005) (Lê, Pagès & Husson, 2008) 8013 28

confidence ellipses methodology 1.Compute the product profiles (averaged by product over the judges) 2.Create the products space 3.Re-sample by bootstraping new panels 4.For each new panel, compute new products profiles 5.Project as illustrative the products on the original product space 6.Steps 3 to 5 are repeated many times (i.e. 500 times) 7.Confidence ellipses around the products containing 95% of the data are constructed principle if ellipses are superimposed, the products are not significantly different the size of the ellipses is related to the variability existing around the products 8013 29

confidence ellipses Confidence ellipses around the products Dim 2 (19.39%) -1 0 1 2 JAdore_EP JAdore_ET Pleasures Cinema LInstant CocoMelle PurePoison LolitaLempicka Chaneln5 Angel Shalimar AromaticsElixir -3-2 -1 0 1 2 3 4 Dim 1 (64.02%) 8013 30

confidence ellipses mean points some products are not significantly different J Adore ET, J Adore EP and Pleasures Cinema and L Instant some products are clearly significantly different Angel and J Adore (ET or EP) Chanel n 5 and Shalimar 8013 31

confidence ellipses as we have two different panels, we can apply this methodology to both creation of confidence ellipses around each product seen by each panel (24 ellipses are created here) comparison of a given product through the two panels (same colour) comparison of the different products within a panel (same type of line) 8013 32

confidence ellipses Confidence ellipses for the partial points Dim 2 (19.39%) -2-1 0 1 2 cons. expert Cinema LInstant JAdore_EPCocoMelle JAdore_ET PurePoison Pleasures LolitaLempicka Angel Chaneln5 Shalimar AromaticsElixir -4-2 0 2 4 Dim 1 (64.02%) 8013 33

confidence ellipses partial points within a product, the ellipses related to the two panels are always superimposed (no differences between the panels) the sizes of the ellipses are equal the higher amount of consumers compensate the higher variability due to the lack of training for consumers 8013 34

conclusions although consumers don t have the habit to describe perfumes (difficult task), they give the same information as the expert panel (and it s identical to the standard description of the perfumes) they also have the same qualities (discrimination and reproducibility) a difference between consumers and experts panel exists in the variability of the results (more variability for consumers), but this is compensated by the larger size of the panel (here 103 vs 12) with consumers, not only intensity, but also ideal and hedonic questions can be asked in the same time 8013 35

references Earthy P., MacFie H & Hedderlay D. (1997). Effect of question order on sensory perception and preference in central locations. Journal of Sensory Studies, vol.12, p215-237 Gazano G., Ballay S., Eladan N. & Sieffermann J.M. (2005). Flash Profile and flagrance research: using the words of the naïve consumers to better grasp the perfume s universe. In: ESOMAR Fragrance Research Conference, 15-17 May 2005, New York, NY. Husson F., Le Dien S. & Pagès J. (2001). Which value can be granted to sensory profiles give by consumers? Methodology and results. Food Quality and Preference, vol.16, p291-296 Husson F., Lê S.& Pagès J. (2005). Confidence ellipses for the sensory profile obtained by principal component analysis. Food Quality and Preference, vol.16, p245-250 Lê S., Pagès J. & Husson F. (2008). Methodology for the comparison of sensory profiles provided by several panels: Application to a cross cultural study. Food Quality and Preference, vol.19, p179-184 8013 36

thank you special thanks to Melanie COUSIN Maëlle PENVEN Mathilde PHILIPPE Marie TOULARHOAT students from AgroCampus-Rennes, who took care of the whole expert panel data. 8013 37

Thank you for your attention!