Auditory Contributions to Multisensory Product Perception

Transcription

1 A CTA A CUSTICA UNITED WITH A CUSTICA Auditory Contributions to Multisensory Product Perception Charles Spence 1,Massimiliano Zampini 1, 2, 3 1 :Department of Experimental Psychology,University of Oxford, South Parks Road, Oxford, OX13UD, England. charles.spence@psy.ox.ac.uk 2 :Department of Cognitive Sciences and Education, University of Trento, Italy 3 :Research Centre for Bioengineering and Motor Sciences, CeBiSM, University of Brescia, Trento and Verona, Italy Summary The sounds that are elicited when we touch or use many everyday objects typically convey potentially useful information regarding the nature of the stimuli with which we are interacting. Here we review the rapidly-growing literature demonstrating the influence of auditory cues (such as overall sound level and the spectral distribution of the sounds) on multisensory product perception. The results of a number of studies now show that the modulation of the auditory cues elicited by our contact or interaction with diff erent surfaces (such as abrasive sandpapers or even our own skin) and products (including electric toothbrushes, aerosol sprays, food mixers, and cars) can dramatically change the way in which they are perceived, despite the fact that we are often unaware of the influence of such auditory cues on our perception. The auditory cues generated by products can also be modified in order to change people s perception of the quality/effi ciency of those products. The principles of sound design have also been used recently to alter people s perception of a variety of foodstuff s. Findings such as these demonstrate the automatic and obligatory nature of multisensory integration, and show how the cues available in one sensory modality can modulate people s perception of stimuli in other sensory modalities (despite the fact that they may not be aware of the importance of such crossmodal influences). We also highlight evidence showing that auditory cues can influence product perception at a more semantic level, as demonstrated by research on signature sounds and emotional product sound design. PACS no y 1. Introduction To date, the majority of research on human sensation and perception has explored the senses individually. That is, researchers have typically studied single sensory modalities, such as vision, touch, or audition in isolation (e.g., see [1, 2]). Howeverthis approach to research flies in the face of the anatomical, physiological and behavioural facts of human perception (see, for example, the chapters in [3]). For, in everyday life, arange of implicit perceptual and cognitive processes are continually engaged in blending and segregating information from diverse sources via sensory systems adapted to process partially complementary information in order to represent the multisensory objects and events that fill the environments in which we live. In other words, the majority of objects and events in our environment are multisensory, providing information to several senses simultaneously (e.g.,[1, 4, 5]). For example, auditory cues are typically elicited whenever we touch or use everyday objects, and these sounds Received 7June 2005, accepted 10 October often convey potentially useful informational regarding the nature (and functioning) of the objects with which we are interacting (e.g.,[6, 7, 8, 9, 10, 11, 12, 13, 14, 15]). Researchers have shown that even when these auditory cues are presented in isolation, they are often informationallyrich enough to provide suffi cient cues for people to assess the size of objects, and even what material they are made from (e.g.,[16, 17, 18, 19, 20, 21, 12, 22, 23, 24, 25, 26, 27, 28, 29]). The stimulus properties that previous research has shown to be discriminable (either categorizable or recognizable) bypeople on the basis of unimodal auditory cues include the perceived hardness of percussive mallets [17] and other material properties ([22]; see also [30]), as well as the geometric shape of various diff erent simple objects ([16, 31, 22, 23]; see also [32, 21, 33]). Research has also revealed that auditory cues can be used to distinguish between breaking and bouncing events [28], to determine the configuration of aperson shands when theyclap [34], and even to determine the sexofaperson from the sounds that their feet makeoncontact with the ground when walking [35]. Results such as these highlight the ability of people to recognize and discriminate between a wide variety of dif- S.Hirzel Verlag EAA 1009

2 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception ferent object properties (what Gibson referred to as the useful dimensions of sensitivity [36]). They also demonstrate that our auditory systems are highly adapted for encoding the spectral and temporal properties of auditory events and for transforming them into mental representations that reflect the important aspects of the physical and spectral properties of the sound sources themselves (see [23, 37]). In fact, according to direct-realist theorists such as Gibson [38], the human perceptual system is able to pick up the ambient cues specifying the physical properties of source interactions in the environment. The suggestion is that there may be adirect recovery of environmental properties from the information available in the stimulus array itself, or to put it another way, the world itself is perceived, not amental representation of it. Gibson [38, 39] used the term aff ordances to describe those organism-relevant properties of the environment that are specified by the structure of the stimulus array itself. However, it is important to note that Gibson, in his ecological, or direct, approach to perception (see also [40, 41, 42]) did not go beyond ageneral description of such potentially relevant auditory cues, and what s more, few others have attempted to quantify them in the years since Gibson s seminal work (see [35]; though see also [43, 44, 45]). In fact, Gibson s theory has primarily been expounded for the case of visual perception. Consequently, critics of the ecological approach to auditory perception have tended to argue that it leaves unspecified many ofthe details concerning the process by which perception takes place, and consequently does not lead to specific empirical predictions [35]. The direct approach to (auditory) perception put forward by Gibson can be contrasted with a more traditional information processing approach to perception, according to which people are thought to use the patterns of stimulation available at their various sensory receptor surfaces in order to construct mental representations that provide the basis for their subsequent perceptual experience. According to this approach, people s perception of the auditory environment is thought to be mediated by their internallyconstructed mental representations of it. Despite the informational richness that empirical research now clearly shows is contained in the auditory feedback provided by our interaction with the objects and surfaces in the environment, people (and this includes participants on consumer panels)are typically unaware of the effect that such surface/product sounds have on their overall perception or evaluation of particular stimuli (i.e., surfaces and products), presumably because of the fact that humans are visually-dominant creatures (e.g.,[1, 46, 47, 48]). Despite this underestimation of the role of auditory cues in multisensory perception, the last decade or so has seen a rapid growth of interest in the auditory aspects of product design (e.g.,[49, 50, 51, 52, 53, 54, 55]). Indeed, a growing body of empirical research now attests to the fact that manipulating the sounds that people hear when they touch and interact with many everyday products and/or surfaces can have adramatic e ff ect on the way inwhich theyperceive and react to them. In this article, we provide an overview ofthe research, both theoretical and applied, on auditory-tactile interactions in the perception of avariety of everyday surfaces and products in use. 2. Audiotactile Illusions Jousmäki and Hari [56] reported aparticularly dramatic demonstration of the auditory modulation of tactile perception (that triggered much of the recent resurgence of theoretical interest in this area) 1,which they labelled the parchment-skin illusion: They showed that people s perception of the palmar surface of their own hands could be changed simply by changing the sounds that they heard when rubbing them together.participants in Jousmäki and Hari s study had to rate the perceived roughness/dryness of their hands while rubbing them together using a roughmoist/smooth-dry composite visual analogue scale. The participants heard the sound made by their hands over headphones (the sounds were picked up by a microphone positioned close to their hands). This auditory feedback could either be veridical (i.e., identical to the actual sound of hand rubbing), or else have been manipulated to reduce the overall sound level (by 15dB) or to amplify or attenuate just the high-frequency sounds (i.e., those above 2kHz). Participants reported that the skin on their hands felt smoother/dryer (like parchment paper) when either the overall sound level was increased, or when just the high frequency sounds were amplified. The participants also judged their hands to feel rougher/moister when sounds in this frequencyrange were attenuated, or when the overall sound level was reduced. Spence, Guest, Chan, Lloyd, McGlone, Phillips, and Jones were able to demonstrate the robustness of the parchment skin illusion at the Royal Society Summer Science Exhibition in 2001 [68]. They found that the majority of the people who visited their stand at the exhibition reported experiencing the illusion. However, Guest, Lloyd, Catmur, and Spence [69] identified a number of methodological limitations with the design of Jousmäki and Hari s original study [56]. For instance, only 11 of the participants (out of the 17 tested) who showed the parchment skin illusion in preliminary testing were allowed to complete the main part of Jousmäki and Hari s experiment, thus making it uncertain whether the parchment skin illusion was robust enough to reach significance if tested in arandomly selected group of participants. 1 Early psychophysical studies on audiotactile interactions tended to focus on studying the extent to which stimuli in one modality (e.g., audition) could mask aperson s ability to perceive, ordiscriminate, the stimuli presented in the other sensory modality (e.g., touch; see [57, 58, 59, 60]). By contrast, more recent studies have tended to focus on genuinely perceptual interactions between the two modalities (e.g., [61, 62, 63, 64, 65, 66]; see also [67] for arecent review). The crucial diff erence between these two approaches is that the auditory stimuli used in recent research have tended to be much more ecologically-valid than the white noise stimuli typically used in previous crossmodal masking studies. Equally important is the fact that the presentation of auditory stimuli in recent studies has typically been time-locked to that of the tactile stimulation. Both of these factors have led to the emergence of evidence supporting far more robust audiotactile perceptual interactions than was reported in many ofthese earlier studies. 1010

3 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA Rougher 100 Mean roughness rating Smoother Dryer Mean dryness rating Wetter a) Attenuate Normal Amplify b) db -40 db -20 db 0 db -40 db -20 db Attenuate Normal Amplify Frequency manipulation Figure 1. Mean magnitude estimates of (a) rough-smoothness and (b) wet-dryness for three overall-attenuation levels (0dB, 20 db, 40 db) against three frequency manipulations (highfrequency attenuated, veridical audio frequencies and amplified high frequencies) in Guest et al. s study [69, Experiment 2; adapted picture, copyright Springer]. Error bars show the between-participants standard errors of the means. Furthermore, Jousmäki and Hari s use of a composite (rough/moist vs. smooth/dry) response scale means that it is unclear whether the auditory manipulations reported in their study resulted in achange in the perceivedroughness of the participant s hands, a change in the perceived moistness of their hands, or changes in both dimensions simultaneously. Guest et al. [69] therefore attempted to replicate and extend Jousmäki and Hari s [56] study using a more rigorous psychophysical testing procedure. Once again, the hand rubbing sounds that participants heard over the headphones were manipulated (asinjousmäki and Hari s, original study,although the high-frequencies were boosted or attenuated by only 12dB), but now the participants were either asked to rate how rough their hands felt or, on other interleaved trials, how moist their hands felt instead (i.e., the response dimensions were presented separately using independent visual analogue scales). When taken as agroup (i.e., including the data from all of the people who took part in the study), the participants reported that their hands felt dryer when the concurrent hand-rubbing sounds were either manipulated by amplifying the overall loudness level or when just the high-frequency sounds were boosted. Somewhat surprisingly, the direction of the shift in tactile roughness perception reported in Guest et al. s [69] study (see Figure 1) diff ered from that described by Jousmäki and Hari [56]. While the amplification of the auditory feedback led to an increase in smooth/dry responses in Jousmäki and Hari s study, the very same manipulation only led to asignificant increase in dry responses in Guest et al. s study, but had no significant main effect on roughness judgments per se. One explanation for this diff erence is that Jousmäki and Hari used a composite response scale ( rough/moist smooth/dry ), thus making it unclear which response dimension was driving participants responses. For example, participants may have based their responses more on the wet-dry dimension than on the rough-smooth dimension. Guest et al. s results, using separate unidimensional psychophysical response scales, therefore suggest that moistness may be a more salient perceptual dimension of auditory skin feel than roughness (since all the eff ects in the former perceptual dimension were significant, whereas the main eff ects of sound pressure level and high-frequency sound manipulation on the perception of roughness were not) 2. Given that the participants in Jousmäki and Hari s [56] study and in Guest et al. s [69, Experiment 1] study were presumably aware at some level of the unchanging physical state of their hands during the course of the experiment, it is diffi cult to rule out the possibility that they may simply have responded in the manner in which they thought the experimenter wanted them to, rather than because they genuinely felt the texture of the skin of their hands changing as a function of the sound manipulation introduced (i.e., it is diffi cult to rule out aresponse bias interpretation of the experimental data, especially giventhe use of a subjective report design). Guest et al. were, however, able to address this concern in a further experiment by showing that the magnitude of the auditory modulation of tactile perception was reduced if a small temporal asynchrony (of 150 ms) was introduced between hand rubbing and the presentation of the associated auditory feedback over headphones (see also [56]). This pattern of results is more consistent with a perceptual than with a decisional interpretation of the audiotactile interaction, as decisional factors should not be aff ected by such modest timing diff erences, whereas multisensory perceptual eff ects often are (e.g., see [71, 72] on this point). Guest et al. [69] also reported afurther experiment in which they showed that peoples perception of the texture of abrasive sandpaper samples could be changed by varying the auditory feedback that people heard when touching them. Participants in this study made speeded discrimination responses (rather than unspeeded perceptual judgments used in the experiments reported so far) regarding the roughness of apair of diff erent abrasive sandpaper samples. The perception of tactile roughness was modulated by the frequency content of the auditory feedback from the sandpaper-rubbing sounds. The amplification of the high frequency sounds led to apattern of results that was consistent with an increase in the percep- 2 Note that Terhardt [70] also reported that changes in overall sound pressure level have little eff ect on unimodal auditory roughness judgments for various tone stimuli (including amplitude- and frequency-modulated tones as well as beating tone pairs and pairs of amplitude-modulated tones) and pulse trains. 1011

4 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception tion of roughness of the sandpaper samples, while the attenuation of the high-frequency sounds gave results that were more consistent with an increase in the perception of smoothness. Taken together, the results of Guest et al. s [69] study and Jousmäki and Hari s [56] study therefore provide convincing empirical evidence demonstrating the modulatory e ff ect of auditory cues on people s tactile perception of a variety of diff erent surfaces (see also [73] for early work in this area; though see [13]). It should be pointed out that both studies focused on the e ff ects of boosting or cutting certain auditory frequencies (i.e., highpass and lowpass filtering), or else varying the overall sound level, on the multisensory perception of the felt roughness of skin or sandpaper. Over the years, many other psychophysical studies have shown that the unimodal perception of auditory roughness does not depend solely upon spectral factors (e.g. [74, 75]), but can also be aff ected by the temporal aspects of the sound that is presented (e.g., [76, 77, 78, 79, 70, 80, 81]; see also [82]). In particular, research has shown that the modulation present in the amplitude envelope (i.e., in the temporal properties of an auditory stimulus) also constitutes an important component of auditory roughness perception. One interesting manipulation to be investigated in future research would therefore be to increase or decrease the amount and shape of the amplitude modulation (AM) of the sound made by aparticipant scontact with agiven surface texture, and to assess what e ff ect it had on their perception of surface roughness. Klatzky et al. [30] have also shown that frequency-dependent diff erences in the decay of synthesized sounds elicited by the contact with amaterial can also play acritical role in the perception of various material qualities. Given the relatively independent development of empirical research on the unimodal perception of auditory (see above) and tactile roughness (e.g., see [83, 84]), it will be an interesting question for future research to determine how the two phenomena map onto (and influence) one another. As, for example, under conditions where participants have to make auditory and/or tactile roughness judgments under various diff erent kinds of crossmodal conflict (cf. [85], for areview ofstudies of roughness perception under conditions of visual-tactile conflict). 3. Auditory Influences on Tactile Phenomena It is worth pausing here to note that various other apparently tactile phenomena may actually reflect the consequences of changes in auditory perception as well. Forexample, Gordon and Cooper [86] showed that people s ability to discriminate the orientation of an undulation on an otherwise smooth surface improved if they held a piece of paper under their fingertips, rather than simply touching the surface directly with their fingers. Following up on this research, Lederman [87, 88] noted that the use of apaper sheaf tends to makesurface textures feel rougher.this e ff ect, which has been known about and used by skilled craftsmen (such as furniture makers and automotive panel beaters) for many years, has been variously attributed by psychologists to the paper sheaf possibly masking the activity of certain classes of receptors in the skin [86], or to changes in the shear forces on the skin induced by the sheaf somehow facilitating tactile perception [87, 88]. However, one of the most noticeable changes induced by the use of the sheaf is how much more one hears with the paper in place (why not try this for yourself). Given Guest et al. s [69] results, it would seem likely that the improved tactile sensitivity reported in these earlier studies may have been attributable, at least in part, to the fact that the paper acts as a sort of amplifier, and that these touchrelated sounds elicited when people touch a textured surface influenced their judgments of how it felt (at either a conscious or subconscious level; though see [88] for evidence that the introduction of the paper sheaf can still exert asignificant e ff ect on roughness judgments even when the contact sounds were masked, showing that touch-produced sounds cannot provide the sole explanation for this eff ect). Artificial auditory cues are now increasingly being used to augment virtual haptic interfaces (e.g., [89, 30, 90, 91, 92, 93, 94] see also [95, 96, 97, 98]). Artificial auditory cues have also been used in various other forms of sensory substitution/augmentation systems as well (e.g., [99, 100, 101]). In fact, researchers are currently developing sensory substitution systems to enable people who have lost tactile sensation in their hands to discriminate felt textures by placing microphones on the fingertips of specially constructed gloves (see [101]). The sound transduced by these microphones is then amplified and presented to people suff ering from tactile loss. Preliminary results suggest that these auditory prostheses can enable people to discriminate between diff erent textures such as glass, metal, wood, and paper, in the absence of either tactile or visual cues (see also [100]). Such results converge with recent cognitive neuroscience studies showing that one can even generate the illusion of tactile perception (in the absence of any tactile stimulus) in many normal individuals simply by presenting them with the sounds that are typically elicited by physical contact with aparticular surface (e.g., see [64, 67]; see also [62]; cf. [102]). 4. Interim Summary The results described thus far show that auditory cues contribute to the multisensory perception of surface texture. The modulatory eff ect of auditory cues (such as the overall sound level and the spectral distribution of energy) has been demonstrated to aff ect the perception of the roughness of sandpapers (see [69, 73], the stimuli that have typically been used in the majority of previous psychophysical research on texture perception (see [85] for a review). Research has also shown that auditory cues can modulate people s perception of the roughness and dryness of their own hands [69, 56]. Auditory contributions to skin feel have even been used in advertising campaigns to illustrate the eff ectiveness of a particular brand of shaving foam. 1012

5 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA For example, Lederman [13] describes a television commercial for ashaving cream in which the edge of acredit card wasdrawn across both sides of amodel sface shown in close-up. One side of their face had been shaved using aproprietary shaving cream, the other with acompetitor product. Although both sides of the model s face looked identical after shaving, the diff erence in the closeness of the shave was eff ectively illustrated by the diff erence in the sound made by the credit card as it was drawn across the skin on the two sides of the model s face. The existence of multisensory illusions such as the parchment-skin illusion helps to illustrate the multisensory nature of our everyday perception. The fact that the majority of these illusions occur automatically and in an obligatory manner highlights the potential utility that a better understanding of the rules governing multisensory integration might o ff er in terms of modulating various aspects of product perception. It seems likely that the particular auditory manipulations that will lead to a change in the perception of roughness most likely depend on the particular surface being judged (i.e., they are relatively surface/stimulus-specific). Having shown that auditory cues contribute to people s perception of the feel of a range of diff erent surfaces, we will next highlight research demonstrating that auditory cues can also modulate the multisensory perception of avariety of everyday products in use. 5. Auditory Contributions to Product Perception Early research on the auditory aspects of product perception highlighted the importance of auditory cues in modulating people s perception of sound-reproducing devices (such as loudspeakers and hearing aids), where sound quality was integral to the functionality of the product itself (see [103, 104, 105]). More recently, however, researchers have also illustrated the importance of auditory cues to many other devices where sound quality might not be thought to be such an obviously important product attribute (e.g., [106, 107]; see also [47]). In particular, researchers have demonstrated the importance of product sound quality to people s evaluation of everything from vacuum cleaners [108, 109, 110, 111, 112, 47] to kettles [113], from coff ee makers [114] to hair dryers (see [115]), and from offi ce machines [116, 117] to dishwashers (see [118, p. 144]; see also [119]). The contribution of auditory cues to product perception has been known about in the automotive industry for many years (e.g., [120, p. 111]), and on occasion the unique, or characteristic, sounds made by avehicle has been linked to aspecific manufacturer or brand name. For example, Harley-Davidson went to great lengths to try and patent the distinctive sound of their motorcycle engines, which theyconsidered to be an essential part of the experience of ownership [121, 122]. Distinctive product sounds (sometimes known as signature sounds ) can denote character and become strongly associated with aparticular product and its functionality (see also [123]). Honda recently advertised their cars on television with the strap-line this is what a Honda feels like, only to illustrate this by means of nothing more than a choir making the appropriate car sounds (see uploaded/ Honda_Civic_Choir_43459.html, downloaded on ). Indeed, research shows that people s perception of the quality of acar can be influenced by the sounds it makes. Car manufacturers have,for many years, employed teams of experts to evaluate and modify such sounds (e.g., [124, 125] [120, p. 111]). There is now alarge body of published empirical research on the evaluation and modification of specific car sounds (e.g., [126, 127, 128, 129, 130]). However, it is not just the sound of the engine (or of the car on the road) that helps to determine consumer preference (though this is, of course, important), but also the other sounds that the car makes, such as, for example, the sound of the door as it is closed (e.g., see [131], cited in [132]; [133, 134, 135, 136, 137, 138, 55]). Recently, researchers have also started to invest more e ff ort into evaluating and modifying the sound quality of acar s horn (e.g., [139]) and in trying to match it to aparticular type of car as well (e.g., Howacar shorn says: Buy me, [140]; see also [141]). Much of the research in this area is based on jury evaluation techniques in which untrained jurors or potential customers listen to pairs of sounds and select the one that is preferred along agiven dimension (such as, for example, powerfulness or sportiness ). By analyzing the preferences of many diff erent observers for agiven set of sounds (compared one pair at atime), researchers are able to rank the product sounds along the sound quality dimension being assessed (e.g., see [128]). Jury testing is, however, both time-consuming and costly, leading a number of manufacturers/researchers to try and develop alternative means for product sound evaluation. One such example comes from recent attempts that have been made to simulate consumer preferences for particular sounds using neural network modelling techniques (e.g., [126, 128]). The ultimate aim of this rapidly-growing field of psychoacoustics research, known as acoustic design (e.g., [142, 143, 144]), is to give particular products what Ungar [54] has described as the sound of quality (see also [145, 146, 147, 148, 149, 150]). In fact, researchers have recently started to investigate the consumer perception of product-related sounds using a variety of diff erent techniques (e.g., [151, 152, 153]). For example, Lageat et al. reported asensory evaluation study in which 12 people evaluated 8 diff erent flip-top cigarette lighter sounds on the basis of their perception of the perceived luxury. These lighters were either already on the market, or else were available in prototype form. The participants were able to listen to each of the lighter sounds as often as they liked and then had to write down (in their own words) the descriptors that they felt were most appropriate for the sounds. Next, the participants had to rate each lighter sound against these descriptors using a10 point scale. Subsequently, the participants had to indicate which of the 8 sample lighter sounds was the most and 1013

6 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception More 100 pleasant a) Headphones Toothbrush Microphone Footpedals Dimension scale Mean pleasantness rating Less pleasant Rougher b) Attenuate Normal Amplify -40 db -20 db 0 db Figure 2. Demonstrating the experimental set-up used in Zampini et al. s study of electric toothbrushes [152, copyright International Association for Dental Research]. Mean roughness rating db -20 db -40 db least representative of each descriptor in order to confirm the appropriateness of the descriptors that theyhad chosen. The descriptor lists were then consolidated across participants and a number of the terms removed. The remaining descriptors were then grouped by common meaning and given aneasily understood common definition. Using this Quantitative Descriptive Analysis technique, Lageat et al. were left with the following 7sound descriptor definitions: intense (perceived sound level), high-pitch (sound frequency), clicking (a dry, sharp sound), fast (speed of spread of the sound), matte (a consistent, full-bodied sound), even (variations in time of the sound emanating from the product), and resonant (echoing envelope following the base sound) (see [51, p. 102]). In the next stage of Lageat et al. s study,200 consumers had to rate the luxury of the various lighter sounds. Principal components analysis was used to ascertain which product sound features were most strongly associated with people s perception of the hedonic benefits of the product. Twodi ff erent groups of consumers were identified, one for whom the luxury of the flicking open of the cigarette lighters was associated with sounds that were matte, even, and low in pitch; For the other group, luxury was associated with the clear, resonant, and clicking auditory descriptors. In a rather diff erent study, Zampini et al. [154] used a psychophysical testing technique adapted from Guest et al. s [69] earlier research to investigate whether people s perception of the pleasantness and roughness of an electric toothbrush might also be a ff ected by the sound that it made while in use. The participants in Zampini et al. s study were required to make stereotypical brushing movements across their front teeth with astandard electric toothbrush while theyrated either the pleasantness or roughness of the vibrotactile stimulation they felt on their teeth (see Figure 2). Participants reported that the brushing of the toothbrush felt more pleasant, and less rough, when either the overall sound level was reduced, or when just the high frequency sounds (2 20 khz) were attenuated. By contrast, Smoother 0 Attenuate Normal Amplify Frequency manipulation Figure 3. Highlighting the results of Zampini et al. s study of electric toothbrushes [152, adapted pictures, copyright International Association for Dental Research]. Mean responses for the (a) unpleasant-pleasant, and (b) rough-smooth response scales for the three overall attenuation levels (0 db, 20 db, or 40 db) against the three frequency manipulations (high frequencies attenuated, veridical auditory feedback, or high frequencies amplified).each data point reflects the average of 100 trials (5 trials for each participant). Error bars represent the between-participants standard errors of the mean. increasing the overall sound level and/or amplifying the high frequency components of the toothbrush sounds associated with participants brushing their teeth both resulted in the vibratory sensations being rated as less pleasant and rougher. There was also an interaction between these two factors, such that the vibratory sensations were judged as being least pleasant when the high-frequency sound components were amplified and the overall sound level was maximized. In fact, the sound frequency manipulations were found to have little impact on participants ratings at the quietest overall sound level, presumably because the overall sound level approached threshold in these conditions. Zampini et al. s results therefore demonstrate that people s judgments of the qualities (specificially the perceived roughness and pleasantness) of a product in use can be dramatically altered by changing the nature of the sound it makes. In particular,their results highlight that the perceivedpleasantness and roughness of an electric toothbrush can be modulated by changing both the intensity and the frequency spectrum of the sound it makes while people use it. The auditory modulation of roughness perception reported in Zampini et al. s [152] study (see Figure 3) is consistent with the eff ect reported earlier by Guest et al. [69], although the magnitude of the sound-induced change in roughness perception of electric toothbrushes in the for- 1014

7 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA mer study was somewhat larger than that reported in Guest et al. s study of skin texture. Given that the design and procedure used in the two studies was very similar, this diff erence may well reflect the fact that participants received more tactile information when rubbing their hands together in Guest et al. s study,than theydid from using an electric toothbrush on their teeth in Zampini et al. s study. Indeed, researchers have argued for many years that the extent to which one sense dominates, or modulates, perception in another sensory modality will depend on the relative strength, reliability,oramount of information presented in the two modalities (e.g., [155, 156]; though see also [157]). One of the key benefits of Zampiniet al. s experimental design [152] was that it allowed the researchers to investigate people s responses to novel product sounds without having to manufacture prototype toothbrushes in advance to deliver those sounds (a process that is generally both expensive and time-consuming). What s more, the participants were actually able to interact with the product (i.e., the toothbrush) inamultisensory (i.e., realistic) manner (contrast this with the consumer evaluations in Lageat et al. s study [51], which were based solely on the more artificial playback of avariety of pre-recorded lighter sounds). There is, however, typically a trade-off between the ecological validity of a particular empirical study and the degree of control that a researcher has over the stimuli and conditions of testing. Indeed, one might question the ecological validity of Zampini et al. s study on the basis that participants had to sit for an hour in asoundproof booth while evaluating a number (nearly 200) of diff erent electric toothbrush sounds. In fact, one might wonder whether any of the participants actually realized that the toothbrush that they were holding never changed its operating characteristics, and that all that was changing during the experiment were the sounds that were being played to them over the headphones. It is diffi cult to be certain that all participants remained naïve as to the purpose of the experimental manipulations. However, when asked by the experimenter at the end of the experiment, the majority of the participants typically reported that the sounds (that were actually presented over headphones) appeared to emanate from the toothbrush itself, presumably due to an audiotactile ventriloquism eff ect (see [72]). What s more, after the experiment, anumber of the participants also spontaneously asked the experimenter how he had managed to change the operation of the electric toothbrush during the course of the experiment. Such reports, albeit anecdotal, provide at least some support for the ecological validity of Zampini et al. s experimental approach (i.e., they support the view that the majority of the participants really were evaluating the feel of the toothbrush rather than simply rating the various diff erent sounds presented over the headphones irrespective of the toothbrush theywere holding in their hands and rubbing across their teeth). It should be noted, however, that in our more recent research, we have attempted to improve the ecological validity of our approach by presenting participants with a diff erent product/sample on each trial (as, for example, in our recent study of people sperception of aerosol spraying sounds; see [153], described below). It is important to note that while reducing the noise made by an electric toothbrush enhanced pleasantness ratings in Zampini et al. s study [152], there are other situations in which reducing the sound made by aproduct may actually have a detrimental eff ect on product perception. For example, Froman ([158]; cited in [159, p. 327]; see also [54]) describes the example of anoiseless food mixer that failed in the marketplace because it didn t seem to haveanypower -it didn t make enough noise (though see [124]). It may be that while loud sounds signify e ffi ciency and power in afood processor, amplifying the high frequency sounds made by an electric toothbrush reminded people of the electric drills used by dentists, and this may be why they found these sound conditions so unpleasant (an association confirmed by many of the participants at the end of Zampini et al. s study; see also [160]). There are anumber of other examples showing that noisier products are sometimes preferred by consumers overtheir quieter counterparts such as, for example, in the case of washing machines [107], vacuum cleaners ([111]; see also [109, 161]), and video recorders [118, pp ]. One factor that may be particularly important here relates to people s expectations concerning what particular classes of product ought to sound like. For example, Jekosch [142, pp ] [143, p. 207] has argued that consumer expectation plays an important role in determining their response to particular product sounds. Such expectancy eff ects will obviously play a larger role in modulating people s judgments of product sounds from familiar product categories (such as for the electric toothbrush sounds evaluated by Zampini et al. [152]), than for less familiar product categories. It would therefore appear that the manner in which aparticular product sound should be modified in order to improve consumer perception (such as, for example, by making it quieter vs. louder) will depend upon the perceived functionality of the specific product, as well as on the particular associations and expectations that specificclasses of consumers have regarding the product-related sounds (cf. [49, 162, 163]). In a more recent study,zampini and Spence investigated whether people s perception of an aerosol spray could also be modified by changing the spraying sounds associated with its operation [153]. Participants in one experiment had to rate the pleasantness and forcefulness of a number of aerosol sprays that were held by an experimenter outside of the testing booth. The participants could see the aerosol being sprayed into the air over a microphone through a window in the side of the booth, while the sound made by spraying the aerosols picked up by the microphone wasmanipulated and played back to the participants over headphones (see Figure 4). The aerosol sprays were rated as being more pleasant when either the overall sound levelwas reduced, or when just the high frequencysounds in the 2 20 khz range were attenuated. By contrast, ratings of the perceived forcefulness of the aerosol sprays were re- 1015

8 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception Response scale presented on computer monitor More pleasant 100 a) Aerosol sounds presented via headphones Response made with footpedals Button to confirm response and advance to next trial Window in soundproof booth Microphone Aerosol spray Figure 4. Illustrates the experimental set-up used in Zampini and Spence s (submitted) study of aerosol spray sounds. Participants were seated in a soundproof booth looking at a microphone, the experimenter s hand delivering the aerosol spray from directly above the microphone, and the visual analogue response scale presented on acomputer monitor (placed directly behind the microphone and aerosol spray). The aerosol sounds were presented to the participants over headphones. Participants responded by using two foot-pedals (one situated under either foot) to move a marker on the response scale to the left or right, respectively. duced when either the overall sound level or just the high frequency sounds were attenuated (see Figure 5). In a follow-up experiment, Zampini and Spence [153] increased the ecological validity of their design by requiring participants to spray various diff erent (but actually physically identical) cans of deodorant onto their own body while once again manipulating the nature of the auditory feedback that they heard. In this study,the participants sprayed the aerosols onto their wrists, forearms, biceps, and armpits on both sides of their body. When the participants were able to feel the aerosol sprays on their own body they were somewhat less influenced in their judgments of the pleasantness and forcefulness of the aerosol spray by changes in the overall intensity of the aerosol sounds than when the aerosols were simply sprayed into the air (presumably due to the additional tactile cues available in the former case). However, the more specifichighfrequency sound manipulation was just as eff ective no matter whether the aerosol was sprayed into the air by the experimenter from outside the soundproof booth versus onto the participant s own skin. Taken together, Zampini et al. s [152, 153] results highlight the potential utility of the psychophysical approach to product sound evaluation and development. Future studies using more ecologically-relevant/plausible filtering would clearly be desirable, given that the low and high pass filtering used in Zampini and colleagues research is somewhat lacking in terms of ecological validity (i.e., given our arbitrary boosting or cutting of all sound frequencies above 2kHz; note though that Mean forcefulness rating Mean pleasantness rating Less pleasant Stronger Weaker b) Attenuate Normal Amplify Frequency manipulation db -20 db Attenuate Normal Amplify 0 db 0 db -20 db -40 db Figure 5. Highlighting the results from Zampini et al. s study [153] of aerosol spray sounds. Mean responses for (a) the pleasantness (unpleasant-pleasant) and (b) the forcefulness (weakstrong) response scales for the three overall attenuation levels (0 db, 20 db, or 40 db) against the three frequency manipulations (high frequencies attenuated, veridical auditory feedback, or high frequencies amplified) used. The error bars represent the between-participants standard errors of the mean. this cut-off was based on previous research [56]; cf. [104, Experiment 2] [164]. One interesting approach that may be relevant here comes from the recent work of Susini et al. [151]. They created 4 new synthetic (or hybrid) airconditioner sounds by interpolating between the sounds of various diff erent pairs of pre-recorded sound samples generated by actual air-conditioning units. The participants in Susini et al. s psychoacoustic study were required to evaluate an intermingled selection of real (15) and synthetic (4) air-conditioning unit sounds presented for 3seconds each. Susini et al. used multi-dimensional scaling (MDS) to analyse their results. This technique has the advantage that it doesn t require the researchers involved to prespecify any particular perceptual dimensions that they want to study. Rather these dimensions are derived from the performance data itself. Interestingly, Susini and his colleagues [151] also reported individual diff erences in people s preferences for the sound associated with diff erent brands and models of air-conditioning units (just as reported in Lageat et al. s earlier study [51]). Of the 200 participants tested in the second part of their study, one group simply appeared to prefer air-conditioning noises that were quieter, while the other group s preferences seemed to depend more upon the spectral centre of gravity (correlated with brightness in studies of musical timbre) of the sounds and also on the ratio of the noisy to the harmonic part of the auditory 1016

9 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA spectrum. Susini et al. s study therefore also highlights the potential utility of using both real and synthesized product sounds in aproduct sound evaluation context. Finally,itisworth pointing out acouple of examples of auditory product design where industrial designers have specifically modified the sounds of particular products to enhance their appeal to consumers at a more emotional (i.e., as opposed to aperceptual) level (see [165, 52]; see also [166]). For example, Donald Norman reports that the designers of the Segway, atwo-wheeled personal transporter: were so obsessed with the details on the Segway HT that they designed the meshes in the gearbox to produce sounds exactly two musical octaves apart when the Segway HT moves, it makes music, not noise [52, p. 120]. Norman also highlights the example of the Alessi 9091 singing whistle kettle designed by Richard Sapper, where considerable eff ort went into modifying the sound produced by the whistling spout achord of e and b, or, asdescribed by Alberto Alessi, inspired by the sound of the steamers and barges that ply the Rhine. ([52, p. 121]; see also [167, p. 83]). 6. Auditory Contributions to the Perception of Food Auditory cues also play an important role in people s evaluation of food and drink, especially their perception of the crispness of dry food products (e.g., [168, 169, 170, 171, 172, 173] see [174, 175, 176] for excellent reviews). In particular,recent research has shown (once again) that the modulation of the frequency composition and the overall sound intensity level ofthe sounds produced by foods can influence our perception of them. Zampini and Spence [177] investigated whether the perception of the crispness and staleness of potato chips would be a ff ected by modifying the sounds produced during the biting action. Participants in this study had to bite into alarge number of potato chips with their front teeth while rating either the crispness or freshness of each potato chip using acomputer-based visual analogue scale. Pringles potato chips were used given their uniform size and shape. The door to the experimental testing booth was opened on each trial and anew Pringle potato chip was presented to the participant. This experimental protocol helped to keep participants ignorant of the fact that all of the Pringles actually came from the same batch/packet. The perception of both the crispness and staleness of the potato chips was systematically altered by varying the loudness and/or frequency composition of the auditory feedback elicited during the biting action. In particular, the potato chips were perceived as being both crisper and fresher when either the overall sound level was increased, or when just the high frequencysounds (inthe range of 2-20 khz) were selectively amplified (see Figure 6) 3.Once again, there was aninter- 3 This research has recently been followed up by Heston Blumenthal at the 3 Michelin-starred restaurant, The Fat Duck, in Bray (see action between these twofactors such that the potato chips were rated as being maximally crispy and fresh when the overall sound level was at its highest and when the highfrequency components of the sounds were boosted. Meanwhile, the sound frequency manipulation had little eff ect at the lowest overall sound level(i.e., when the sound was attenuated by 40 db), presumably due to the fact that all of the sounds were again presented at near-threshold levels under these conditions. Given that bone-conducted (and not just air-conducted) sounds have also been shown to influence people s sensory evaluation of foods (e.g., see [180]) it would be interesting in future research to determine whether a similar pattern of results to that reported by Zampini and Spence would have been observed if participants were forced to break the crisps with their fingers (rather than between the teeth) in order to eliminate any contribution of bone-coducted sounds to perception. Although the overall sound intensity leveland the spectral profile of the sounds have been shown to be twoofthe most important auditory factors contributing to our perception of the crispness of foods (e.g., [168, 181, 182, 169, 183]; see also [184]), it is important to note that the impulsive nature of the sound created by biting into crispy and crunchyfoods (such as cereals, biscuits, crisps, celery and carrots) also carries information about the rheological properties of the foodstuff being consumed. Indeed, research by Vickers and Wasserman [183] has highlighted the significant role that the temporal attributes of food eating sounds, such as their degree of unevenness or discontinuity can also play in modulating people s subjective perception of crispness of various diff erent dry food products (see also [185, 186, 187]). Meanwhile, amplitudetime plots of the sounds associated with the consumption of crispy foods have been used to reveal the irregular variations in loudness overtime ([188]; see [174] for areview). In future research it would therefore be particularly interesting to look at the consequences for perception of introducing frequency-dependent variations in rise-time and damping rate of food-eating sounds, perhaps using signal analysis methods such as Prony (e.g., [189] [190, pp ] [191]) orthe matrix pencil method (e.g., [192]). In a follow-up study, Zampini and Spence [193] investigated the role of auditory cues in the perception of carbonation in beverages (see also [170]). Participants had to rate aseries of sparkling water samples held in their hands in terms of their perceivedcarbonation using avisual analogue scale. The water sounds were modified by changing the loudness and/or frequency composition of the auditory feedback emitted by the water samples. The carbonated water samples were judged to be more carbonated when Guests eating certain courses on his taster menu (the menu degustation) are provided with apair of headphones. A microphone attached to the headphones picks up the sound of the diner eating aparticular noisy foodstuff (such as acarrot dipped in asauce; or a slice of chocolate cake in which popping candy has been introduced) and amplifies the sound that is played back overthe headphones. The aim here is both to alert people to the influence of sound on their perception of what they eat, and also to try and enhance the flavour of the food by boosting the sound that diners hear (see [178, 179]). 1017

10 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception Crisper 100 a) Mean freshness rating Mean crispness rating Softer Fresher 100 Staler b) Attenuate Normal Amplify Attenuate Normal Amplify Frequency manipulation 0 db -20 db -40 db 0 db -20 db -40 db Figure 6. Highlighting the results from Zampini et al. s study of the perception of crispness and freshness in potato chips [177, adapted pictures, copyright Blackwell Publishing]. Mean responses for the soft-crisp (a), and fresh-stale (b) response scales for the three overall attenuation levels (0 db, 20 db, or 40 db) against the three frequency manipulations (high frequencies attenuated, veridical auditory feedback, or high frequencies amplified). Error bars represent the between-participants standard errors of the means. the overall sound level was increased and/or when the high frequency components (2 20kHz)of the water sound were amplified. Given the uncertain ecological validity of the auditory manipulation used in Zampini and Spence s original experiment, it was reassuring to find that participants in a subsequent experiment could correctly judge the actual carbonation of sparkling water samples that had been adulterated with a varying proportion of still water solely by the sounds that they made. By contrast, varying the distance that the cup of water washeld from the participant s ear reduced the overall sound level(the sound pressure level varied from 50dB at a distance of 1cm to 30dB at 50 cm from the ear)but had virtually no e ff ect on participant s ratings of the carbonation of the water samples, thus suggesting the existence of some kind of carbonation constancy eff ect (e.g., [154, 194]) 4.Interestingly, however, a 4 Zampini andspence [193] also conducted an experiment in which they made a number of recordings of the sounds made by the bubbles in a cup of sparkling water. These sound recordings were then either speeded-up (i.e., played back at twice their normal rate) or else slowed-down (played at half their normal rate). The participants rated the sounds that had been speeded-up and the normal carbonation sounds as being more carbonated than the recordings in which the rate of popping had been slowed-down (cf. [195, p. 152] for related work showing that an increased density of sound occurrence can also lead to a change of perceived crispness). While these results suggest that the rate of popping in a fizzy beverage subsequent experiment failed to demonstrate any eff ect of these auditory manipulations on the perception of carbonation and oral irritation of water samples that were held in the mouth. Taken together, these results therefore highlight the significant role that auditory cues (specifically, the overall sound level and the amplification/attenuation of high-frequency sounds above 2 khz) play in modulating our perception of the carbonation of beverages in the hand, and the dominance of oral-somatosensory and nociceptive cues over auditory cues in the perception of carbonation of beverages in the mouth (cf. [155]). They also highlight the importance of studying auditory perception in the appropriate multisensory product context. It is worth noting here that auditory cues are not only capable of modulating food perception at aperceptual level, but can also do so at a more semantic level (e.g., [143]). For example, anumber of food products appear to have signature sounds associated with them: Think of the snap of the Kitkat or the distinctive crack of the chocolate breaking as one bites into a Magnum ice cream, the sound of the release of carbonation as a bottle of Schweppes is opened, or even the distinctive pop that one hears as a bottle of Snapple is opened. Lindstrom [123, p.12]), in his book Brand Sense, describes how Kellog s even went so far astoattempt to patent the specific crackling sound made by their corn flakes in the 1980 s. Advertisers have also used the sound of crisps to signify freshness and tastiness. Forexample, Engelen [162] describes a Dutch crisp manufacturer (Crocky) whose advert specifically focused on the crack of their crisps when being eaten, a sound so loud that (in the advert) it apparently cracks the viewer s television screen. The packaging of a food product can also come to advertise the auditory qualities of the product within -think of the noisy packets in which crisps are often packaged (cf. [197]). Finally,Jordan [118, p. 108] highlights the example of a beer manufacturerwho tried to elicit the sound of quality by varying the hiss that was heard when their cans of beer were opened. 7. Summary The last few years have seen arapid growth of interest in the auditory aspects of product design (e.g., [49, 143, 50, 53, 54, 55]). The research reviewed here highlights the profound eff ect that auditory cues (such as variations in the overall sound leveland variations in the spectral distribution of energy) can have on people s perception of everything from textured surfaces (e.g., sandpapers)to foodstuff s(such as potato chips and carbonated beverages), and from everyday products in use (such as aerosol sprays, electric toothbrushes, food mixers, and cars) to the perception of the texture of our own skin (see also Bradley contributes to the perception of carbonation it is important to note that varying the speed at which the recordings were played back inevitably led to changes in the pitch of the sound as well [196]. Consequently,Zampini and Spence were unable to rule out the possibility that the pitch-shifting e ff ect may also have contributed to participant s perception of carbonation. 1018

11 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA [198] and Sound of clothes: Blow, clap, talk and hum [199] for recent examples showing that designers have also starting to think about the possibilities associated with manipulating the sound of clothing). The available empirical research now demonstrates that one can alter people s perception of product qualities such as the perceived powerfulness, pleasantness, or forcefulness simply by modifying the auditory cues associated with the operation of the product. The auditory cues associated with a particular product can also be manipulated to provide a signature sound that can be linked to aparticular brand (e.g., [123, 121, 122]), and/or to provide the sound of quality in aproduct (e.g., [142]). There has also been agrowth of interest in trying to modify the sound of various products in order to enhance their appeal to consumers at amore emotional level ([52]; see also [165]), and to convey a number of other product attributes/qualities auditorily [143]. One important finding to have emerged from this research is that there is no simple universal auditory manipulation that will have the same e ff ect on the sound quality of all products/surfaces alike [51, p. 107]; [121] [151, p. 764] [54]. While research suggests that changing the auditory qualities of a product can often influence a consumer s perception of the quality and functionality of that product, the direction in which the sound should be changed (i.e., making it quieter vs. louder, or boosting vs. attenuating certain sound frequencies) appears to be rather product specific. It will therefore be particularly interesting in future research to investigate whether there are any particular auditory frequency components that may be specifically associated with pleasantness / roughness (or forcefulness) judgments for products such as electric toothbrushes or aerosols, given the uncertain ecological validity of the particular frequency manipulations used in many ofthe studies reported to date (see [11, 69, 152, 193, 153]; though see also [151]). Given the widespread evidence for the important role that auditory cues play in modulating consumer perception of a wide variety of products [200], it comes as something of a surprise to realize how little auditory product testing is currently being carried out in many appliance domains (see [135, 148]). It is also important to note that the auditory contributions to multisensory perception typically take place without people (i.e., consumers/participants) being consciously aware that what they are hearing is influencing their overall product experience (e.g., [123, 5]). Nevertheless, the results reviewed here are consistent with the growing body of neurophysiological and electrophysiological data demonstrating the close links between the processing of auditory and tactile sensory inputs at a neural level in the brain (e.g., [201, 202, 203, 204, 205]). For example, recent neuroimaging studies have demonstrated that audiotactile interactions in information processing can take place at the very early stages of information processing (i.e., within 50 ms of stimulus onset; e.g., [206]; see also [207]. Results such as these help to emphasize the limitations that may be associated with relying solely upon introspection and verbal report when trying to measure and account for consumer behavior (see also [4]). These neurophysiological findings may also help to explain the results of various product-related research that has highlighted the close link between product sound and vibration in consumer s overall evaluation of a number of diff erent products (e.g., [195, 152]; see also [208]). What s more, there is now growing evidence that visual cues can also play a significant role in modulating people s putatively auditory evaluation of specific product, and environmental, sounds (e.g., [209, 210, 211] [132, pp ] [212, 213, 115, 214, 141, 215, 216]. Given findings such as these, there must remain some question over the appropriate interpretation of the results of studies (e.g., [51]) where researchers have attempted to study product sound quality in isolation (i.e., in the absence of any of the other multisensory cues, such as visual, tactile, and proprioceptive that would normally contribute to a user s overall evaluation of a flip-top cigarette lighter, as evaluated in Lageat et al. s study [51]). Future research on the perception of product sound qualities will clearly need to consider the extent to which the perception of those properties might be mediated by the automatic multisensory interaction processes that are typically unavailable to introspection (see [1, 123]), and hence must be tested both indirectly through implicit measures of perception, as well as through the probing of consumers verbal responses. Another trend that appears to be emerging in anumber of areas where product sound design is popular (including the study of vehicle sounds, air-conditioner,aerosol spray, and electric toothbrush sounds) isthe move from the evaluation and subsequent re-engineering of prototypes with aparticular sound to the increased use of simulation tools and techniques (e.g., see [128, 217] and the greater use of synthesized product sounds in the product sound design cycle [139, 151, 152, 193, 153]. One advantage of having participants evaluate synthesized product sounds (as well as actual product sounds) is that it allows for a much more rapid evaluation of sounds (i.e., without the need for manufacturing a prototype that can generate each unique product sound) than when only sounds of actual products or prototypes are assessed (the process is also much cheaper, and hence also more cost eff ective). While it is certainly true that many ofthe product sound manipulations that have been assessed thus far have not necessarily been as ecologically valid as they might have been (e.g., see [152, 177, 193, 153]; note that these studies were essentially proof-of-principle studies demonstrating the validity of the psychophysical approach outlined), future studies using more realistic synthetic sounds would appear to hold great promise (e.g., see [151], for a more ecologicallyvalid approach to synthesizing novel product sounds by interpolating between pairs of real product sounds). It seems probable that the use of synthesized sounds in the product design/evaluation cycle can only increase in the years to come. Another growing trend in the area of product sound design is toward active sound design. This is where product 1019

12 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception sounds that have over time been engineered out of a particular product are actively reintroduced, not because of any functional necessity, but rather because of the quality, functionality, and/or performance that such sounds have come to connote in the minds of consumers (e.g., [109]). To give just acouple of examples here, researchers have realized the importance that consumers attach to the mechanical clicking of the relays that were traditionally used to control the direction indicators in a car; They have also realized the importance (in terms of perceived functionality) that consumers associate with the noise made by a vacuum cleaner when it is switched on (e.g., [109, 110]; [118, pp ]). A final interesting topic for future research on auditory product design relates to the topic of individual diff erences. Several of the studies that have been described in this review have demonstrated significant individual differences in terms of the preferences of particular groups of people for qualitatively diff erent product sounds (e.g., see [51, 24, 151, 153]). If such individual diff erences prove to be reliable then we may increasingly start to see specific product sounds that have been tailored for a particular section of the population. What smore, giventhe sensory decline increasingly being experienced by the growing aging population (see [218, 219, 5, 220]), it also seems increasingly likely that we will start to see products (and product sounds) that have been specifically designed to appeal to this particular segment of the population as well (e.g., see [221]). Another area that has, as yet, received relatively little attention from researchers relates to possible cultural diff erences in product sound perception ([222]; see also [223]). To give just one (albeit anecdotal) finding, we recently conducted a public science demonstration in Athens (Ecolife, 2006). To our surprise, we found that boosting the high-frequencysounds made when people bit into crisps (which in the UK has been shown to lead to a systematic modulation of perceived freshness and crispiness; see [177]) typically led the Greek people at our exhibit (both old and young) to spontaneously report that the crisps tasted saltier instead. Should such cultural differences prove to be reliable then they will obviously also provide a very fruitful direction for future research on the auditory contributions to multisensory product perception. References [1] J. Driver, C. Spence: Multisensory perception: Beyond modularity and convergence. Current Biology 10 (2000) R731 R735. [2] U. Neisser: Cognition and reality. Freeman, San Francisco, [3] G. A. Calvert, C. Spence, B. E. Stein (eds.): The handbook of multisensory processes. MIT Press, Cambridge, MA, [4] E. C. Hirschman, M. B. Holbrook: Hedonic consumption: Emerging concepts, methods and propositions. Journal of Marketing 46 (1982) (Summer) [5] C. Spence: The ICI report on the secret of the senses. The Communication Group, London, [6] S. Aljishi, J. Tatarkiewicz: Why does heating water in a kettle produce sound? American Journal of Physics 59 (1991) [7] T. Ananthapadmanaban, V. Radhakrishnan: An investigation on the role of surface irregularities in the noise spectrum of rolling and sliding contacts. Wear 83 (1982) [8] P. A. Cabe, J. B. Pittenger: Human sensitivity to acoustic information from vessel filling. Journal of Experimental Psychology: Human Perception and Performance 26 (2000) [9] R. A. Collacott: Condition monitoring by sound analysis. Non-Destructive Testing: Research and Practice 8 (1975) [10] M. Morgan: Molyneux s question: Vision, touch and the philosophy ofperception. Cambridge University Press, Cambridge, [11] W. W. Gaver: What in the world we hear? An ecological approach to auditory event perception. Ecological Psychology 5 (1993) [12] D. Katz: The world of touch. Erlbaum, Hillsdale, NJ, 1925/1989. [13] S. J. Lederman: Auditory texture perception. Perception 8 (1979) [14] D. A. Norman: The design of everyday things. MIT Press, London, [15] J. D. C. Talamo: The perception of machinery indicator sounds. Ergonomics 25 (1982) [16] C. Carello, K. L. Anderson, A. J. Kunkler-Peck: Perception of object length by sound. Psychological Science 9 (1998) [17] D. J. Freed: Auditory correlates of perceived mallet hardness for aset of recorded percussive sound events. Journal of the Acoustical Society of America 1990 (87) [18] B. L. Giordano, S. McAdams: Material identification of real impact sounds: E ff ect of size variation in steel, glass, wood, and plexiglass plates. Journal of the Acoustical Society of America 119 (2006) [19] M. Grassi: Do we hear size or sound: Balls dropped on plates. Perception and Psychophysics 67 (2005) [20] J. H. Howard Jr., J. A. Ballas: Perception of simulated propeller cavitation. Human Factors 25 (1983) [21] M. Kac: Can one hear the shape of adrum? Mathematics Monthly 13 (1966) [22] A. J. Kunkler-Peck, M. T. Turvey: Hearing shape. Journal of Experimmental Psychology: Human Perception and Performance 26 (2000) [23] S. Lakatos, S. McAdams, R. Caussé: The representation of auditory source characteristics: Simple geometric form. Perception and Psychophysics 59 (1997) [24] R. A. Lutfi: Auditory detection of hollowness. Journal of the Acoustical Society of America 110 (2001) [25] R. A. Lutfi,E.L.Oh: Auditory discrimination of material changes in astruck-clamped bar. Journal of the Acoustical Society of America 102 (1997) [26] S. McAdams, A. Chaigne, V. Roussarie: The psychomechanics of simulated sound sources: Material properties of impacted bars. Journal of the Acoustical Society of America 115 (2004) [27] N. J. VanDerveer: Ecological acoustic: Human perception of environmental sounds. Dissertation Abstracts International 40 (1979) 4543B (University Microfilms No ). 1020

13 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA [28] W. H. Warren Jr., R. R. Verbrugge: Auditory perception of breaking and bouncing events: Acase study in ecological acoustics. Journal of Experimental Psychology: Human Perception and Performance 10 (1984) [29] R. P. Wildes, W. A. Richards: Recovering material properties from sound. In: Natural computation. W. Richards (ed.). MIT Press, Cambridge, MA, 1988, [30] R. L. Klatzky, D. K. Pai, E. P. Krotkov:Perception of material from contact sounds. Presence: Teleoperators and Virtual Environments 9 (2000) [31] S. W. Coward, C. J. Stevens: Extracting meaning from sound: Nomic mappings, everyday listening, and perceiving object size from frequency. Psychological Record 54 (2004) [32] B. Cipra: Youcan thear the shape of adrum. Science 225 (1992) [33] C. Gordon, D. Webb: Youcan t hear the shape of adrum. American Scientist 84 (1996) [34] B. H. Repp: The sound of two hands clapping: An exploratory study. Journal of the Acoustical Society of America 81 (1987) [35] X. Li, R. J. Logan, R. E. Pastore: Perception of acoustic source characteristics: Walking sounds. Journal of the Acoustical Society of America 90 (1991) [36] J. J. Gibson: The useful dimension of sensitivity. American Psychologist 18 (1963) [37] S. McAdams: Recognition of sound sources and events. In: Thinking in sound: The cognitive psychology of human audition. S. McAdams, E. Bigand (eds.). Clarendon Press, Oxford, 1993, [38] J. J. Gibson: The senses considered as perceptual systems. Houghton Miffl in, Boston, [39] J. J. Gibson: The ecological approach to visual perception. Houghton Miffl in, Boston, [40] J. J. Jenkins: Acoustic information for places, objects, and events. In: Persistence and change: Proceedings of the First International Conference on Event Perception. W. H. Warren, R. E. Shaw (eds.). Erlbaum, Hillsdale, NJ, 1985, [41] C. F. Michaels, C. Carello: Direct perception. Prentice- Hall, Englewood Cliff s, NJ, [42] R. Shaw, J. Bransford: Introduction: Psychological approaches to the problem of knowledge. In: Perceiving acting and knowing: Toward an ecological psychology. R. Shaw, J. Bransford (eds.). Erlbaum, Hillsdale, NJ, 1977, [43] C. Fowler: An event approach to the study of speech perception from a direct realist perspective. Journal of Phonetics 14 (1986) [44] C. A. Fowler: Sound-producing sources as objects of perception: Rate normaliziation and nonspeech perception. Journal of the Acoustical Society of America 88 (1990) [45] C. A. Fowler: Auditory perception is not special: We see the world, we feel the world, we hear the world. Journal of the Acoustical Society of America 89 (1991) [46] M. I. Posner, M. J. Nissen, R. M. Klein: Visual dominance: An information-processing account of its origins and significance. Psychological Review 83 (1976) [47] H. N. J. Schiff erstein: The perceived importance of sensory modalities in product usage: A study of self-reports. Acta Psychologica 121 (2006) [48] C. Spence, D. I. Shore, R. M. Klein: Multisensory prior entry. Journal of Experimental Psychology: General 130 (2001) [49] J. Blauert, U. Jekosch: Sound-quality evaluation A multi-layered problem. Acustica united with Acta Acustica 83 (1997) [50] W. Keiper: Sound quality evaluation in the product cycle. Acustica united with Acta Acustica 83 (1997) [51] T. Lageat, S. Czellar, G. Laurent: Engineering hedonic attributes to generate perceptions of luxury: Consumer perception of an everyday sound. Marketing Letters 14 (2003) [52] D. A. Norman: Emotional design: Why welove (or hate) everyday things. Basic Books, New York, [53] S. J. Orfield: Perception and product design. Appliance Manufacturer 43 (1995) [54] E. E. Ungar: Quality sound quality. Sound and Vibration 28 (1994) 5. [55] H. Vander Auweraer, K. Wyckaert, W. Hendricx: From sound quality to the engineering of solutions for NVH problems: Case studies. Acustica united with Acta Acustica 83 (1997) [56] V. Jousmäki, R. Hari: Parchment-skin illusion: Soundbiased touch. Current Biology 8 (1998) [57] S. Carlin, W. D. Ward, A. Gershon, R. Ingraham: Sound stimulation and its eff ect on dental sensation threshold. Science 138 (1962) [58] G. A. Gescheider,W.G.Barton, M. R. Bruce, J. H. Goldberg, M. J. Greenspan: Eff ects of simultaneous auditory stimulation on the detection of tactile stimuli. Journal of Experimental Psychology 81 (1969) [59] G. A. Gescheider, R. K. Niblette: Cross-modality masking for touch and hearing. Journal of Experimental Psychology 74 (1967) [60] E. Jacobson: Experiments on the inhibition of sensations. Psychological Review 18 (1911) [61] J.-P. Bresciani, M. O. Ernst, K. Drewing, G. Bouyer, V. Maury,A.Kheddar: Feeling what you hear: Auditory signals can modulate tactile tap perception. Experimental Brain Research 162 (2005) [62] S. Donohue, S. Sur, C. Moore, M. Merzenich: Touching sounds: Auditory induction of aphantom tactile illusion. Journal of Cognitive Neuroscience 14 (2002)(Supp.)172. [63] H. Gillmeister, M. Eimer: Tactile enhancement of perceived loudness. Paper presented at The 5th Meeting of the International Multisensory Research Forum, Sitges, Spain, 2-5th June [64] N. Kitagawa, Y. Igarashi: Subjective experience of touch induced by hearing asound. Poster presented at The 4th Meeting of the International Multisensory Research Forum, Hamilton, Canada, 2-5th June [65] N. Kitagawa, M. Zampini, C. Spence: Audiotactile interactions in near and far space. Experimental Brain Research 166 (2005) [66] M. Schürmann, G. Caetano, V. Jousmäki, R. Hari: Hands help hearing: Facilitatory audiotactile interaction at low sound-intensity levels. Journal of the Acoustical Society of America 115 (2004) [67] N. Kitagawa, C. Spence: Audiotactile interactions in information processing. Japanese Psychological Research in press. [68] C. Spence, S. Guest, J. Chan, D. Lloyd, F. McGlone, N. Phillips, L. A. Jones: Fooling the senses. Brochure produced for The Royal Society New Frontiers in Science Exhibition,

14 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception [69] S. Guest, C. Catmur, D. Lloyd, C. Spence: Audiotactile interactions in roughness perception. Experimental Brain Research 146 (2002) [70] E. Terhardt: On the perception of periodic sound fluctuations (roughness). Acustica 30 (1974) [71] P. Bertelson, G. Aschersleben: Automatic visual bias of perceived auditory location. Psychonomic Bulletin and Review 5 (1998) [72] A. Caclin, S. Soto-Faraco, A. Kingstone, C. Spence: Tactile capture of audition. Perception and Psychophysics 64 (2002) [73] P. von Schiller: Die Rauhigkeit als intermodale Erscheinung (Roughness as an intermodal phenomenon). Zeitschrift für Psychologie Bildung 127 (1932) [74] R. Plomp, W. J. M. Levelt: Tonal consonance and critical bandwidth. Journal of the Acoustical Society of America 38 (1965) [75] H. L. F. von Helmholtz: On the sensations of tone as the physiological basis for the theory of music (2nd ed. trans. A. J. Ellis, from German 4th ed. and 1885). Reprinted, Dover, NewYork, [76] W. von Aures: Ein Berechnungsverfahren der Rauhigkeit [A roughness calculation method]. Acustica 58 (1985) [77] P. Daniel, R. Weber: Psychoacoustical roughness: Implementation of an optimized model. Acustica united with Acta Acustica 83 (1997) [78] D. Pressnitzer, S. McAdams: Twophase eff ects in roughness perception. Journal of the Acoustical Society of America 105 (1999) [79] E. Terhardt: Über akustische Rauhigkeit und Schwankungsstärke [On acoustic roughness and the strength of oscillation/fluctuation]. Acustica 20 (1968) [80] U. Widmann, H. Fastl: Calculating roughness using timevarying specific loudness spectra. Proceedings of the Sound Quality Symposium 98, 1998, [81] E. Zwicker, H. Fastl: Psychoacoustics, facts and models. Springer-Verlag, Berlin, [82] R. W. Wendahl: Some parameters of auditory roughness. Folia Phoniatrica 18 (1966) [83] W. M. Bergmann Tiest, A. M. L. Kappers: Analysis of haptic perception of materials by multidimensional scaling and physical measurements of roughness and compressibility. Acta Psychologica 121 (2006) [84] L. A. Stone: Subjective roughness and smoothness for individual judges. Psychonomic Science 9 (1967) [85] S. J. Lederman, R. L. Klatzky: Multisensory texture perception. In: The handbook of multisensory processes. G. A. Calvert, C. Spence, B. E. Stein (eds.). MIT Press, Cambridge, MA, 2004, [86] I. E. Gordon, C. Cooper: Improving one s touch. Nature 256 (1975) [87] S. J. Lederman: Heightening tactile impressions of surface texture. In: Active touch: The mechanism of recognition of objects by manipulation. An interdisciplinary approach. G. Gordon (ed.). Pergamon Press, Oxford, 1978, [88] S. J. Lederman: Improving one s touch...and more. Perception and Psychophysics 24 (1978) [89] D. DiFilippo, D. K. Pai: The AHI: An audio and haptic interface for contact interactions. UIST 00 (13th Annual ACM Symposium on User Interface Software and Technology). Proceedings of ACM UIST 2000, San Diego, CA, [90] M. R. McGee, P. Gray, S. Brewster: The eff ective combination of haptic and auditory textural information. In: First international workshop on haptic human-computer-interaction. S. Brewster, R. Murray-Smith (eds.). Springer-Verlag, Berlin, 2000, [91] R. M. Miner, B. Gillespie, T. Caudell: Examining the influence of audio and visual stimuli on ahaptic display. Proceedings of the 1996 IMAGE Conference, Phoenix, AZ, June, [92] K. van den Doel, P. G. Kry, D. K. Pai: Foleyautomatic: Physically-based sound eff ects for interactive simulation and animation. Computer Graphics (ACM SIGGRAPH 2001 Conference Proceedings), August 2001, [93] K. van den Doel, D. K. Pai: The sounds of physical shapes. Presence: Teleoperators and Virtual Environments 7 (1998) [94] M. A. Zahariev, C. L. MacKenzie: Auditory, graphical and haptic contact cues for areach, grasp, and place task in an augmented environment. Proceedings of the 5th International Conference on Multimodal Interfaces, 2003, [95] T. Hempel, E. Altınsoy: Multimodal user interfaces: Designing media for the auditory and the tactile channel. In: Handbook of human factors in web design. R. W. Proctor, K.-P. L. Vu(eds.). Erlbaum, Mahwah, NJ, 2005, [96] S. J. Lederman, R. L. Klatzky, C. Hamilton, T. Morgan: Integrating multimodal information about surface texture via aprobe: Relative contributions of haptic and touchproduced sound sources. Proceedings of the 10th Annual Symposium on Haptic Interfaces for Teleoperators and Virtual Environment Systems, IEEE VR 02, 2002, [97] S. J. Lederman, R. L. Klatzky, A. Martin, C. Tong: Relative performance using haptic and/or touch-produced auditory cues in aremote absolute texture identification task. Proceedings of the 11th Annual Haptics Symposium for Virtual Environments and Teleoperator Systems, IEEE VR 03, 2003, [98] D. Rocchesso, F. Fontana: The sounding object. Mondo Estremo, Firenze, Italy,2003. [99] C. Galera-Garcia: Vibratory stimulation to the teeth as a communication aid for profoundly deaf persons. International Journal of Neuroscience 21 (1983) [100] C. Galera-Garcia, M. Nombela-Gomez: Sensorial substitution using sound-vibratory stimuli on the teeth: A new approach to the rehabilitation of the profoundly deaf. International Journal of Neuroscience 49 (1989) [101] G. Lundborg, B. Rosén, S. Lindberg: Hearing as asubstitution for sensation: A new principle for artificial sensibility. Journal of Hand Surgery 24A (1999) [102] E. Kohler, C. Keysers, M. A. Umiltà, L. Fogassi, V. Gallese, G. Rizzolatti: Hearing sounds, understanding actions: Action representation in mirror neurons. Science 297 (2002) [103] A. Gabrielsson: Problems and methods in judgments of perceived sound quality. Journal of the Acoustical Society of America 70 (1981) (supp.) S64. [104] A. Gabrielsson, U. Rosenberg, H. Sjögren: Judgments and dimension analyses of perceived sound quality of soundreproducing systems. Journal of the Acoustical Society of America 55 (1974) [105] A. Gabrielsson, H. Sjögren: Perceived sound quality of sound-reproducing systems. Journal of the Acoustical Society of America 65 (1979)

15 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA [106] C. L. Fog: Use of product sound optimization tools. Sixth International Congress on Sound and Vibration, Lyngby, Denmark, 5-8th July,1999, [107] J. Poggenburg, K. Genuit: How we may succeed in product sound quality. Sixth International Congress on Sound and Vibration, Lyngby, Denmark, 5-8 July 1999, 82. [108] E. Altinsoy, G. Kanca, H. T. Belek, A. S. Senur: Acomparative study on the sound quality of wet-and-dry type vacuum cleaners. Sixth International Congress on Sound and Vibration, Lyngby, Denmark, 5-8th July 1999, [109] M. Bodden, H. Iglseder: Active sound design: Vacuum cleaner. Revista de Acustica (special issue Forum Acusticum, Sevilla, Spain) 33 (2002). [110] F. Guyot, P. Christine, M. Castellengo, B. Fabre: Characterization of the sound quality of vacuum cleaners. Journal of the Acoustical Society of America 97 (1995) [111] G. C. Lauchle, T. A. Brungart: Fannoise in vacuum cleaners. Appliance Design 49 (2001) [112] H. S. Sagoo: Reduction of noise levels in vacuum cleaners. Unpublished PhD thesis, Aston University, UK, [113] C. Churchill, S. Maluski, T. Cox: Simplified sound quality assessment for UK manufacturers. 33rd International Congress and Exposition on Noise Control Engineering, 2004, 1 7. [114] J. Blauert, M. Bodden: Gütebeurteilung von Geräuschen Waren ein Problem? (Evaluation of sound quality Where is the problem?). In: Sound-engineering. Kundenbezogene Akustikentwicklung in der Fahrzeugtechnik. Q.-H. Vo (ed.). Expert-Verlag, Renningen-Malmshein, 1994, 1 9. [115] M. A. Milošević, A. M. Mitić, M. S. Milošević : Parameters influencing noise estimation. Facta Universitatis: Working and Living Environmental Protection 2 (2004) [116] T. Baird, N. Otto, W. Bray, M. Stephan: Impulsive noise of printers: Measurement metrics and their subjective correlation. Paper presented at The Joint meeting of ASA and NoiseCon, [117] R. D. Hellweg: Acceptability of noises from offi ce machines. Sound Quality Symposium, 1998, [118] P. W. Jordan: Designing pleasurable products: An introduction to the new human factors. Taylor and Francis, London, [119] P. W. Jordan, H. Engelen: Sound design for consumer products. Proceedings of Stockholm, hey listen!, Royal Swedish Music Academy, Stockholm, 1988, [120] V. Packard: The hidden persuaders. Penguin Books, Harmondsworth, Middlesex, [121] R. H. Lyon: Product sound quality From perception to design. Sound and Vibration, March [122] M. B. Sapherstein: The trademark registrability of the Harley-Davidson roar: Amultimedia analysis. bc_org/ avp/ law/ st_org/ iptf/ articles/ content/ html, [123] M. Lindstrom: Brand sense: How tobuild brands through touch, taste, smell, sight and sound. Kogan Page, London, [124] W. Keiper: Psychoacoustics in industry: Needs and benefits. Acustica united with Acta Acustica 85 (1999) [125] A. Miś kiewicz, T. Letowski: Psychoacoustics in the automotive industry. Acustica united with Acta Acustica 85 (1999) [126] M. Allman-Ward, R. Williams, G. Dunne, P. Jennings: The evaluation of vehicle sound quality using an NVH simulator. Paper presented at The 33rd International Congress and Exposition on Noise Control Engineering, Prague, Czech Republic, August 2004, [127] H. Doi, M. Koike: Development of new index capable of optimally representing automobile aerodynamic noise. Mitsubishi Motors Technical Review 14 (2002) [128] P. Jennings, J. Fry, G. Dunne, R. Williams: Predicting customers evaluations of new vehicle sounds. Paper presented at The Tenth International Congress on Sound and Vibration, Stockholm, Sweden, 7-10 July [129] R. Hoeldrich: An optimized model for the objective assessment of roughness sensations in vehicle noise. Paper presented at AVL Sound Engineering Conference, Graz, July [130] L. Sang-Kwon, K. Byung-Soo, P. Dong-Chul: Objective evaluation of the rumbling sound in passenger cars based on an artificial neural network. Proceedings of the I MECH EPart D, Journal of Automobile Engineering, D4, 2005, [131] T. G. Filippou, H. Fastl, S. Kuwano, S. Namba, S. Nakamura, H. Uchida: Door sound and image of cars. Fortschr. Akust. DAGA 03, Dt. Ges. f. Akustik, Oldenburg, [132] H. Fastl: Psycho-acoustics and sound quality. In: Communication acoustics. J. Blauert (ed.). Springer, Berlin, Germany, 2005, [133] K. Kanie, Y. Kurono, Y. Nagata, I. Koori: Vehicle door design based on sounds and noise control. JSAE Review 8 (1987) [134] S. Kuwano, H. Fastl, S. Namba, S. Nakamura, H. Uchida: Subjective evaluation of car door sound. Proceedings of the Sound-Quality Symposium, Dearborn MI, [135] R. H. Lyon: Asound guide to product acceptance. The Industrial Physicist 4 (1998) [136] D. E. Malen: Engineering for the customer: Decision methodology for preliminary design. Unpublished PhD thesis, University of Michigan, [137] D. E. Malen, R. A. Scott: Improving automobile doorclosing sound for customer preference. Noise Control Engineering Journal 41 (1993) [138] Talking heads. The Sunday Times, April 27th (4), 12, [139] G. Lemaitre, P. Susini, S. Winsberg, S. McAdams: Perceptively based design of new car horn sounds. Proceedings of the 2003 International Conference on Auditory Display, Boston, MA, USA, July 6-9, 2003, ICAD03 47 ICAD [140] How acar s horn says: Buy me. go/pr/ fr/-/ 2/hi/ uk_news/ wales/ stm, [141] Y. Suzuki, K. Abe, K. Ozawa, T. Sone: Factors for perceiving sound environments and the eff ects of visual and verbal information on these factors. In: Contributions to psychological acoustics: Results of the 8th Oldenburg symposium on psychological acoustics. A. Schick, C. Reckhardt (eds.). BIS, Oldenburg, 2000, [142] U. Jekosch: Basic concepts and terms of quality, reconsidered in the context of product-sound quality. Acta Acustica united with Acustica 90 (2004) [143] U. Jekosch: Assigning meaning to sounds Semiotics in the context of product-sound design. In: Communication acoustics. J. Blauert (ed.). Springer, Berlin, 2005, [144] D. Västfjäll: Contextual influences on sound quality evaluation. Acta Acustica united with Acustica 90 (2004)

16 ACTA ACUSTICA UNITED WITH ACUSTICA Spence, Zampini: Auditory contributions to product perception [145] N. Chouard, T. Hempel: Asemantic diff erential design especially developed for the evaluation of interior car sounds. Journal of the Acoustical Society of America 105 (1999) [146] K. Genuit: The future of sound quality of the interior noise of vehicles. Proceedings of Internoise 2000, Nice, France, 2000, [147] T. Hempel, N. Chouard: Evaluation of interior car sounds with anew specific semantic diff erential design. Journal of the Acoustical Society of America 105 (1999) [148] S. Maluski, C. Churchill, T. J. Cox: Sound quality testing and labelling of domestic appliances in the UK. 33rd International Congress and Exposition on Noise Control Engineering, Prague, Czech Republic, August 2004, 1 8. [149] P. Susini, S. McAdams, S. Winsberg: Perceptual characterisation of vehicular noises. EEA symposium: Psychoacoustics in industry and universities, IRCAM Centre Georges-Pompidou, Paris, January, [150] P. Susini, N. Misdariis, S. Winsberg, S. McAdams: Caractérisation perceptive de bruits (Perceptual characterization of sounds). Acoustique et Techniques 13 (1998) [151] P. Susini, S. McAdams, S. Winsberg, I. Perry, S. Vieillard, X. Rodet: Characterizing the sound quality of airconditioning noise. Applied Acoustics 65 (2004) [152] M. Zampini, S. Guest, C. Spence: The role of auditory cues in modulating the perception of electric toothbrushes. Journal of Dental Research 82 (2003) [153] M. Zampini, C. Spence: Assessing the influence of auditory cues on the perception of the forcefulness and pleasantness of aerosol sprays Manuscript submitted for publication. [154] A. Kopinska, L. R. Harris: Simultaneity constancy. Perception 33 (2004) [155] M. O. Ernst, M. S. Banks: Humans integrate visual and haptic information in astatistically optimal fashion. Nature 415 (2002) [156] R. B. Welch, D. H. Warren: Immediate perceptual response to intersensory discrepancy. Psychological Bulletin 3 (1980) [157] P. W. Battaglia, R. A. Jacobs, R. N. Aslin: Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A 20 (2003) [158] R. Froman: Marketing research, you get what you want. In: Readings in marketing. J. H. Westing (ed.). Prentice- Hall, New York, [159] D. F. Cox: The sorting rule model of the consumer product evaluation process. In: Risk taking and information handling in consumer behavior. Graduate School of Business Administration, Harvard University, Boston, MA, 1967, [160] G. F. Boyd: Auditory irritants and impalpable pain. Journal of General Psychology 60 (1959) [161] The silent pc soundscapes markets for acoustic comfort [162] H. Engelen: Sound design for consumer electronics. http: engelen.html, [163] R. H. Lyon: Designing for product sound quality. Marcel Dekker, New York, [164] C. Patsouras, H. Fastl, D. Patsouras, K. Pfa ff elhuber: How faristhe sound quality of adiesel powered car away from that of agasoline powered one? Proceedings of Forum Acusticum, Sevilla, 2002, CD ROM NOI IP. [165] E. A. Björk: The perceived quality of natural sounds. Acustica 57 (1985) [166] R. Guski: Psychological methods for evaluating sound quality and assessing acoustic information. Acustica united with Acta Acustica 83 (1997) [167] K. Okakura: The book of tea. Kodansha International, London, [168] B. K. Drake: Food crunching sounds. An introductory study. Journal of Food Science 28 (1963) [169] Z. M. Vickers: Crispness and crunchiness -textural attributes with auditory components. In: Food texture: Instrumental and sensory measurement. H. R. Moskowitz (ed.). Dekker,New York, 1987, [170] Z. Vickers: Sound perception and food quality. Journal of Food Quality 14 (1991) [171] Z. Vickers, M. C. Bourne: Crispness in foods -Areview. Journal of Food Science 41 (1976) [172] D. Kilcast: Sensory techniques to study food texture. In: Food texture: Measurement and perception. A. J. Rosenthal (ed.). Aspen, Gaithersburg, MD, 1999, [173] H. Luyten, J. Plijter, T. Van Vliet: Understanding the sensory attributes crispy and crunchy: An integrated approach. In: Proceedings of the 3rd International Symposium on Food Rheology and Structure. P. Fischer, I. Marti, E. J. Windhab (eds.). Zurich, 2003, [174] L. Duizier: Areviewofacoustic research for studying the sensory perception of crisp, crunchyand crackly textures. Trends in Food Science and Technology12 (2001) [175] G. Roudaut, C. Dacremont, B. Valles Pamies, C. B., M. Le Meste: Crispness: Acritical review onsensory and material science approaches. Trends in Food Science and Technology 13 (2002) [176] J. V. Verhagen, L. Engelen: The neurocognitive bases of human multimodal food perception: Sensory integration. Neuroscience and Biobehavioral Reviews 30 (2006) [177] M. Zampini, C. Spence: The role of auditory cues in modulating the perceived crispness and staleness of potato chips. Journal of Sensory Science 19 (2004) [178] Listen, this food is music to your ears. The Sunday Times, August 29th, News (1),11, [179] The crunch factor. The Guardian (Magazine), February 21st [180] C. Dacremont, B. Colas, F. Sauvageot: Contribution of airand bone-conduction to the creation of sounds perceived during sensory evaluation of foods. Journal of Texture Studies 22 (1992) [181] Z. M. Vickers: Crispness and crunchiness -Adi ff erence in pitch? Journal of Texture Studies 15 (1984) [182] Z. M. Vickers: The relationships of pitch, loudness and eating technique to judgments of the crispness and crunchiness of food sounds. Journal of Texture Studies 16 (1985) [183] Z. M. Vickers, S. S. Wasserman: Sensory qualities of food sounds based on individual perceptions. Journal of Texture Studies 10 (1979) [184] P. Courcoux, L. Chaunier, G. DellaValle, D. Lourdin, M. Séménou: Paired comparisons for the evaluation of crispness of cereal flakes by untrained assessors: Correlation with descriptive analysis and acoustic measurements. Journal of Chemometrics 19 (2005) [185] J. Chen, C. Karlsson, M. Povey: Acoustic envelope detector for crispness assessment of biscuits. Journal of Texture Studies 36 (2005)

17 Spence, Zampini: Auditory contributions to product perception ACTA ACUSTICA UNITED WITH ACUSTICA [186] J. Chen, C. Karlsson, M. Povey: Crispness assessment of roasted almonds by an integrated approach to texture description: Texture, acoustics, sensory and structure. Journal of Chemometrics in press. [187] C. Dacremont: Spectral composition of eating sounds generated by crispy, crunchy and crackly foods. Journal of Texture Studies 26 (1995) [188] Z. Vickers, M. C. Bourne: A psychoacoustical theory of crispness. Journal of Food Science 41 (1976) [189] J. D. Markel, A. H. Gray: Linear prediction of speech. Springer Verlag, New York, [190] T. W. Parks, C. S. Burrus: Digital filter design. John Wiley and Sons, Chichester,1987. [191] J. O. Smith: Introduction to digital filters. Center for Computer Research in Music and Acoustics (CCRMA), Stanford University. jos/filters05/, [192] Y. Hua, T. K. Sarkar: Matrix pencil method for estimating parameters of exponentially damped/undamped sinusoids in noise. IEEE Transactions on Acoustics, Speech, and Signal Processing 38 (1990) [193] M. Zampini, C. Spence: Modifying the multisensory perception of a carbonated beverage using auditory cues. Food Quality and Preference 16 (2005) [194] V. Walsh, J. Kulikowski: Perceptual constancy: Why things look as they do. Cambridge University Press, Cambridge, UK, [195] Z. M. Vickers: Sensory,acoustical, and force-deformation measurements of potato chip crispness. Journal of Food Science 52 (1987) [196] J. C. Cooper: A tutorial on lip sync errors, the sources and solutions. Pixel Instruments Corporation, Los Gatos, CA, [197] R. L. Brown: Wrapper influence on the perception of freshness in bread. Journal of Applied Psychology 42 (1958) [198] S. Bradley: Garments for the 5senses. Presented at Sense and Sensuality 2006, London, [199] Sound of clothes: Blow, clap, talk and hum [200] R. Wolkomir: Decibel by decibel, reducing the din to a very dull roar. Smithsonian Magazine (February 1996) [201] R. Gobbelé, M. Schürrmann, N. Forss, K. Juottonen, H. Buchner, R. Hari: Activation of the human posterior and tempoparietal cortices during audiotactile interaction. Neuroimage 20 (2003) [202] S. Levänen, V. Jousmäki, R. Hari: Vibration-induced auditory-cortex activation in a congenitally deaf adult. Current Biology 8 (1998) [203] C. E. Schroeder, R. W. Lindsley, C. Specht, A. Marcovici, J. F. Smiley, D. C. Javitt: Somatosensory input to auditory association cortex in the macaque monkey. Journal of Neurophysiology 85 (2001) [204] G. vonbékésy: Neural volleys and the similarity between some sensations produced by tones and by skin vibrations. Journal of the Acoustical Society of America 29 (1957) [205] G. vonbékésy: Similarities between hearing and skin sensations. Psychological Review 66 (1959) [206] M. M. Murray, S. Molholm, C. M. Michel, D. J. Heslenfeld, W. Ritter, D. C. Javitt, C. E. Schroeder, J. J. Foxe: Grabbing your ear: Auditory-somatosensory multisensory interactions in early sensory cortices are not constrained by stimulus alignment. Cerebral Cortex 15 (2005) [207] A. A. Ghazanfar, C. E. Schroeder: Is neocortexessentially multisensory? Trends in Cognitive Sciences 10 (2006) [208] H. N. J. Schiff erstein, M. P. H. D. Cleiren: Capturing product experiences: A split-modality approach. Acta Psychologica 118 (2005) [209] K. Abe, K. Ozawa, Y. Suzuki, T. Sone: The eff ects of visual information on the impression of environmental sounds. Proceedings of Internoise 99, [210] C. Dacremont, B. Colas: Eff ect of visual clues on evaluation of bite sounds of foodstuff s. Sciences des Aliments 13 (1993) [211] H. Fastl: Audio-visual interactions in loudness evaluation. 18th International Congress on Acoustics, ICA, Kyoto, 2004, [212] T. Hashimoto, S. Hatanoa: Eff ects of factors other than sound to the perception of sound quality. Proceedings of the International Congress on Acoustics, Rome, 2-7 September [213] S. Iwamiya: The interaction between auditory and visual processing when listening to music via audio-visual media. Journal of the Acoustical Society of Japan 48 (1992) [214] C. Patsouras, T. G. Filippou, H. Fastl: Influences of color on the loudness judgment. sea/ sevilla02/ psy05002.pdf. [215] A. Tamura: E ff ects of landscaping on the images of a space. In: Contributions to psychological acoustics: Results of the 7th Oldenburg symposium on psychological acoustics. A. Schick, M. Klatte (eds.). BIS, Oldenburg, 1997, [216] D. Västfjäll, P. Larsson, M. Kleiner: Cross-modal interaction in sound quality evaluation: Experiment using the virtual aircraft. Journal of Sound and Vibration in press. [217] X. Liu, J. Tan: Acoustic wave analysis for food crispness evaluation. Journal of Texture Studies 30 (2000) [218] J. Alpiner, P.McCarthy: Rehabilitative audiology: Children and adults. Williams and Wilkins, Baltimore, [219] J. F. Corso: Sensory processes and age eff ects in normal adults. Journal of Gerontology 26 (1971) [220] Aging america, trends and projections. US Senate Special Committee on Aging (in association with the American Association of Retired Persons, the Federal Council on the Aging, and the Administration on Aging), [221] J. J. Pirkl:Transgenerational design; products for an aging population. VanNostrand Reinhold, New York, [222] A. S. MacDonald: The scenario of sensory encounter: Cultural factors in sensory-aesthetic experience. In: Pleasure with products. W. S. Green, P. W. Jordan (eds.). Taylor and Francis, London, 2002, [223] D. Howes: The varieties of sensory experience: Asourcebook in the anthropology of the senses. University of Toronto Press, Toronto,