What Audio Engineers Should Know About Human Sound Perception. Part 2. Binaural Effects and Spatial Hearing

Transcription

1 What Audio Engineers Should Know About Human Sound Perception Part 2. Binaural Effects and Spatial Hearing AES 112 th Convention, Munich AES 113 th Convention, Los Angeles Durand R. Begault Human Factors Research & Technology Division NASA Ames Research Center Moffett Field, California

2 Overview ILD, ITD differences and lateralization HRTF spectral changes for 3D imagery Binaural versus monaural influence of echoes Effects of reverberation on perception of the environmental context Cues to auditory distance Cognitive and multisensory cues

3 Communication chain for acoustic events Sound source(s), interaction with room acoustics SOURCE MEDIUM RECEIVER Frequency Amplitude Spectrum Location

4 Communication chain for acoustic events Sound source(s), interaction with room acoustics Recording & playback: acousticalelectrical- acoustical transformation SOURCE MEDIUM RECEIVER Frequency Amplitude Spectrum Location

5 Communication chain for acoustic events Sound source(s), interaction with room acoustics Recording & playback: acousticalelectrical- acoustical transformation Hearing: perception, cognition, multisensory interaction SOURCE MEDIUM RECEIVER Frequency Amplitude Spectrum Location Pitch Loudness Timbre Localization

6 Communication chain for acoustic events Sound source(s), interaction with room acoustics Recording & playback: acousticalelectrical- acoustical transformation Hearing: perception, cognition, multisensory interaction SOURCE MEDIUM RECEIVER Mismatch between prescribed & perceived spatial events

7 Psychologically-driven Acoustic signal-driven Model of the binaural hearing system Binaural hearing (localization; signal separation & detection): forming spatial auditory events from acoustical (bottom-up) and psychological (top-down) inputs Figure adapted from Jens Blauert, Spatial Hearing. The Pychophysics of Human Sound Localization. Revised Edition. 1983, MIT Press.

8 Model of the binaural hearing system Binaural hearing (localization; signal separation & detection) Filtering of acoustic signal by pinnae, ear canal

9 Model of the binaural hearing system Binaural hearing (localization; signal separation & detection) Filtering by inner ear; frequency-specific neuron firings Filtering of acoustic signal by pinnae, ear canal

10 Model of the binaural hearing system Binaural hearing (localization; signal separation & detection) Physiological evaluation of interaural timing and level differences Filtering by inner ear; frequency-specific neuron firings Filtering of acoustic signal by pinnae, ear canal

11 Psychologically-driven Acoustic signal-driven Model of the binaural hearing system Multi-sensory information; cognition Binaural hearing (localization; signal separation & detection) Physiological evaluation of interaural timing and level differences Filtering by inner ear; frequency-specific neuron firings Filtering of acoustic signal by pinnae, ear canal.

12 Two important functions of the binaural hearing system for recording engineers: Localization (lateral and 3-dimensional) Binaural masking: Echo supression, room perception

13 Lateral localization of auditory images Duplex theory of localization ILD (interaural level difference) ITD (interaural time difference)

14 Lateral spatial image shift ILD (interaural level difference) caused by head shadow of wavelengths > 1.5 khz Level difference (db) Level difference (db)

15 Perceptual decoding of spatial cues in a cross-coincident microphone recording is based on ILDs rotation

16 Lateral image shift ITD (interaural time difference)

17 Lateralization demo. A simple time or level difference can make headphone images move from side to side inside the head. Lateral shift from center of the head Interaural level difference (db) (max) 0 (center) Interaural time difference (msec) Adapted from Toole & Sayers, 1965 and Blauert, 1983: click stimuli Adapted from Blauert, 1983: broadband noise 1. ILD DEMO: 2 db 4 db 6 db 8 db 12 db 2. ITD DEMO: 0.00 ms 0.25 ms 0.50 ms 0.75 ms 1.00 ms 1.50 ms

18 Elevation and front-back discrimination: HRTF, pinnae cues

19 The cone of confusion causes reversals for virtual sources with identical or near-identical ITD or ILD Source Left Lis t ener Exte rnaliz e d pe rc e pt ion Source Left 3 0

20 Head-related transfer function cues (HRTFs) provide cues for front-back discrimination and elevation 10 Right 30, elevated 0 Log Magnitude (db) -10 Right 90, ear level -20 Right 120, below Frequency 3. audio example: HRTF clock positions 45, 0 135, 0

21 Variation in HRTF magnitude with elevation at one azimuth 4. Audio example: 120 degree azimuth: at +36, 0, -36 degrees elevation Graphic by William L. Martens, University of Aizu

22 Perceptual errors with headphone 3-D sound include inside-the-head localization (solution: reverberation cues) and reversals (solution: head tracking) TARGET POSITION REVERSAL ERROR WITH LOCALIZATION ERROR (ELEVATION) TARGET POSITION REGION OF LOCALIZATION ERROR (AZIMUTH) INTRACRANIAL LOCATION (DISTANCE ERROR) REVERSAL ERROR

23 Localization error for headphone stimuli (azimuth) Anechoic Speech: Individual differences 30 Mean values for different reverberation conditions Unsigned azimuth error (degrees) anechoic early reflections full auralization reverberation treatment

24 Echoes, reverberation and background sound: perception of the environmental context

25 Spatial hearing fundamentally involves perception of the location of a sound source at a point in space (azimuth, elevation, distance). Listener Distance Image Size Elevation But a sound source simultaneously reveals information about its environmental context. Azimuth -reverberation -image size & extent Environmental Context

26 Effect of delay time for a single echo image shift image broadening echo Approximate delay time to left channel (msec) Sound examples: 5. stereo echo- 6. monaural echo Relative to the reference condition, spatially separated echoes create spatial percepts; non-spatially separated echoes create timbral effects

27 Early and late reverberant sound fields D R1 7. Audio examples: -direct sound -direct w/ 1st, 2nd order ERs -direct with full auralization R2 Relative amplitude Direct sound Early reflections Late reflections (dense reverberation) Time

28 Early and late reverberant sound fields 8. audio examples: normal and 0.25 speed impulse response -5 Relative amplitude Relative amplitude Direct sound Early reflections Late reflections (dense reverberation) Time x 1 0 4

29 Echo thresholds Sensitivity can increase as much as 10 db if echoes occur at different locations Late reverberation can decrease sensitivity Sensitivity increases with increasing time delay

30 Although thresholds for reverberation are relatively low, background noise (e.g., NC 35) can mask the reverberant decay Noise Criteria (NC) curves 60 speech NC 65 NC 60 Reverberation threshold (speech) re 60 db SPL Small Medium Large fbw fbw fbw reverberation threshold Approximate Threshold of Hearing for Continuous Noise NC 5 One-Third Octave Band Center Frequency, Hz NC 55 NC 50 NC 45 NC 40 NC35 NC 30 NC 25 NC 20 NC 15 NC k 2k 4k 8k Octave-Band Center Frequency (fbw=full bandwidth)

31 Distance perception: amplitude cues The inverse square law states that sound decays 6 decibels per doubling of distance in a reflection-free environment. db SPL ' 2 ' 4 ' 8 ' 9. sound example

32 Distance perception: amplitude cues However, half-as-loud corresponds to a 10 db reduction in level with distance db SPL ' 2 ' 4 ' 8 ' 10. sound example

33 Distance perception: reverberant ratio cues An increase in reverberant level indicates movement into the diffuse sound field Anechoic w/ ER w/ ER + LR distance (feet) 8 10

34 Concert Hall reverberation physicalperceptual parameters Reverberance (reverberation time, strength) Apparent source width (ASW) (interaural cross-correlation) Envelopment (spatial diffusion of reflections from all around) Clarity (ratio of first ms of early sound to late sound) Warmth (ratio of bass frequency RT to mid-band RT)

35 Cognitive cues; multisensory cues

36 Cognitive cues to distance perception Shouting Whispering

37 Auditory localization can be influenced or biased by cognitive mapping

38 Influence of visual, vibratory cues Helicopter fly-overs Explosions & crashes

39 Summary ILD, ITD differences and lateralization HRTF spectral changes for 3D imagery Binaural versus monaural influence of echoes Effects of reverberation on perception of the environmental context Cues to auditory distance Cognitive and multisensory cues