The eyes have it Face recognition, gaze perception, and the gaze aftereffect

Transcription

1 The eyes have it Face recognition, gaze perception, and the gaze aftereffect Dr. Malte Persike Dr. Yvonne Wiepen

2 Introduction Face Recognition Thompson, P. (1980). Margaret Thatcher: a new illusion. Perception, 9. Adaptation Face recognition a special(ized) ability?

3 Introduction Face Recognition Haxby, J. et al. (1994) The functional organization of human extrastriate cortex. The Journal of Neuroscience, 14. = Adaptation Viewing faces with activation while viewing nonsense pictures activates different cortical regions Face selective regions are located within the orbitofrontal cortex Area responds particularly energetic to beautiful faces and also to positive reward

4 Introduction Face Recognition Kanwisher, N. et al. (1997). The Fusiform Face Area: a module in human extrastriate cortex specialized for face perception. The Journal of Neuroscience, 17. Fusiform Face Area Adaptation Inferior occipital region Neurons in the inferior temporal cortex respond selectively to face stimuli The area is usually labeled Fusiform Face Area (FFA) Selective activation is also found in earlier occipital visual areas (inferior occipital cortex)

5 Introduction Face Recognition Haxby, J. et al. (1999). The effect of face inversion on activity in human neural systems for face and object perception. Neuron, 22. Adaptation Face recognition also involves neural regions in the Superior Temporal Sulcus (STS) Activation is similar for upright and inverted faces no neural correlate for the Thatcher illusion (yet)

6 Introduction Adaptation Face Recognition Hietanen, J. K., Perrett, D. I., Oram, M. W., Benson, P. J. & Dittrich, W. H. (1992). The effects of lighting conditions on responses of cells selective for face views in the macaque temporal cortex. Experimental Brain Research, 89. Neuronal responses in the STS are invariant to lighting, contrast, size Different populations encode different preferred views of faces

7 Introduction Face Recognition Somerville, L.H., Kim, H., Johnstone, T., Alexander, A.L., Whalen, P.J. (2004). Human amygdala responses during presentation of happy and neutral faces. Biological Psychiatry, 55. Adaptation =

8 Introduction Face Recognition Breiter, H. et al. (1996) Response and habituation of the human amygdala during visual processing of facial expression. Neuron, 17. = Adaptation The amygdala is activated by perception of fear in the face of another As such, the amygdala may be conceived as part of an extended face recognition system (Haxby, 2000)

9 Introduction Face Recognition Synopsis of cortical regions supposed to be involved in face perception Adaptation Superior Temporal Sulcus (STS) Static pictures of faces and bodies Movements (posture, hands, body) [Invariant to size, lighting etc.] Inferior temporal Cortex Fusiform Face Area (Kanwisher et al., 2004) Amgydala Emotional Facial expression (Breiter et al., 1996) Orbitofrontal Cortex Facial characteristics (Haxby et al., 2000)

10 Introduction Face Recognition But Gauthier, I. et al. (1999) Can face recognition really be dissociated from object recognition? Journal of Cognitive Neuroscience, 11. Adaptation

11 Introduction Face Recognition Psychophysics to the rescue There are neuron populations responding to face stimuli Adaptation Face sensitive neural circuits are located in spatially bounded regions of the cortex It is unclear whether these neural circuits operate selectively on face stimuli Neuroimaging methods are too limited in spatial and temporal resolution to provide definite proof about the existence of dedicated face recognition pathways

12 Introduction Face Recognition Adaptation a brief primer Adaptation Long-term exposition to an adapting stimulus modulates the percept of a probe stimulus The probe stimulus (i.e., vertical lines) appears tilted towards the opposite direction

13 Introduction Population Coding Why multiple eyes not only see more but better than one Nope. Nope. -5 Nope. Adaptation Nope Nope. 10 Yes. Probe Stimulus 15 Detectors Nope. Decision stage

14 Introduction Population Coding Why multiple eyes not only see more but better than one Nope. Nope. -5 Nope. Adaptation Nope Yes. winner take all rule 10 Yes. Probe Stimulus 15 Detectors Yes. Decision Stage

15 Introduction Population Coding Why multiple eyes not only see more but better than one Nope. - Nope. Adaptation How do individual detectors -5 Nope. respond to arbitrary stimulus levels? Nope How are signals from different detectors 5 Yes. integrated? all rule 10 Yes. winner take Probe Stimulus 15 Detectors Yes. Decision Stage

16 Introduction Population Coding The Tuning Curve Tuned Detectors Adaptation 12.2 Stimulus

17 Introduction Population Coding Integration of multiple signals Adaptation deg Tuned Detectors Responses to 12.2 stimulus Integration & Decision

18 Introduction Population Coding Adaption Tuned Detectors Adaptation 15 Adaption Probe Stimulus

19 Introduction Population Coding Adaption Adaptation deg Tuned Detectors Responses to 12.2 stimulus Integration & Decision

20 Introduction Adaptation to Faces Webster, M. A., Kaping, D., Mizokami, Y. & Duhamel, P. (2004). Adaptation to natural facial categories. Nature, 428. Adaptation Adaptation Male Indifference Female Long term exposition to faces shifts the indifference point towards the adapting stimulus Faces previously perceive as male are judged as more female (or vice versa, depending on adaptation gender)

21 Introduction Adaptation to Faces Yamashita, J. A., Hardy, J. L., De Valois, K. K. & Webster, M. A. (2005). Stimulus selectivity of figural aftereffects for faces. Journal of Experimental Psychology: HPP, 31. Adaptation Adaptation Distorted Normal Distorted + Long term exposition to distorted faces shifts the normal point towards the adapting stimulus Adaptation effects carry over to other faces (race, gender etc.) but not qualitatively different objects.

22 Introduction Face recognition Summary Adaptation There are neuron populations responding to face stimuli Face sensitive neural circuits are located in spatially bounded regions of the cortex Neurons It is unclear are whether selective these to face neural stimuli circuits but operate intermixed selectively with on face neural stimuli units responsible for the processing of other real-world objects (houses, animals ) Conclusion: There are specialized neuron populations for face perception in defined cortical regions but no dedicated cortical areas per se.

23 Experiments Horley, K. et al. (2003). Social phobics do not see eye to eye: A visual scanpath study of emotional expression processing. Journal of Anxiety Disorders, 17. Social phobics Normal subjects attend to face stimuli according to an inverted triangular scanpath Eyes are fixated longest and most often during saccading scanning Social phobics tend to avoid the eye region

24 Experiments Itier, R. J. et al. (2007). Eyes always attract attention but gaze orienting is taskdependent: Evidence from eye movement monitoring. Neuropsychologia, 45. Frontal view 3/4 view Task group A: Determine gaze direction ( Gaze task ). Task group B: Determine head orientation ( Head task ).

25 Experiments Itier, R. J. et al. (2007). Eyes always attract attention but gaze orienting is taskdependent: Evidence from eye movement monitoring. Neuropsychologia, 45. The eye region of faces is a strong attractor for fixations ( attentional capture ) Direction of saccades and fixation times are not fully automatic but task dependent to a certain degree

26 Bottom-up control of attention Langton, S., & Bruce, V. (1999). Reflexive visual orienting in response to the social attention of others. Cognition, 6 (5). time Letter Detection Task Group 1: 100 ms Group 2: 1000 ms

27 Bottom-up control of attention Langton, S., & Bruce, V. (1999). Reflexive visual orienting in response to the social attention of others. Cognition, 6 (5). Targets in the cued location Targets outside the cued location Gaze direction modulates attentional locus of other people Effect persists even if subjects are told that the gaze cue is uninformative, which is not observed for iconic cues (arrow)

28 Bottom-up control of attention Hoffman, E. & Haxby, J. (2000). Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nature Neuroscience, 3. modulation involves regions in the intraparietal sulcus (IPS) The IPS is known to participate in shifts of overt attention

29 We perceive gaze because we can Kobayashi, H. & Koshima, S. (1997). Unique morphology of the human eye. Nature, 387. Man Gibbon The human eye has the largest ratio of white sclera to colored iris compared to other primates This allows for quick and precise estimation of gaze direction of others The human eye morphology lends itself to communicating complex social signals by just eye direction alone

30 Gaze perception accuracy Gamer, M., & Hecht, H. (2007). Are you looking at me? Measuring the cone of gaze. Journal of Experimental Psychology, 33. Stimuli Gaze is perceived as straight within a symmetric cone of about 4. This amounts to an iris movement of approx. 1mm, or about 1 min / arc at 2m viewing distance Discrimination of averted gaze is significantly worse

31 Gaze perception, a special ability? Von Grünau, M. & Anston, C. (1995). The detection of gaze direction: a stare-inthe-crowd effect. Perception,

32 Gaze perception, a special ability? Ricciardelli, P., Baylis, G. & Driver, J. (2000). The positive and negative of human expertise in gaze perception. Cognition, 77.

33 Gaze perception, a special ability? Ricciardelli, P., Baylis, G. & Driver, J. (2000). The positive and negative of human expertise in gaze perception. Cognition, 77. positive-eyes condition negative-eyes condition The stare-in-the crowd and contrast inversion effects indicate specific neural circuits dedicated to gaze perception

34 Gaze perception, a special ability? Grosbras, M. H., Laird, A. R. & Paus, T. (2005). Cortical regions involved in eye movements, shifts of attention, and gaze perception. Human Brain Mapping, 25. Method: Meta-analysis of 59 brain-imaging experiments on cortical regions dedicated to gaze perception. Areas crucial to gaze perception are located in the intraparietal sulcus, the superior temporal sulcus (STS), the postcentral gyrus and many more

35 Gaze perception, a special ability? Hooker, C. I., Paller, K. A., Gitelman, D. R., Parrish, T. B., Mesulam, M. M. & Reber, P. J. (2003). Brain networks for analyzing eye gaze. Brain Research. Cognitive Brain Research, 17. Other than the intraparietal attention area, only neurons in the STS specifically respond to and encode eye gaze

36 Face Recognition Synopsis of cortical regions supposed to be involved in face perception Superior Temporal Sulcus (STS) Static pictures of faces and bodies Movements (posture, hands, body) [Invariant to size, lighting etc.] Gaze perception Inferior temporal Cortex Fusiform Face Area (Kanwisher et al., 2004) Amgydala Emotional Facial expression (Breiter et al., 1996) Orbitofrontal Cortex Facial characteristics (Haxby et al., 2000)

37 Modulation of gaze perception Langton, S. R. H., Watt, R. J. & Bruce, V. (2000). Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Science, 4. Gaze direction is identical among both pictures Gaze perception is modulated by contextual information

38 The Eye Direction Aftereffect (EDAE) Jenkins, R., Beaver, J. D., & Calder, A. J. (2006). I thought you were looking at me: direction-specific aftereffects in gaze perception. Psychological Science, 17. Left, right, or straight Fixation 750 ms Probe display 1500 ms Blank 1000 ms

39 The Eye Direction Aftereffect (EDAE) Acuity Pretest - Instructions In the experiment, you ll be presented with pictures of faces. Please determine for each picture: Did the person gaze towards your left, your right, or straight ahead?

40

41 The Eye Direction Aftereffect (EDAE) Acuity Pretest - Results Pretest Results Cone of gaze (Gamer & Hecht, 2007) Indifference point of gaze perception is at about 4-5.

42 The Eye Direction Aftereffect (EDAE) Adaption Phase - Trial Target display 4 sec 4 sec 4 sec 4 sec (5 min)

43 The Eye Direction Aftereffect (EDAE) Acuity Pretest - Instructions In the experiment, you ll be presented with pictures of faces. Please press the Space key when the green dots appear. (Bogus task)

44

45 The Eye Direction Aftereffect (EDAE) Posttest - Trial Left, right, or straight Adapting Stimulus Probe display 1500 ms Blank 1000 ms TopUp display 8000 ms

46 The Eye Direction Aftereffect (EDAE) Posttest - Instructions In the experiment you ll see a succession of two faces Please determine for the SECOND face: Did the person gaze towards your left, your right, or straight ahead?

47

48 Percent straight Methoden der The Eye Direction Aftereffect (EDAE) Posttest - Results Very strong adaptation effect Gaze angle Tuning curve of straight responses is shifted towards the adapted side

49 The Eye Direction Aftereffect (EDAE) Jenkins, R., Beaver, J. D., & Calder, A. J. (2006). I thought you were looking at me: direction-specific aftereffects in gaze perception. Psychological Science, 17. Eye gaze is processed by specific, and specialized, neural networks Responsible areas are presumably located in the superior temporal sulcus (STS) Areas are subject to classical adaptation phenomena, just like very low-level visual areas Really?

50 The Eye Direction Aftereffect (EDAE) Jenkins, R., Beaver, J. D., & Calder, A. J. (2006). I thought you were looking at me: direction-specific aftereffects in gaze perception. Psychological Science, 17. Pretest and posttest design are not equivalent How much of the effect is attributable to the experimental manipulation?

51 The Eye Direction Aftereffect (EDAE) Jenkins, R., Beaver, J. D., & Calder, A. J. (2006). I thought you were looking at me: direction-specific aftereffects in gaze perception. Psychological Science, 17. False Epistemologic state of theory Undecided True Pretest and posttest design are not equivalent How much of the effect is attributable to the experimental manipulation?

52 The Eye Direction Aftereffect (EDAE) Controlling for the effect of Top-Up stimuli alone 1. Pretest 3. Posttest 2. Adaptation 15 Min. 5 Min. 30 Min. Condition 1: Replication of original paradigm 1. Pretest 2. Pause 3. Posttest 5 Min. Condition 2: Cancel adaptation phase 15 Min. 30 Min.

53 Percent straight Methoden der The Eye Direction Aftereffect (EDAE) Controlling for the effect of Top-Up stimuli alone Adapt Adapt Gaze angle No systematic differences between Adaption+Top-Up and Top-Up alone Are just 8 seconds Top-Up sufficient to yield adaptation?

54 The Eye Direction Aftereffect (EDAE) Probing the temporal characteristics of the Top-Up stimulus 1. Pretest 2. Top Up Phase 8 Min. Presentation of TopUp stimulus for 1, 2, 4, 8 sec (in separate experiments)

55 Percent straight Methoden der The Eye Direction Aftereffect (EDAE) Probing the temporal characteristics of the Top-Up stimulus Gaze angle Strong adaptation effects in all four conditions Effect is largest of longest Top-Up durations (p=.002)

56 The Eye Direction Aftereffect (EDAE) Jenkins, R., Beaver, J. D., & Calder, A. J. (2006). I thought you were looking at me: direction-specific aftereffects in gaze perception. Psychological Science, 17. False Epistemologic state of theory Undecided True One second of exposition time to an adapting stimulus is most likely not sufficient to produce adaptation phenoma in such high cortical areas like the STS no adaptation Maybe the repeated presentation of Top-Up stimuli over 160 trials (approx. 20 min in total) yields implicit adaptation

57 The Eye Direction Aftereffect (EDAE) Flip-Flop adaptation Let direction of Top-Up stimulus alter in every other trial (left right left right ) Pretest TopUp Phase 1: 4 Seconds TopUp Phase 2: 8 seconds 4 Min. 40 Min. 25 Min. Randomized order of Top-Up phases.

58 Percent straight Methoden der The Eye Direction Aftereffect (EDAE) Flip-Flop adaptation - Results N = 8 Gaze angle EDEA is observable but significantly weaker than with prolonged exposition to identical Top-Up directions

59 The Eye Direction Aftereffect (EDAE) Jenkins, R., Beaver, J. D., & Calder, A. J. (2006). I thought you were looking at me: direction-specific aftereffects in gaze perception. Psychological Science, 17. Epistemologic state of theory False True Undecided The EDEA requires prolonged exposition to build up What does prolonged exposition mean? 1. Long temporal duration 2. Repetition

60 The Eye Direction Aftereffect (EDAE) Top-Up duration - Reanalysis Full gaze bias for given timing. In each condition, the EDAE establishes within 4 trials Bias magnitude is higher than in the Flip-Flop experiment, although Top-Up duration was 8 seconds there

61 The Eye Direction Aftereffect (EDAE) Summary is a rapidly occuring perception bias that requires no explicit adaption phase A single presentation of an inducer stimulus with 4 or 8 seconds duration is sufficient to evoke the EDAE Not adaptation duration (implicit or explicit) but repetition of the inducer is the key modulator of bias strength This indicates more complex processing schemes than just that of adapting neural circuits (e.g., higher level cognitive control) False Epistemologic state of theory Undecided True

62 Thank you.