Introduction to eye-tracking. Susanne Brouwer, MPI for Psycholinguistics, Nijmegen Workshop on Eye-tracking; April 16, 2010

Transcription

1 1 Introduction to eye-tracking Susanne Brouwer, MPI for Psycholinguistics, Nijmegen Workshop on Eye-tracking; April 16, 2010

2 Overview 2 Three parts I. What is eye-tracking? II. Research areas and application III. My research

3 Eye-tracking demo 3

4 4 I. What is eye-tracking? 1. The visual field 2. Eye-tracker 3. Tools 4. Calibration

5 The human eye 5 Complex organ consisting of muscles, tissues, and nerve sensors Vision

6 The visual field (1) 6 Cornea The transparent tissue that covers the front of the eye (pupil and iris) Responsible for focusing Two primary types of vision Foveal vision: clear, colorful, rods Peripheral vision: blurry, less colorful, cones

7 Eye movements

8 Saccades 8 Movement of the eye No visual input Very short duration: 80 ms Fastest movements produced by the body Saccade latency Time period to plan and execute a saccade Adults: 200 ms (Matin et al., 1993) Children: 400 ms (Miller, 1969) Infants: ms (Gredeback et al., 2006)

9 Fixations 9 Time between two saccades during which eyes are relatively stationary Visual input Duration: ms

10 Saccades versus fixations 10 Differences between fixations and saccades as a function of a particular task (Rayner, 1998, p. 373)

11 Where is Waldo? 11

12 Eye-tracking 12 Infrared cameras in an eye tracking-device pick up the reflections from the cornea (= optical method) Eye-tracking is the process of measuring the eye position ("where we are looking") and the eye movement

13 Equipment 13 Three types Head-mounted Tower-mounted (chin rest) Remote CANADA GERMANY SWEDEN

14 Tools (1) 14 Head-mounted Eyelink I & II two cameras track the eyes Tower-mounted EyeLink 1000 infrared mirror tracks the eyes, one camera

15 Tower-mounted EyeLink 1000

16 Tools (2) 16 Remote Tobii T60 & T120 device with built-in-screen Remote Tobii 1750 device with built-in-screen

17 Specifications 17 Sampling rate Hz e.g., 1000 ms (1 sec) / 50 = 20 ms e.g., 1000 ms (1 sec) / 2000 = 0.5 ms Binocular or monocular Accuracy: gaze position < 0.5 o Detection: pupil, corneal reflection Adapted from a talk by Fritzsche & Brandt

18 Calibration (1) 18 Procedure that maps the eye position onto the visual display By combining position of the pupils, the corneal reflections, the head, and the position on the visual display Participants are asked to look at corner locations on the screen Adapted from a talk by Fritzsche & Brandt

19 19 Calibration (2)

20 20 Calibration for young children

21 Overview 21 Three parts I. What is eye-tracking? II. Research areas and application III. My research

22 22 II. Research areas and application 1. Advertisement 1. Advertisement 2. Psycholinguistic research

23 Advertisement 23 Woman is looking AWAY Woman is looking AT IGNORING the product LOOKING AT the product

24 Research areas 24 Psychology Scene perception Memory Mental imagery Decision making Attention Linguistics Spoken word recognition Reference resolution Syntactic processing Spoken language production Language acquisition Reading

25 Spoken word recognition Cooper (1974) 25 Participants were listening to a story While on a safari in Africa when suddenly I noticed a hungry lion, slowly moving through the tall grass Language processing and eye movements are closely time-locked

26 Competition effects 26 SPEAKER BEAKER BEETLE CARRIAGE Pick up the BEAKER, now put it below the diamond People look at words which match in onset (beetle) and offset position (speaker) Allopenna et al. (1998)

27 Printed words 27 Effects were replicated with a printed-word version ONSET MATCH OFFSET MATCH buffel koppel rotje panel visser buffer veter lotje McQueen & Viebahn (2007)

28 Dependent variables & design issues 28 Eye movement data Latencies: saccadic eye movements Fixations: probability of fixating a particular object First look: first fixations Objects in the display should be equidistant from center Randomize position of objects in the display Careful selection of visual stimuli Adapted from Tanenhaus et al. (1995)

29 Instructions 29 Subjects receive instructions to look at, touch, or move real objects, pictures of objects, printed words on a computer screen using a mouse No explicit task to avoid meta-linguistic judgments, memorization, or speed of processing

30 Advantages 30 Online measurement Real-time precision (time scale of milliseconds!) No metalinguistic judgement Use of a realistic environment Disadvantages Eye-tracker is expensive Price: Euro Sophisticated data-analysis is necessary Adapted from Tanenhaus et al. (1995)

31 Overview 31 Three parts I. What is eye-tracking? II. Research areas and application III. My research

32 32 III. My research 1. Research with Holger Mitterer and Falk Huettig (MPI) Recognition of reduced forms in casual speech 2. Research with Sharon Unsworth and Pim Mak (UU) Processing of definite determiner in children and adults

33 Casual conversation (1) 33 Dutch Ook naar beneden die sluit dan aan The one going downwards, too, it connects then to Unreduced form beneden - /bunedun/ Buigt het zo af en dan valt het naar beneden, dat is echt It bends like this and then it falls down, that s really Reduced form beneden - /muneju/

34 Casual conversation (2) 34 Dutch De sfeer alsof het een wedstrijd is The atmosphere as if it is a match Unreduced form wedstrijd - /wetstrijt/ Het ergste is nog als de wedstrijd dus afgelopen is The worst thing is that if the match has been finished Reduced form wedstrijd - /wes/

35 Other languages 35 American English yesterday - /yesay/ We were supposed to see it yesterday but I felt really bad German guten Tag /ngun ta/ Ja, guten Tag, dann fang ich einfach mal an

36 Mechanisms to explain reduced form recognition 36 Storage (Goldinger, 1998; Johnson, 2004) Reconstruction (Gaskell, 2003; Mitterer & Blomert, 2003) / bunedu/ /munedu/ /muneju/ /bunedun/ /bunedun/ Reduced input: /muneju/

37 Visual display 37 Unreduced form competitor (Ucomp) Similar to /bunedun/ Reduced form competitor (Rcomp) Similar to /muneju/ RECONSTRUCTION benadelen meneer STORAGE juweel vakantie Distractors COMPETITION AS A MEASURE

38 Overview of the experiments 38 Experiment 1: Forms in isolation Unreduced: /bunedun/ Reduced: /muneju/ Experiment 2: Forms in a sentence context Unreduced: Ook naar /bunedun/ die sluit dan aan Reduced: Buigt het zo af en dan valt het naar /muneju/, dat is echt

39 Exp 1: Forms in isolation Unreduced Canonical forms 0.5 Reduced forms Fixation proportions Time since target onset (ms) Ucomp (benadelen) Rcomp (meneer) Averaged Distractors

40 Exp 2: Forms in a sentence context Unreduced Canonical forms 0.5 Reduced forms Fixation proportions storage reconstruction Time since target onset (ms) Ucomp (benadelen) Rcomp (meneer) Averaged Distractors

41 Recognition of reduced forms 41 Recognition performance depends on the amount of linguistic context Early in time ( ms) listeners look at reduced form competitor, because they ignore the speech input Indication of storage Late in time ( ms) reconstruction may occur, but it is a time-consuming process

42 42 III. My research 1. Research with Holger Mitterer and Falk Huettig (MPI) Recognition of reduced forms in casual speech 2. Research with Sharon Unsworth and Pim Mak (UU) Processing of definite determiner in children and adults

43 Overview of Dutch gender 43 Gender of noun Indefinite Singular Definite Plural Common een de de Neuter een het de Common gender: de schoen the shoe Neuter gender: het huis the house

44 Acquisition of Dutch gender 44 Overgeneralization of de with neuter nouns until at least age 6 (e.g., Blom et al., 2008) *de common huis neuter instead of het neuter huis neuter Almost all previous studies: production data only Research question Do children use grammatical gender to recognize words in online comprehension?

45 45 Design (following Lew-Williams & Fernald 2007)

46 Conditions 46 EXPERIMENTAL Different gender het gele huis vs. de gele schoen Same gender het gele huis vs. het gele bed BASELINE Different gender mijn gele huis vs. mijn gele schoen Same gender mijn gele huis vs. mijn gele bed mijn my = non-gender-marked possessive adjective

47 Design (cont d) 47 6 nouns per gender (n = 60) Fillers to distract (n=24) All nouns presented as Det-Adj-N, embedded in short sentence (Kijk eens naar Look at! )

48 48 Fillers

49 Participants adults 49 children (4 7 yrs) Two groups: Young: 4 5 yrs (n=26) Old: 6 7 yrs (n=23)

50 Different gender: Experimental vs. Baseline Target = common, distractor = neuter 50 Adults, n = 37 Children 6-7, n = 23 Children 4-5, n = 26 Proportion of looks to the target * * Time since determiner onset (ms) target: de COMMON gele schoen vs. distractor: het NEUTER gele huis target: mijn COMMON gele schoen vs. distractor: mijn NEUTER gele huis

51 Different gender: Experimental vs. Baseline Target = neuter, distractor = common 51 Adults, n = 37 Children 6-7, n = 23 Children 4-5, n = 26 Proportion of looks to the target * * Time since determiner onset (ms) target: het NEUTER gele huis vs. distractor: de COMMON gele schoen target: mijn NEUTER gele huis vs. distractor: mijn COMMON gele schoen

52 Conclusion 52 Adults use grammatical gender as a cue Most pronounced when target has neuter gender Similar pattern for older children, although somewhat later No effect for younger children, indicating that at age 4-5 Dutch-speaking children are not able to make consistent use of grammatical gender in speech processing

53 53 Introduction to eye-tracking What is eye-tracking? Research areas and application My research Thank you!

54 References (1) 54 Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: evidence for continuous mapping models. Journal of Memory and Language, 38, Blom, E., Polišenská, D., & Weerman, F. (2008). Articles, adjectives and ageof onset: the acquisition of Dutch grammatical gender. Second Language Research, 24 (3), Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language. A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6, Gaskell, M. G. (2003). Modelling regressive and progressive effects of assimilation in speech perception. Journal of Phonetics, 31,

55 References (2) 55 Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105(2), Gredebäck, Örnkloo, & von Hofsten (2006). The development of reactive saccade latencies. Experimental Brain Research, 173(1), Johnson, K. (2004). Massive reduction in conversational American English. In K. Yoneyama & K. Maekawa (Eds.), Casual Speech: Data and Analysis. Proceedings of the 1st session of the 10th International Symposium. Tokyo, Japan (pp ). Kemps, R., Ernestus, M., Schreuder, R., & Baayen, H. (2004). Processing reduced word forms: The suffix restoration effect. Brain and Language, 90,

56 References (3) 56 Lew-Williams, C. & Fernald, A. (2007). Young children learning Spanish make rapid use of grammatical gender in spoken word recognition. Psychological Science, 18(3), Matin, E., Shao, K. C., & Boff, K. R. (1993). Saccadic overhead: Informationprocessing time with and without saccades. Perception and Psychophysics, 53, McQueen, J. M., & Viebahn, M. (2007). Tracking recognition of spoken words by tracking looks to printed words. The Quarterly Journal of Experimental Psychology, 60(5), Miller (1969). Eye-movement latency as a function of age, stimulus uncertainty, and position in the visual field. Perceptual & Motor Skills, 28,

57 References (4) 57 Mitterer, H., & Blomert, L. (2003). Coping with phonological assimilation in speech perception: Evidence for early compensation. Perception and Psychophysics, 65(6), Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research, Psychological Bulletin, 124, Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268,