1 Introduction to eye-tracking Susanne Brouwer, MPI for Psycholinguistics, Nijmegen Workshop on Eye-tracking; April 16, 2010
Overview 2 Three parts I. What is eye-tracking? II. Research areas and application III. My research
Eye-tracking demo 3 http://www.youtube.com/watch?v=lo_a2cfbugc
4 I. What is eye-tracking? 1. The visual field 2. Eye-tracker 3. Tools 4. Calibration
The human eye 5 Complex organ consisting of muscles, tissues, and nerve sensors Vision
The visual field (1) 6 Cornea The transparent tissue that covers the front of the eye (pupil and iris) Responsible for focusing Two primary types of vision Foveal vision: clear, colorful, rods Peripheral vision: blurry, less colorful, cones
Eye movements
Saccades 8 Movement of the eye No visual input Very short duration: 80 ms Fastest movements produced by the body Saccade latency Time period to plan and execute a saccade Adults: 200 ms (Matin et al., 1993) Children: 400 ms (Miller, 1969) Infants: 450 600 ms (Gredeback et al., 2006)
Fixations 9 Time between two saccades during which eyes are relatively stationary Visual input Duration: 200-300 ms
Saccades versus fixations 10 Differences between fixations and saccades as a function of a particular task (Rayner, 1998, p. 373)
Where is Waldo? 11
Eye-tracking 12 Infrared cameras in an eye tracking-device pick up the reflections from the cornea (= optical method) Eye-tracking is the process of measuring the eye position ("where we are looking") and the eye movement
Equipment 13 Three types Head-mounted Tower-mounted (chin rest) Remote CANADA GERMANY SWEDEN
Tools (1) 14 Head-mounted Eyelink I & II two cameras track the eyes Tower-mounted EyeLink 1000 infrared mirror tracks the eyes, one camera
Tower-mounted EyeLink 1000
Tools (2) 16 Remote Tobii T60 & T120 device with built-in-screen Remote Tobii 1750 device with built-in-screen
Specifications 17 Sampling rate 50-2000 Hz e.g., 1000 ms (1 sec) / 50 = 20 ms e.g., 1000 ms (1 sec) / 2000 = 0.5 ms Binocular or monocular Accuracy: gaze position < 0.5 o Detection: pupil, corneal reflection Adapted from a talk by Fritzsche & Brandt
Calibration (1) 18 Procedure that maps the eye position onto the visual display By combining position of the pupils, the corneal reflections, the head, and the position on the visual display Participants are asked to look at corner locations on the screen Adapted from a talk by Fritzsche & Brandt
19 Calibration (2)
20 Calibration for young children
Overview 21 Three parts I. What is eye-tracking? II. Research areas and application III. My research
22 II. Research areas and application 1. Advertisement 1. Advertisement 2. Psycholinguistic research
Advertisement 23 Woman is looking AWAY Woman is looking AT IGNORING the product LOOKING AT the product
Research areas 24 Psychology Scene perception Memory Mental imagery Decision making Attention Linguistics Spoken word recognition Reference resolution Syntactic processing Spoken language production Language acquisition Reading
Spoken word recognition Cooper (1974) 25 Participants were listening to a story While on a safari in Africa when suddenly I noticed a hungry lion, slowly moving through the tall grass Language processing and eye movements are closely time-locked
Competition effects 26 SPEAKER BEAKER BEETLE CARRIAGE Pick up the BEAKER, now put it below the diamond People look at words which match in onset (beetle) and offset position (speaker) Allopenna et al. (1998)
Printed words 27 Effects were replicated with a printed-word version ONSET MATCH OFFSET MATCH buffel koppel rotje panel visser buffer veter lotje McQueen & Viebahn (2007)
Dependent variables & design issues 28 Eye movement data Latencies: saccadic eye movements Fixations: probability of fixating a particular object First look: first fixations Objects in the display should be equidistant from center Randomize position of objects in the display Careful selection of visual stimuli Adapted from Tanenhaus et al. (1995)
Instructions 29 Subjects receive instructions to look at, touch, or move real objects, pictures of objects, printed words on a computer screen using a mouse No explicit task to avoid meta-linguistic judgments, memorization, or speed of processing
Advantages 30 Online measurement Real-time precision (time scale of milliseconds!) No metalinguistic judgement Use of a realistic environment Disadvantages Eye-tracker is expensive Price: 20.000-100.000 Euro Sophisticated data-analysis is necessary Adapted from Tanenhaus et al. (1995)
Overview 31 Three parts I. What is eye-tracking? II. Research areas and application III. My research
32 III. My research 1. Research with Holger Mitterer and Falk Huettig (MPI) Recognition of reduced forms in casual speech 2. Research with Sharon Unsworth and Pim Mak (UU) Processing of definite determiner in children and adults
Casual conversation (1) 33 Dutch Ook naar beneden die sluit dan aan The one going downwards, too, it connects then to Unreduced form beneden - /bunedun/ Buigt het zo af en dan valt het naar beneden, dat is echt It bends like this and then it falls down, that s really Reduced form beneden - /muneju/
Casual conversation (2) 34 Dutch De sfeer alsof het een wedstrijd is The atmosphere as if it is a match Unreduced form wedstrijd - /wetstrijt/ Het ergste is nog als de wedstrijd dus afgelopen is The worst thing is that if the match has been finished Reduced form wedstrijd - /wes/
Other languages 35 American English yesterday - /yesay/ We were supposed to see it yesterday but I felt really bad German guten Tag /ngun ta/ Ja, guten Tag, dann fang ich einfach mal an
Mechanisms to explain reduced form recognition 36 Storage (Goldinger, 1998; Johnson, 2004) Reconstruction (Gaskell, 2003; Mitterer & Blomert, 2003) / bunedu/ /munedu/ /muneju/ /bunedun/ /bunedun/ Reduced input: /muneju/
Visual display 37 Unreduced form competitor (Ucomp) Similar to /bunedun/ Reduced form competitor (Rcomp) Similar to /muneju/ RECONSTRUCTION benadelen meneer STORAGE juweel vakantie Distractors COMPETITION AS A MEASURE
Overview of the experiments 38 Experiment 1: Forms in isolation Unreduced: /bunedun/ Reduced: /muneju/ Experiment 2: Forms in a sentence context Unreduced: Ook naar /bunedun/ die sluit dan aan Reduced: Buigt het zo af en dan valt het naar /muneju/, dat is echt
Exp 1: Forms in isolation 39 0.5 Unreduced Canonical forms 0.5 Reduced forms Fixation proportions 0.4 0.3 0.2 0.1 0.4 0.3 0.2 0.1 0 0 200 400 600 800 10001200 Time since target onset (ms) 0 0 200 400 600 800 10001200 Ucomp (benadelen) Rcomp (meneer) Averaged Distractors
Exp 2: Forms in a sentence context 40 0.5 Unreduced Canonical forms 0.5 Reduced forms Fixation proportions 0.4 0.3 0.2 0.1 0.4 0.3 0.2 0.1 storage reconstruction 0 0 200 400 600 800 10001200 Time since target onset (ms) 0 0 200 400 600 800 10001200 Ucomp (benadelen) Rcomp (meneer) Averaged Distractors
Recognition of reduced forms 41 Recognition performance depends on the amount of linguistic context Early in time (400-800 ms) listeners look at reduced form competitor, because they ignore the speech input Indication of storage Late in time (800-1200 ms) reconstruction may occur, but it is a time-consuming process
42 III. My research 1. Research with Holger Mitterer and Falk Huettig (MPI) Recognition of reduced forms in casual speech 2. Research with Sharon Unsworth and Pim Mak (UU) Processing of definite determiner in children and adults
Overview of Dutch gender 43 Gender of noun Indefinite Singular Definite Plural Common een de de Neuter een het de Common gender: de schoen the shoe Neuter gender: het huis the house
Acquisition of Dutch gender 44 Overgeneralization of de with neuter nouns until at least age 6 (e.g., Blom et al., 2008) *de common huis neuter instead of het neuter huis neuter Almost all previous studies: production data only Research question Do children use grammatical gender to recognize words in online comprehension?
45 Design (following Lew-Williams & Fernald 2007)
Conditions 46 EXPERIMENTAL Different gender het gele huis vs. de gele schoen Same gender het gele huis vs. het gele bed BASELINE Different gender mijn gele huis vs. mijn gele schoen Same gender mijn gele huis vs. mijn gele bed mijn my = non-gender-marked possessive adjective
Design (cont d) 47 6 nouns per gender (n = 60) Fillers to distract (n=24) All nouns presented as Det-Adj-N, embedded in short sentence (Kijk eens naar Look at! )
48 Fillers
Participants 49 37 adults 49 children (4 7 yrs) Two groups: Young: 4 5 yrs (n=26) Old: 6 7 yrs (n=23)
Different gender: Experimental vs. Baseline Target = common, distractor = neuter 50 Adults, n = 37 Children 6-7, n = 23 Children 4-5, n = 26 Proportion of looks to the target * * Time since determiner onset (ms) target: de COMMON gele schoen vs. distractor: het NEUTER gele huis target: mijn COMMON gele schoen vs. distractor: mijn NEUTER gele huis
Different gender: Experimental vs. Baseline Target = neuter, distractor = common 51 Adults, n = 37 Children 6-7, n = 23 Children 4-5, n = 26 Proportion of looks to the target * * Time since determiner onset (ms) target: het NEUTER gele huis vs. distractor: de COMMON gele schoen target: mijn NEUTER gele huis vs. distractor: mijn COMMON gele schoen
Conclusion 52 Adults use grammatical gender as a cue Most pronounced when target has neuter gender Similar pattern for older children, although somewhat later No effect for younger children, indicating that at age 4-5 Dutch-speaking children are not able to make consistent use of grammatical gender in speech processing
53 Introduction to eye-tracking What is eye-tracking? Research areas and application My research Thank you!
References (1) 54 Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: evidence for continuous mapping models. Journal of Memory and Language, 38, 419-439. Blom, E., Polišenská, D., & Weerman, F. (2008). Articles, adjectives and ageof onset: the acquisition of Dutch grammatical gender. Second Language Research, 24 (3), 289-323. Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language. A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6, 84-107. Gaskell, M. G. (2003). Modelling regressive and progressive effects of assimilation in speech perception. Journal of Phonetics, 31, 447 463.
References (2) 55 Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105(2), 251-279. Gredebäck, Örnkloo, & von Hofsten (2006). The development of reactive saccade latencies. Experimental Brain Research, 173(1), 159-164. Johnson, K. (2004). Massive reduction in conversational American English. In K. Yoneyama & K. Maekawa (Eds.), Casual Speech: Data and Analysis. Proceedings of the 1st session of the 10th International Symposium. Tokyo, Japan (pp. 29-54). Kemps, R., Ernestus, M., Schreuder, R., & Baayen, H. (2004). Processing reduced word forms: The suffix restoration effect. Brain and Language, 90, 117-127.
References (3) 56 Lew-Williams, C. & Fernald, A. (2007). Young children learning Spanish make rapid use of grammatical gender in spoken word recognition. Psychological Science, 18(3), 193-198. Matin, E., Shao, K. C., & Boff, K. R. (1993). Saccadic overhead: Informationprocessing time with and without saccades. Perception and Psychophysics, 53, 372-380. McQueen, J. M., & Viebahn, M. (2007). Tracking recognition of spoken words by tracking looks to printed words. The Quarterly Journal of Experimental Psychology, 60(5), 661-671. Miller (1969). Eye-movement latency as a function of age, stimulus uncertainty, and position in the visual field. Perceptual & Motor Skills, 28, 631-636.
References (4) 57 Mitterer, H., & Blomert, L. (2003). Coping with phonological assimilation in speech perception: Evidence for early compensation. Perception and Psychophysics, 65(6), 956-969. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research, Psychological Bulletin, 124, 372-422. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632-1634.