2. Eye Tracking Approaches. 1. Introduction

Transcription

1 Proceedings of he 22 American Conrol Conference Alaska, 22 A Non-inrusive Kalman Filer-Based Tracker for Pursui Eye Movemen * Wael Abd-Almageed M. Sami Fadali George Bebis Elecrical and Compuer Eng. Dep. Elecrical Engineering Dep. Compuer Science Dep. Universiy of New Mexico Universiy of Nevada Universiy of Nevada Albuquerque, NM Reno, NV 897 Reno, NV 897 wamageed@eece.unm.edu Fadali@ee.unr.edu bebis@cs.unr.edu Absrac In his paper, we inroduce a new non-inrusive approach o esimaing he eye posiion during pursui moion of he eye. We inroduce a new characerizaion for he pursui eye movemen. Our characerizaion is based on he decomposiion of he pursui eye moion ino a deerminisic componen and a random componen. We use a discree Kalman filer o esimae he random componen and calculae he deerminisic componen. We add he wo componens o obain an esimae of he eye posiion. Simulaion resuls are provided o illusrae he eye posiion esimaion. Key words: Kalman filer, eye racking, gaze racking, pusui moion 1. Inroducion Eye racking has long been recognized as an imporan ask in boh Compuer Vision and Human-Compuer Ineracion (HCI.) In HCI, for example, deermining he focus of aenion plays an imporan role in responding o he user s inensions. Two main HCI applicaions ha require eye racking are hands-free cursor conrol and objec movemen. In hese applicaions, accurae racking of eye posiion is required. An auomaic ex reader is anoher applicaion ha requires accurae eye posiion esimaion. In virual environmens, he user should be able o inerac wih he objecs in he environmen in naural and easy ways. Several sudies have focused on developing ineracion echniques using hands [8] wih less emphasis on eyebased mehods. Curren eye racking approaches rely heavily on inrusive echniques such as measuring he reflecion of some ligh (usually infrared) ha is shone ino he eye, measuring he elecric poenial of he skin around he eyes or applying special conac lenses ha faciliae he eye racking process [1]. User accepance for any of hese inrusive echniques has always been a problem when applying hem. Oher noninrusive echniques, such as neural neworks [2], have also been proposed bu wih less success han he inrusive ones. * This work was funded, in par, by he Naional Science Foundaion (NSF) gran number 8886 In his paper, we inroduce a new non-inrusive approach o racking eyes undergoing pursui moion. We use a discree Kalman filer o rack he user s eye posiion. The main advanages of he proposed approach are: 1) i requires no physical conac wih he user, 2) i resuls in high racking accuracy, and 3) low compuaional complexiy so ha i can be implemened for real-ime applicaions. This paper is organized as follows. Secion 2 provides a shor survey of eye racking echniques. Secion 3 presens a brief overview of differen ypes of eye movemen. Secion 4 is he core par of he paper in which we inroduce he proposed approach. In secion we discuss eye moion simulaion resuls using our esimaion approach. In secion 6 we provide conclusions and suggesions for fuure work. 2. Eye Tracking Approaches We can broadly classify eye racking echniques ino hree major caegories according o he way hey conac he user [1]. The firs caegory is based on direcing a beam of ligh (ypically infrared) ino he eye and hen measuring he ligh refleced from he eye. The second caegory is based on measuring he poenial of he skin around he eye. The las caegory is based on applying special ype of conac lenses ha faciliae eye racking. 2.1 Measuring ligh reflecance echniques Four racking echniques use he ligh refleced from he eye. These echniques are: 1. Limbus racking (Limbus is he boundary beween he whie sclera and he dark iris.) 2. Pupil racking (he boundary beween he pupil and he iris is usually used.) 3. Purkinje Images racking (Purkinje Images are he differen reflecions from he boundaries of he lens and cornea. See Figure 1.) 4. Corneal Reflecions racking. 2.2 Elecric Poenial of he Skin This class of echniques is based on he fac ha here exiss an elecrosaic field ha roaes along wih he eye. By recording small differences in he skin poenial around he eye, he posiion of he eye can be deeced. 2.3 Conac Lenses

2 Acual Targe Posiion Eye Posiion Smooh Eye Movemens Saccadic Eye Movemens Figure 1: Purkinje Images In his class of echniques, he user wears a special ype of conac lenses ha allows accurae deecion of he direcion of he gaze. As expeced, any inrusive echnique is no easily acceped by he users and canno be used in everyday life. For example, inrusive echniques are no suiable for HCI applicaions. This moivaes he developmen of new noninrusive echniques ha achieve same level of accuracy. 3. Eye Movemens Eye movemen is generally classified ino four main ypes [3], based on wha he eye is viewing a a specific momen. These four classes are described as follows. 1. Saccadic Eye Movemens Saccades are very fas jumps from one eye posiion o anoher. The velociy of saccades can be as high as 8 degrees/second [3]. This kind of eye movemen is usually used while following a rapidly moving objec or scene. 2. Pursui Eye Movemens As is name implies, his ype of eye movemen is used while racking (or pursuiing) a slowly moving objec. I consiss of wo componens: a slowly varying moion componen plus a saccadic componen. This saccadic componen occurs occasionally as a correcion mechanism for he eye posiion [3]. The saccadic componen occurs when he eye curren eye posiion is no accurae wih respec o he moving objec. 3. Fixaion Eye Movemens During fixaing on a saic scene or objec, he eye acually undergoes hree differen ypes of movemens [3]. The firs is a coninuos high frequency remor componen. The frequency of his componen ranges from 3 o 8 cycles per second. The second eye movemen componen during fixaion is a slow drif of he eyeballs in a random direcion. The las componen is a flickering eye movemen ha correcs he eye posiion ha changed by he drif componen. 4. Nysagmus Eye Movemen Figure 2: Decomposiion of Pursui Eye Moion This paern of eye movemen occurs when viewing a fas moving repeiive scene (he rain window phenomenon.) I consiss of pursui moion in one direcion o follow a posiion in he scene followed by fas moion in he opposie direcion o selec a new posiion in he scene. 4. Kalman Filer Tracker The Kalman filer has been used exensively in Compuer Vision research. Mos of he applicaions ha uilize Kalman filer focus on he problems of objec racking and srucure from moion [7]. To he bes of our knowledge, Kalman filer has no been used o rack eye movemens. In his paper, we apply he Kalman filer o rack he pursui moion of he eye. As discussed in Secion 3, he pursui eye movemen occurs when he eye is racking a slowly moving objec. Figure 2 shows ha he pursui eye movemen can be decomposed ino wo componens. One componen is a simple posiion-velociy-acceleraion moion and he oher one is he saccadic componen. This saccadic componen could be hough of as a random componen ha correcs he posiion of he eye. The saccadic componen has a random ime of occurrence, a random duraion and a random ampliude. The duraion of hese saccadic movemens ypically range from 3 o 12 milliseconds, while heir velociy can be as large as 8 degrees/second. This means ha, he acceleraion of he saccadic componen increases rapidly and decreases rapidly causing a fas move and an accurae sop a he required posiion. We, herefore, model he acceleraion of his moion as a Gauss-Markov process [6]. The auocorrelaion funcion of his process is shown in Figure 3 and is given by he equaion R( τ) σ 2 exp( βτ) (1)

3 R() where φ exp( F ), w is he process Gaussian whie noise and v is he measuremen Gaussian whie noise. To deermine he covariance marix Q of he process noise w for he model of equaion (8) and (9), we use he Van-Loan procedure []. We now have he discree saespace sysem model and he noise characerizaion needed o design a discree Kalman filer.. Resuls and Discussion Figure 3: Auocorrelaion Funcion of Gauss- Markov Process The power specral densiy of Gauss-Markov process is 2σ 2 β S n ( s) (2) s 2 + β 2 Because he sandard Kalman filer formulaion requires he noise o be Gaussian whie noise, we use a shaping filer. The specral decomposiion for he Gauss-Markov process is 2σ 2 β 2σ 2 β Ss ( ) (3) s + β s + β We characerize he pursui eye movemen by he following sae-space equaions x1 x2 1 z 1 + n () (4) () where, and n () represens he posiion, velociy and Gauss-Markov random acceleraion componen respecively. We augmen he sae vecor x wih an addiional sae variable o accoun for he Gauss-Markov process as follows z 1 1 β 2σ 2 β + 1 (6) (7) where u () is a uniy Gaussian whie noise. In he discree form, he model is x k z k φ k + w k H k x k + v k (8) (9) u () The objecive of he proposed echnique is o develop a racker for he pursui moion of he eye. According o our formulaion of he racking problem, he inpu o he Kalman filer can be considered as a zero-mean Gaussian whie noise. To es he performance of he proposed racker, we have simulaed he acual (rue) posiion of he eye by generaing slowly varying sequences of numbers ha represen he acual eye posiion (e.g. large wavelengh sinusoidal signal.) The generaed sequences were hen corruped by zeromean Gaussian whie noise. The composie signal represens he measured (corruped) posiion of he eye. The composie signal is fed ino he filer o esimae he original signal, he correc eye posiion. Figures 4a-7a show four simulaions o he inpu signal o he filer, he measured eye posiion. Figures 4b-7b show he filering resuls of he four simulaion experimens. The figures on he lef row show he inpu signal o he filer, which is composed of he rue eye posiion corruped by zero-mean Gaussian whie noise. The figures on he righ row show he filering resuls superimposed on he original signal. The filered signals represen he correc eye posiion. Table 1 shows he error levels of he corruping noise and he noise variance in each case of he four experimens. We have implemened he proposed approach using MAT- LAB. The ime needed o process 18 samples was.3 seconds (i.e..17 ms/sample) on a 1GHz P-III machine running MS-Windows 2 operaing sysem. This ime is he ime required o filer one sample of he corruped signal o obain he original one. This ime does no include he ime required o measure he eye posiion from he eye image. Our resuls show a significan racking error reducion wih he use of he Kalman filer compared o direcly using he measuremen of he eye posiion. This error reducion is he main advanage of he proposed approach a a small cos of addiional compuaional ime o filer he measuremens. The simulaion resuls shown are for pursui moion in one direcion. However, o implemen he racker in he 2D space (spaial dimensions of images), wo independen filers mus be used, one filer for each spaial dimension. The wo filers can be independenly designed assuming ha he moion in one direcion is saisically independen of he moion in he oher direcion.

4 Table 1: Experimenal Resuls Noise Variance Noise-o-Signal % Esimaion Erroro-Signal % Error Reducion Experimen % 1.619% 78.61% Experimen % 2.637% 81.68% Experimen % 3.22% % Experimen %.439% % True Eye Posiion Esimaed Eye Posiion Simulaion of he Measured Signal Figure 4.a: Experimen 1: Tes Sequence Simulaing Measured Eye Posiion Figure 4.b: Experimen 1: Filering Resuls Superimposed on True Eye Posiion 2 2 True Eye Posiion Esimaed Eye Posiion Simulaion of he Measured Signal Figure.a: Experimen 2: Tes Sequence Simulaing Measured Eye Posiion Figure.b: Experimen 2: Filering Resuls Superimposed on True Eye Posiion 1 1 True Eye Posiion E sim aed Eye Posiion Simulaion of he Measured Signal Figure 6.a: Experimen 3: Tes Sequence Simulaing Measured Eye Posiion Figure 6.b: Experimen 3: Filering Resuls Superimposed on True Eye Posiion

5 True Eye Posiion E sim aed Eye Posiion Simulaion of Measured Signal Figure 7.a: Experimen 4: Tes Sequence Simulaing Measured Eye Posiion Figure 7.b: Experimen 4: Filering Resuls Superimposed on True Eye Posiion 6. Conclusions and Fuure Work In his paper, we propose a new non-inrusive approach for racking he posiion for an eye undergoing pusui movemen. Our approach is based on he physiological fac ha he pursui moion can be decomposed ino a regular posiion-velociy-acceleraion componen and a random componen ha occurs occasionally. We presened some simulaion resuls based on compuer generaed simulaion daa. Our simulaion resuls show ha he Kalman filer can be used effecively o esimae he rue eye posiion during pursui moion. The hree main advanages of he proposed approach are: (1) his is a non-inrusive approach which will no be rejeced by he users compared o inrusive echniques, (2) he proposed approach significanly reduces he measuremen error of he eye posiion, and (3) he shor ime required o process he measuremens represens a low compuaional overhead. This processing ime does no include he ime needed o process he image o measure he eye posiion. The low compuaional complexiy of he algorihm arises from he scarciy of he model marices, which simplifies he marix muliplicaions and inversions when running he Kalman loop. This low compuaional complexiy characerisic of he proposed approach makes i suiable for real-ime applicaions. The nex sep in our research is o es he racker performance using real image daa. To develop a feaure exracor ha measures he eye posiion reliably, we need o have a head-mouned camera o eliminae he effec of head and body movemens. Hence, we can obain image sequences for he eye wihou he superimposed head and body movemens. An alernaive approach, ha does no require he use of he cosly camera, is o augmen he curren model o compensae for he head and body moion. This would allow us o implemen he proposed approach wihou he use of a cosly head-mouned camera. References [1] A. Glensrup and T. Angell-Nielsen, "Eye Conrolled Media, Presen and Fuure Sae," Technical Repor, Universiy of Copenhagen, hp:// panic/eyegaze/, 199. [2] S. Baluja and D. Pomerleau, "Non-inrusive Gaze Tracking Using Arificial Neural Neworks," Technical Repor CMU-CS Carnegie Mellon Universiy, [3] M. Yanoff and J. Duker, Ophhalmology, Mosby Inernaional Ld., [4] R. Jacob, "Eye Tracking in Advanced Inerface Design," In Advanced Inerface Design and Virual Environmens, ed. W. Barfield and T. Furness, Oxford Universiy Press, Oxford, [] C. F. van Loan, "Compuing Inegrals Involving he Marix Exponenial," IEEE Trans. Auomaic Conrol, AC-23, 3, 39-44, June [6] R.BrownandP.Hwang,"InroducionoRandomSignals and Applied Kalman Filering," 3rd ediion, Wiley, [7] Ali Azarbayejani and Alex Penland, Recursive esimaion of moion, srucure, and focal lengh, IEEE Trans. PAMI, 17(6), 62--7,June 199. [8] Bowman D.A. Bowman and L.F. Hodges, "An Evaluaion of Techniques for Grabbing and Manipulaing Remoe Objecs in Impressive Virual Environmens," Proc. Of he 1997 Symposium on Ineracive 3D Graphics, 1997, pp