Eye Center Localization on a Facial Image Based on Multi-Block Local Binary Patterns

Transcription

1 Eye Center Localzaton on a Facal Image Based on Mult-Bloc Local Bnary Patterns Anatoly tn, Vladmr Khryashchev, Olga Stepanova Yaroslavl State Unversty Yaroslavl, Russa anatolyntnyar@gmal.com, vhr@yandex.ru, olgastepanova@yandex.ru Igor Kostern Fre and Rescue Academy Ivanovo, Russa osternv@gmal.com Abstract In ths paper an eye center localzaton algorthm based on mult-bloc local bnary patterns s descrbed. Performance of the suggested algorthm s compared to another methods based on Bayesan approach and mage gradents. Vsual examples of eye center localzaton results are provded. I. ITRODUCTIO Beng probably the most expressve and salent features of the human face, eyes are very mportant sources of nformaton for face analyss. Accurate and effcent eye locatng n a gven face mage s very mportant for wde range of practcal problems n computer vson, ncludng face recognton, face algnment, gaze and pose estmaton, expresson analyss, etc [-4]. It s not uncommon to mstaenly thn that eye localzaton s a smple tas snce eyes are just a smple structure n the face. However, eyes have ts unque geometrc, photometrc and moton characterstcs, and changes due to these three characterstcs would lead to a very complcated nonlnear manfold of eyes. In partcular, the followng factors have sgnfcant nfluence on the states of the eyes [5]: wde varety of colors and shapes of eyes; emotons and facal expressons: for example, eyes of laughng person can be almost closed; dfferent occlusons: eyes are often occluded by har, myopa glasses or sunglasses; pose: t s possble that one eye s completely occluded n a profle face; magng condton and qualty: envronment factors, such as lghtng (varyng n spectra, source dstrbuton, and ntensty), may change the appearance of the eyes n dfferent ways. Moreover, the commonly seen factors n the real world, such as low resoluton, blurrng or detaled texture mssng, may also lead to poor mage qualty. The great challenges of eye localzaton under the uncontrolled condtons llustrated on mages from Yale [6] and Robotcs [7] databases (Fg. ). Eye localzaton can be successfully used as a preprocessng stage for face recognton []. Many authors [8-] observed that the accuracy of eye localzaton has a sgnfcant effect on face recognton accuracy, whch urge the need for developng robust and accurate eye localzaton technques n real-lfe scenaros. In the recent thrty years, problem of eye localzaton has ganed ncreasng attenton from both the academc and ndustral communtes. However, ths problem s stll far from beng resolved. Most approaches suggested by researches n the recent tme can be dvded nto four groups dependng on the way of buldng of eye model [5]: measurng eye characterstcs: ths type of method explots the nherent features of eyes: dstnct shapes, strong ntensty contrast, etc; learnng statstcal appearance model: ths type of method tres to extract useful vsual features from photometrc appearance, eye model s then learned from a large set of tranng mages; explotng the spatal structure of the face: ths approach explores the geometrcal regularty between eyes and other facal features n the face context: descrbng the local structure of the mage: ths approach s based on mage representaton as a lowdmensonal vector usng local structure of the mage (local bnary patterns, for example). The major contrbuton of ths paper s to present an eye center localzaton algorthm based on local bnary patterns (LBP). The structure of the algorthm s explaned n Secton II. Secton III s devoted to methodology of evaluaton. The results of testng of the algorthm n comparson to other popular methods are presented n Secton IV. ISS

2 II. EYE CETRE LOCALIZATIO APPROACH BASED O MULTI- BLOCK LOCAL BIARY PATTER The basc dea of the LBP s to avod mage representaton as a hgh-dmensonal vector whch contans a lot of redundant nformaton, and descrbe the local structure of an mage. Extracted features wll have lower dmenson. LBP s computatonally effcent because t wors wth nteger arthmetc (t can acheve real-tme performance n some tass), and t s nvarant to changes n the brghtness of the mage caused by shootng n dfferent lghtng condtons [3]. In the recent years a number of approaches based on Mult-Bloc LBP (MB-LBP) has ganed ncreasng attenton from researches n the feld of mage processng [4-7]. The basc dea of MB-LBP s to process rectangle regons of an mage wth classc LBP operator. To encode the rectangles, the MB-LBP operator s defned by comparng the central rectangle s average ntensty wth those of ts neghborhood rectangles. The followng steps are offered to apply MB-LBP to the problem of eye center localzaton: A. Collectng the eye tranng samples: The samples of eyes and eye-le negatve patterns are collected n three dfferent scales: Fg.. An llustraton of the great challenges of eye localzaton under the uncontrolled condtons The orgnal LBP was ntroduced by Ojala [2]. Ths theoretcally very smple yet effcent multresoluton operator s defned for each pxel by thresholdng the 3 3 neghborhood pxel value wth the center pxel value. In ths way, t can gve us a bnary sequence defned by local structure of an mage. a) larger scale samples contan both eye and eyebrow nformaton for better processng of eyes wth myopa glasses or sunglasses. The geometry of these eye tranng samples s shown n the left of Fg. 2; b) standard scale samples only contan the eye tself but exclude the eyebrow, nose and others. The geometry of these eye tranng samples s shown n the center of Fg. 2; c) smaller scale samples only contan the central part of eye tself. The geometry of these eye tranng samples s shown n the rght of Fg. 2. The tranng s made for every scale ndependently. The tranng set s represented as pars of values where s the class label of the example () (b) (c) Fg. 2. The geometry of eye tranng samples: a larger scale; b standard scale; c smaller scale

3 B. Choosng the start weghts w for samples: The start weghts are chosen accordng to the number of eye samples and eye-le negatve patterns. In case of equal number of eye samples and eye-le negatve patterns the start weghts could be chosen as followng: w =. In our experments we selected larger weghts for eye samples, snce the number of eye samples was lower than the number of eye-le negatve patterns. In any case the start weghts should be normalzed: = w =. C. Constructng of ntegral mage: Integral mage s used for rapd computaton of MB-LBP operators. The ntegral mage at pxel locaton s constructed as follows: where ( x y) ( x, y) = ( x', y' ) x' x, y' y, s the ntegral mage, (x, s the orgnal mage. Usng the followng par of recurrences the ntegral mage can be computed n one pass over the orgnal mage: y) () (2) (3) s where ( x y) (, y) = 0. ( x, y) = s( x, y ) + ( x, y) ( x, y) = ( x, y) + s( x, y) s, s the cumulat D. Calculaton of all possble MB-LBP values for the selected wndow sze: MB-LBP s a generalzed form of local patterns that processes rectangle regons of an mage. To encode the rectangles, the MB-LBP operator s defned by comparng the central rectangle s average ntensty wth those of ts neghborhood rectangles. In ths way, t can gve us a bnary sequence. An output value of the MB-LBP operator can be obtaned as follows [4]: MB LBP = where s s sgma functon: s 8 = 2, x 0 = 0, x < 0 s the average ntensty of the center rectangle, are ntenstes of ts neghborhood rectangles (see Fg. 3 for an example). In general, 256 varous patterns may be obtaned for a 3x3 wndow sze, some of them are presented n Fg. 4., (4) tve row sum, ( x, ) = 0 s, s g g ), (5) ( c (6) Fg. 3. Example of countng of MB-LBP. MB-LBP n ths case s equal to

4 Fg. 4. A randomly chosen subset of the MB-LBP features E. Choosng wea classfers For each MB-LBP feature, we adopt mult-branch tree as wea classfers. The mult-branch tree totally has 256 branches, and each branch corresponds to a certan dscrete value of MB-LBP features. The wea classfer can be defned as: f where a, f x =... S ( x ) = f ( x,, x,, x ) = a j, f x = j (7)... a256, f x = 256, a j s defned as follows: a j = w yδ ( x = j). (8) wδ ( x = j) x,..., x,..., Feature vector elements for mages from the tranng set are values of all possble MB-LBP. The general number s of all possble MB-LBP features s defned by chosen wndow sze. In step t, the wea classfer f t (x) s chosen so as to mnmze the weghted squared error: f F 2 f ( x ) = mn w ( y f ( x )). (9) t = After that the weghts are updated: w ' y f t ( x ) = we. (0) x S F. Constructng strong classfer: The strong classfer s constructed as a superposton of wea classfers: T F = f. () = For three scales of eye patterns three strong classfers F (x), F (x), F (x) are constructed accordng (). To acheve hgh localzaton precson we used a probablstc cascade (P-Cascade) [5]. Frstly, we search m ponts on face mage n larger scale wth the hghest probablty values F p e = F F. (2) e + e These ponts are rough assessment of eye poston. For selected m ponts and ther neghbor pxels n standard scale we choose m 2 wth the hghest probablty F e p = F F ( x). (3) e + e The fnal eye coordnate s the pont wth the hghest probablty p, calculated smlar to (2) and (3) for smaller case. For facal mages wth low resoluton only coarse classfer s used. Smlarly, for mages wth hgh resoluton the bgger number of classfers can be constructed. Thus, the algorthm can be adapted for mages of dfferent qualty. III. EVALUATIO To llustrate the performance of the proposed method, we conduct experments on the BoID database [8] and FERET database [9]. The left and rght eye centres are

5 annotated and provded together wth the mages n ths databases. Samples of mages from the BoID and FERET are shown n Fg. 5. In experments, the proposed method based on MB-LBP s compared wth state-of-the-art methods: the smple Bayesan approach [20] and algorthm based on mage gradents [2]. The FERET mages are all taen ndoors, wth good resoluton, mage qualty, and lmted varaton n lghtng. Pose of the faces n these mages s typcally very close to frontal. In the experments, we dvde 3,363 mages from FERET nto two parts, one for tranng the other for testng. Ground truth eye postons are provded for all mages. MB- LBP and Bayesan algorthms are traned on 000 of 3,363 mages whle tranng for gradent detector s not requred. Of the rest 2,363 frontal mages, the 2,350 mages for whch the face detector detected a face correctly were retaned for testng. The BoID database conssts of,52 grey level mages of 23 dfferent subjects and has been taen n dfferent locatons and at dfferent daytmes, whch result n varable llumnaton condtons comparable to outdoor scenes. Of the,52 mages, the,469 mages for whch the face detector detected a face correctly were used as the test set for all algorthms. The normalzed error [22] s used to evaluate the error between the localzed eye postons and the ground truth: max( l l g, r rg ) err =, (4) l r g where l g and r g are the ground truth postons of the left and rght eye respectvely; l and r are the eye postons localzed by an algorthm. g (a) Fg. 5. Samples of mages from databases: a FERET; b BoID (b)

6 Fg. 6. Results of eye center localzaton for the BoID database Fg. 7. Results of eye center localzaton for FERET database IV. EXPERIMETAL RESULTS In ths secton we report results of the three approaches: the smple Bayesan approach, MB-LBP and algorthm based on mage gradents. The normalzed error curves on FERET and BoID are shown n Fg. 6 and Fg. 7. The curves show the normalzed number of mages whch has the normalzed error rate equal to or less than correspondng value on axs of abscssae. From the results on the BoID database (Fg. 6) we can see that MB-LBP has better precson n large error range, but poorer precson n small error range. MB-LBP localzes eye centers n 00% of mages wth error rate equal to or less than It also has error rate equal to or less than 0. for more than 90% of mages whle other methods process the same number of mages wth much bgger error rate. From the results on FERET database (Fg. 7) we can see that the performances of all methods are very smlar due to the good qualty of FERET. The tme costs for the smple Bayesan approach, MB-LBP and algorthm based on mage gradents are shown n Table I. It should be noted that tme costs are recorded on facal mage wth resoluton 70x70 pxels. TABLE I. LISTS OF TIME COSTS EVALUATED O FACIAL IMAGE Algorthm Gradent Bayesan MB-LBP Tme of processng 587 ms 367 ms 44 ms Vsual examples of results of eye center localzaton made by MB-LBP algorthm are shown on Fg. 8. V. COCLUSIO Ths paper focuses on the eye center localzaton problem. The new algorthm based on MB-LBP s descrbed. The results of testng show that MB-LBP algorthm has better precson n large error range especally on the challengng BoID database and outperforms the smple Bayesan approach and algorthm based on mage gradents. More than that, MB-LBP s almost 0 tmes faster than two others state-of-the-art approaches. Thus, MB-LBP can be a good decson for the speed-accuracy trade-off n eye localzaton. In the structure of Emergency Control Mnstry of Russa eye localzaton and face recognton technology s planned for use n vdeo survellance systems n the hardwaresoftware complex "Safe Cty". Ths program based to ntroduce the streets of Russan ctes, ncludng the major busness centers crowded wth people, the newest automated survellance, montorng and control, desgned for the needs of the emergency servces. The development of ths research wll help to solve the problem of creatng addtonal opportuntes to optmze the exstng system of montorng the state of publc safety; create and develop a system of stuatonal analyss of the causes of destablzaton and forecastng of exstng and potental hazards for publc safety. ACKOWLEDGMET Ths wor was supported by project #060 n the framewor of the base part of the state of R&D jobs of P.G. Demdov Yaroslavl State Unversty and RFBR grants ( and )

7 Fg. 8. Vsual examples of results of eye center localzaton made by MB-LBP algorthm on FERET database REFERECES [] T. Ropa, T. Boult, The eyes have t, n Proceedngs of ACM SIGMM Multmeda Bometrcs Methods and Applcatons Worshop, 2003, pp [2] Z. Zhu, K. Fujmura, Q. J, Real-tme eye detecton and tracng under varous lght condtons, n Proceedngs of the 2002 Symposum on Eye Tracng Research and Applcatons, vol. 25, 2002, pp [3] Z. Zhu, Q. J, Robust real-tme eye detecton and tracng under varable lghtng condtons and varous face orentatons, Computer Vson and Image Understandng 98 (), 2005, pp [4] P. Wang, M. Green, Q. J, J. Wayman, Automatc eye detecton and ts valdaton, n Proceedngs of IEEE Conference on Computer Vson and Pattern Recognton, vol. 3, 2005, pp [5] F. Song, X. Tan, S. Chen, Z.H. Zhou, A lterature survey on robust and effcent eye localzaton n real-lfe scenaros, Pattern Recognton, 46(2), 203 pp [6] The Yale Face Database, Web: //cvc.yale.edu/projects/ yalefaces/yalefaces.html. [7] Robotcs Database, Web: Databases/. [8] J. Marques,.M. Orlans, A.T. Pszcz, Effects of eye poston on egenface-based face recognton scorng, Techncal Paper of the MITRE Corporaton, October [9] Z. Cao, Q. Yn, X. Tang, J. Sun, Face recognton wth learnngbased descrptor, n Proceedngs of IEEE Conference on Computer Vson and Pattern Recognton, 200, pp [0] R. Javer, V. Rodrgo, C. Maurco, Recognton of faces n unconstraned envronments: a comparatve study, European Assocaton for Sgnal Processng Journal on Advances n Sgnal Processng, 2009, pp. 9. [] S. Shan, Y. Chang, W. Gao, B. Cao, P. Yang, Curse of ms- algnment n face recognton: problem and a novel ms-algnment learnng soluton, n: Proceedngs of IEEE Conference on Automatc Face and Gesture Recognton, 2004, pp [2] Ojala, T., Petänen M., Harwood D. Performance evaluaton of texture measures wth classfcaton based on Kullbac dscrmnaton of dstrbutons, 2th IAPR Internatonal Conference on Pattern Recognton (ICPR), vol., 994. pp [3] Petru, V.I., Samorodov A.V., Sprdonov A.., Usng of Local Bnary Patterns n Face Recognton, Bulletn of the Moscow State Techncal Unversty named after Bauman, 20. S. pp [4] L. Zhang, R.Chu, S. Xang, S. Lao, S.Z. L, Face Detecton Based on Mult-Bloc LBP Representaton, Advances n Bometrcs, Lecture otes n Computer Scence, vol. 642, 2007, pp. -8. [5] D. Y, Z. Le, S.Z. L. A robust eye localzaton method for low qualty face mages, Internatonal Jont Conference on Bometrcs (IJCB), 20, pp [6] P.K.Sur, Amt Verma, Robust Face Detecton Usng Crcular Mult Bloc Local Bnary Pattern and Integral Haar Features, (IJACSA) Internatonal Journal of Advanced Computer Scence and Applcatons, Specal Issue on Artfcal Intellgence, 20. [7] Y. Martínez-Díaz, H. Méndez-Vázquez, Y. Plasenca-Calaña, E.B. García-Reyes, Dssmlarty Representatons Based on Mult-Bloc LBP for Face Detecton, Progress n Pattern Recognton, Image Analyss, Computer Vson, and Applcatons, Lecture otes n Computer Scence, vol. 744, 202, pp [8] The BoID database, Web: Face-Database. [9] The FERET database, Web: ad/humand/feret. [20] M. Everngham, A. Zsserman, Regresson and classfcaton approaches to eye localzaton n face mages, Automatc Face and Gesture Recognton, FGR [2] F. Tmm, E. Barth, Accurate eye centre localsaton by means of gradents, n Proceedngs of the Int. Conference on Computer Theory and Applcatons (VISAPP), vol., 20, pp [22] Z.-H. Zhou and X. Geng. Projecton functons for eye detecton, Pattern Recognton, 37(5): 2004, pp