FACIAL EXPRESSION RECOGNITION BASED ON CLOUD MODEL

FACIAL EXPRESSION RECOGNITION BASED ON CLOUD MODEL Hehua Chi a, Liahua Chi b *, Meg Fag a, Juebo Wu c a Iteratioal School of Software, Wuha Uiversity, Wuha 430079, Chia - hehua556@163.com b School of Computer Sciece ad Techology, Huazhog Uiversity of Sciece ad Techology, Wuha 430074, Chia - liahua_chi@163.com c State Key Laboratory of Iformatio Egieerig i Surveyig, Mappig ad Remote Sesig, Wuha Uiversity, Wuha 430079, Chia - wujuebo@gmail.com KEY WORDS: Cloud Model, Facial Expressio Recogitio, Backward Cloud Geerator ABSTRACT: Facial expressio is oe of the major features of facial recogitio i recet years, ad it has become a hotspot. I this paper, we preset a ovel method of facial recogitio based o cloud model, i combiatio with the traditioal Facial expressio system. Firstly, we carry out the trasformatio from images ito grids with M by N, where M ad N deote the actual image positioig of the grid. Each grid is a gray value (0-255) ad the grids stad for the data from data poits to data sets based o cloud model. Secodly, we do data pre-processig for the origial facial expressios of iput images. Cloud droplets image ca be obtaied as the iput of backward cloud geerator i order to extract the three umerical characteristics, that is, Ex, E ad He. With these three characteristics, facial expressio ca be realized. Fially, i order to demostrate the feasibility of the preseted method, we coduct a case study of facial expressio recogitio based o cloud model. The results show that the method is feasible ad effective i facial expressio recogitio. 1. INTRODUCTION Expressio is a basic way to express makid s feeligs ad is oe kid of effective commuicatio. The facial expressios have correspodig chage before people expressig their emotios. The facial expressio ca ot oly express their thoughts ad feeligs accurately ad subtly, but also describe the others cogitive attitudes ad ier world. Facial expressio cotais rich huma behaviours ad is a kid of iformatio resources i huma-computer iteractio with more effective, atural ad direct way. If computers ad robots have the ability to uderstad ad express feeligs as me s adaptig to the eviromet, it will chage the relatioship betwee the computer ad huma fudametally. If that, the computer ca better service to makid. It is the research meaig of this topic o facial recogitio with emotio uderstadig ad emotio expressio [1-3]. Therefore, the facial expressio recogitio ca be achieved through the observatio ad aalysis of face images. The facial recogitio is a idetificatio task with a o-cotact way, which is so vital to realize the iteractio betwee ature ad ma-machie [4]. Facial expressio recogitio is a hot research topic i computer visio, emotio ad image processig, which ca be widely applied i huma-computer iteractio, multi-media, security, medical assistace, ad behavioural sciece, etc. May scholars have lauched a lot of studies o facial expressio recogitio ad the mai research results are as follows: M. Patic et al. preseted a method of emotioal expressio classificatio based o a expert system [5]. Y. L. Tia et al. itroduced recogizig actio uits for facial expressio aalysis based o the behaviour idetifyig [6]. X. X. Yua et al. gave a way for face recogitio based o the wavelet aalysis ad support vector machie [7]. I this paper, we proposed a ovel approach of facial expressio recogitio based o cloud model, aimig to mie the hidde kowledge of facial expressio ad the facial features with cloud model. 2.1 Cloud model 2. BASIC PRINCIPLE Defiitio: Suppose U is a quatitative uiverse of discourse with precise umerical value: X U ad T is qualitative cocept of space U. If the certaity of x (x X) belogig with T is a radom umber with stable tedecy, that is, C T (x) [0, 1], the the distributio of cocept T from U mappig to [0, 1] i data space is called cloud [8], where meets: C T (x):u [0,1] Cloud model has three characteristics: x X (X U) x C T (x) Expectatio (Ex) is the prototype value (cetre or stadard value) of cocept, ad is the most represetative value of the qualitative cocept. Etropy (E) is the measuremet of cocept ucertaity while Hyper-etropy (He) is the measuremet of etropy ucertaity, that is, the etropy of etropy. Cloud model has the characteristics with macro accurate, micro fuzzy, macro cotrollable ad micro ucotrollable. Its essetial uit is cocept cloud composed of cloud droplets, icludig radomess ad fuzziess. It is the orgaic sythesis of fuzziess ad radomess i ature laguage, ad cotais the mappig betwee quatitative ad qualitative data. The theory is a breakthrough for limitatios of hard computatio i * Correspodig author. liahua_chi@163.com. 124

probability ad statistics, but also solves the iheret defect of membership fuctios. It is a ew method ad ew techology for solvig problems i data miig, ad breaks the limitatio of boudary sets. As a geeral mathematics theory, the cloud model cleverly realizes the aalysis betwee qualitative ad quatitative data. With the mathematical coversio method ad techology developmet, it has bee widely ad successfully applied i the kowledge discovery, the spatial data miig system, itelliget cotrol, efficiecy evaluatio, solutio or explai atural, social problems or pheomeo, ad have achieved remarkable results. 2.2 Backward cloud geerator expressio recogitio system maily icludes the followig steps: the acquisitio of the facial expressio images, the face detectio, the facial expressio feature extractio ad the facial expressio recogitio. Its structure is show i Figure 2. For a automated facial expressio recogitio system, first, we should obtai static images or facial image sequeces; The secod step is the facial image pre-processig, icludig the face detectio ad the image ormalizatio ; The third step is the facial image feature extractio, icludig the origial feature acquisitio, the feature dimesioality ad extractio, the feature separatio; The fourth step is the facial expressio recogitio, that is, accordig to the extracted features ad some criteria, we realize the classificatio. Backward cloud geerator is the model of ucertaity trasformatio betwee umerical value ad laguage value mappig from the qualitative to quatitative data [9-10]. It turs a certai amout of accurate data to correspodig qualitative value {Ex, E, He} effectively, ad reflects the whole cloud droplets accordig to these accurate data. The more the amout of cloud droplets are, the more accuracy the cocepts will be. Backward cloud geerator is a process of cloud geerator idirectly ad reversely, which regards a group of cloud droplets Drop(x i, C T (x i )) with a certai distributio as samples, ad geerates the three umerical characteristics {Ex, E, He} correspodig to the cocepts (show as figure 1). Through the forward ad the backward geerator, the cloud model makes the establishmet betwee the qualitative ad quatitative relatioship. Figure 1. The iput ad output of backward cloud geerator Iput: The sample poits ad their certaity degrees C T ( x i ) (i=1, 2..., N) Output: The umerical characteristics of qualitative cocept, Ex, E ad He. The details are as follows: x i (1) Accordig the sample, calculate the sample mea: 1 X = x i i= 1, the first-order absolute cetre distace: 1 2 1 2 M Ι = x i X S = ( x i X ) i= 1, ad the variace 1 i= 1. (2) Compute Ex, Ex = X. π E = M Ι (3)Compute E, 2. 2 2 (4)Compute He, He = S E 3. FACIAL EXPRESSION RECOGNITION As a challegig cross-subject betwee the biological feature recogitio ad the affective computig field, the facial expressio recogitio techology drive by a variety of applicatios has the rapid developmet [11]. The facial Figure 2: The model of facial expressio recogitio 3.1 Facial image pre-processig I the process of pre-treatmet, the facial detectio ad localizatio are applied firstly, amely, to fid the positio ad existig face to face segmetatio from the backgroud from the iput. The the facial images are doe data ormalizatio, such as gray ormalizatio, etc. 3.2 Face detectio ad localizatio Face detectio ad localizatio is the primary problem to solve i the automatic idetificatio system of facial expressio, icludig the face detectio i simple backgroud ad complex backgroud. At the begiig of the study, the face images i database are all simple backgroud, which meas the differece betwee face ad backgroud is big ad most of them are positive images. The research value of the latter is more practical ad theoretical. I detectig ad locatig face image, the image must be ormalized i order to facilitate subsequet processig. 3.3 Image ormalizatio Image ormalizatio icludes geometry ormalizatio ad gray ormalizatio. The former meas covertig the face results to the same positio ad size. The latter is to stretch the image gray ad improve the image cotrast. It also icludes light compesatio ad to overcome of the light chages. After that, 125

we ca get more suitable images i the process by image preprocessig ad edge extractio. 3.4 Facial Feature Extractio After the pre-processig, we ca make the feature extractio ad selectio o the facial images, that is, extractig the facial expressio feature iformatio. The purpose is to obtai a set of category features, that is to say, we obtai the feature vectors that the umber of features ad the classificatio error rate are fewer. It is a very importat part. The effect will directly affect the correct recogitio rate of the facial expressio. 3.4.1 The origial feature acquisitio: We use some iformatio to obtai the origial features of the expressios, such as features of shape, geometric relatios, local texture, optical flow ad so o. This step is called as the origial features acquisitio. However, these primitive characteristics geerally exists the redudacy ad other issues. I order to more effectively characterize the ature of the facial expressio, we eed to make the process o the origial feature data. 3.4.2 Feature dimesioality ad extractio: Because the dimesios of the origial features are usually very large, we should covert them ito the low-dimesioal subspace. That ot oly make the dimesio of the origial features sigificatly reduced, but also the validity of these lowdimesioal space will be icreased. I recet years, we make some ew research o the methods of the feature dimesioality ad extractio. 3.4.3 Feature separatio: Facial images cotai a wealth of iformatio. For differet recogitio tasks, the iformatio also varies. The facial detectio is to fid the cosistecy of the facial images. The facial recogitio eed to preset the idividual differeces amog the facial expressios. Recetly, a ew solutio is to separate the differet factors of the huma facial expressio, such as the expressio factors ad idividual factors, avoidig the iterferece of other factors. 3.5 Facial expressio classificatio Expressio classificatio refers to the defiitio of a group of categories, ad to desig appropriate mechaisms for the expressio recogitio. If the expressios are classified accordig to the facial movemets (FACS), facial actios are classified ito 44 AUs (actio uits). I accordace with the emotio classificatio, the expressios are classified ito seve kids of basic emotios (cryig, surprise, happiess, ecstasy, uwillig, frustratio, fear). 4. FACIAL RECOGNITION BASED ON CLOUD MODEL 4.1 The process of facial recogitio based o cloud model Cloud theory portrays the distace relatioship betwee each elemet i the domai ad its core cocept usig the membership. The greater degree of the membership, the elemets are much closer to the core cocept. This feature is the same as the facial expressio recogitio s feature that we obtai the facial expressio feature ad make the classificatio. So we ca use the cloud theory to obtai the facial expressio feature, usig the umerical characteristics of the cloud to express the facial expressio features. Makig use of the cloud model algorithm to extract facial expressio features, we propose a ew facial expressio recogitio method - facial expressio recogitio based o cloud model. First, iput a group of primitive facial images; Secod, preprocess the iput images to get a set of stadard cloud droplet images; Third, make use of backward cloud geerator to realize the image feature extractio ad output the umerical characteristics (Ex, E, He) of this cloud droplet images; Fourth, make the umerical characteristics (Ex, E, He) as the facial expressio features; At last, use the umerical characteristics (Ex, E, He) to realize the facial expressio classificatio. The chart of facial expressio recogitio based o Cloud Model is show i Figure 3. Facial Expressio Images A group of cloud droplet images Facial expressio features based o cloud model Facial Expressio Classificatio Facial Expressio image preprocessig Facial Feature Extractio Backward cloud geerator Figure 3: The structure of facial recogitio based o cloud model 4.2 A group of images of cloud droplets Macro accuracy ad micro fuzziess are the features of cloud model, with macro cotrollable ad micro ucotrollable. Its essetial uit is cloud droplets, which ca form cloud with such cloud droplets ad realize the trasformatio betwee qualitative ad quatitative data. It reflects the ucertaity of kowledge represetatio. After image pre-processig,a group of cloud droplets images ca be obtaied for the origial face expressios. Such images are cosidered as the stadard iput images for the followig processig. 4.3 The characteristics of facial expressio based o cloud model The umerical characteristics of cloud reflects quatitative feature of qualitative cocept with Ex, E ad He. It is the umerical basis for describig cloud model ad miig kowledge from ucertaity data. The facial expressio is a kid of ucertaity data. This paper uses cloud geerator to fid kowledge from facial expressios, that is {Ex, E, He}, ad to achieve facial expressio recogitio upo such characteristics. 126

5. THE EXPERIMENT OF FACIAL EXPRESSION RECOGNITION BASED ON CLOUD MODEL differeces, we ca get the image category ad achieve facial expressio recogitio. 5.1 The experimet of facial recogitio The data source comes from Japaese Female Facial Expressio (JAFFE) database, which is a ope face image database (http://www.kasrl.org/jaffe_dowload.html). It cotais 10 wome's expressios, icludig the people KA, KL, KM, KR, MK, NA, NM, TM, UY ad YM. Each perso has 7 differet expressio as AN, DI, FE, HA, NE, SA ad SU. Each expressio has 3 or 4 samples ad the total umber is 216. I this experimet, we coduct kowledge miig by cloud model for facial expressio images, aimig to fid the umerical characteristics ad realize the facial expressio recogitio. 5.2 Sample sets traiig 5.2.1 The experimet of differet expressios for oe perso: I JAFFE database, the attribute KA is selected at the begiig, while the origial images are chose from the te Japaese wome's KAs, icludig AN, DI, FE, HA, NE, SA ad SU. By backward cloud geerator, the KAs of the same expressio from differet Japaese wome ca trasform to the three umerical characteristics {Ex, E, He} of cloud, as show i lie 1, table 1. By usig the same method as KA processig, the correspodig umerical characteristics ca be obtaied for KL, KM, KR, MK, NA, NM, TM, UY, YM ad the results are show i table 1 from colum 2 to colum 10. Exp /P KA KL KM KR MK NA NM TM UY YM Ex E He AN DI FE HA NE SA SU Ex E He 5.2.2 The same expressios of differet people: I JAFFE database, the attribute AN is selected at the begiig, while the origial images are chose from the te Japaese wome's ANs, icludig KA, KL, KM, KR, MK, NA, NM, TM, UY ad YM. By backward cloud geerator, the ANs of the same expressio from differet Japaese wome ca trasform to the three umerical characteristics {Ex, E, He} of cloud, as show i colum 1, table 1. By usig the same method as AN processig, the correspodig umerical characteristics ca be obtaied for DI, FE, HA, NE, SA, SU ad the results are show i table 1 from colum 2 to colum 7. 5.3 Sample sets traiig (1) Every lie i table 1 meas the iput is the differet expressios of the same perso, ad the output is the umerical characteristics of cloud of such iput images {Ex, E, He}. (2) Each colum i table 1 meas the iput is the same facial expressio of te persos, ad the output is the umerical characteristics of cloud of such iput images {Ex, E, He}. (3) The problem is that how to idetify the facial image belog to whose expressio if existig a face image for recogitio? The method is as follows: Firstly, geerate the umerical characteristics {Ex, E, He} of cloud for the origial face expressios. Secodly, compute the umerical characteristics {Ex, E, He} of cloud by backward cloud geerator by addig the image to be idetified to such origial face expressios. Fially, compare two groups of umerical characteristics {Ex, E, He} of cloud to fid the differeces. Accordig to such Table 1: The traiig samples of facial expressio 5.4 Facial recogitio based o the samples After the traiig of samples, we choose two groups as the origial facial image for perso, as show i table 2. Perso AN DI FE HA NE SA SU KA KL Table 2 two groups of origial facial expressio images The face image for idetificatio The experimetal steps are: Step 1: Choose the first lie i table 2 as the origial image with backward cloud geerator ad calculate the {Ex, E, He} of the image. The results are show i lie 1 i table 1. Step 2: Add the facial expressio images ito lie 1 i table 2 for idetificatio. By backward cloud geerator, compute the {Ex, E, He} of the image as show i lie 1 i table 3. Step 3: Select lie 2 i table 2 as the origial facial expressio image. Based o backward cloud geerator, geerate the {Ex, E, He} of image as show i lie 2 i table 1. Step 4: Add the facial expressio images ito lie 2 i table 2 for idetificatio. By backward cloud geerator, compute the {Ex, E, He} of the image as show i lie 2 i table 3. 127

The lie 3 i table 3 shows the differece value of {Ex, E, He} for images i the secod to first steps while the lie 4 i table 3 gives the fourth to third steps. Characteristic The secod The fourth Secod-First Fourth-Third Ex E He Table 3 The results ad comparative results Observig from lie 3 ad 4 i table 3, it ca be obviously see that the {Ex, E, He} of the former differet image is ot clearer tha the latter's. We ca kow that the facial expressio of the image for idetificatio is more close to A, which is correct. Therefore, it is feasible of that cloud model ca achieve face facial image recogitio. The research develops the cogitio of cloud model theory ad further expads the applicatio fields of cloud model. 6. CONCLUSIONS As a mathematical trasformatio model with kowledge ucertaity, the cloud model itegrates the fuzziess with the radomess ad forms the qualitative ad the quatitative mappig betwee them. This paper put forward a ew method of facial expressio recogitio based o cloud model. By usig cloud model, the facial expressio recogitio ca be carried out effectively, ad it expressed the ucertaity of facial expressio. The quatitative umerical characteristics {Ex, E, He} of facial expressios were mied by the backward geerator of cloud model. I this paper, the hidde kowledge i facial expressio images were obtaied with the umerical characteristics {Ex, E, He} of cloud model. Ex is the characteristics of the facial image i commo, E is the persoality deviatio of geeral commo kowledge, ad He is the discrete level of kowledge. I aalyses of facial image kowledge, by the umerical characteristics {Ex, E, He}, the facial expressio ca be realized. The experimetal results showed that this method ca effectively achieve facial recogitio. Furthermore, the facial expressio recogitio ad its applicatio based o cloud model should be further study i ext step. [2] X. M. Liu, H. C. Ta, Y. J. Zhag. New Research Advaces i Facial Expressio Recogitio. Joural of Image ad Graphics, 11(10), 2006. [3] K. MaSe, A. Petlad. Recogitio of Facial Expressio From Optical flow [J]. IEICE Tras, 74(10), 1991, pp.3474-3484. [4] M. Patic, L. J. M. Rothkratz, Automatic Aalysis of Facial Expressios:the State of the Art. IEEE tras. Patter Aalysis ad Machie Itelligece, 22(12), 2000, pp.1424-1445. [5] M. Patic, L. J. M. Rothkratz. A Expert System for Multiple Emotioal Classificatio of Facial Expressios. Pro.11th IEEE It. Cof.o Tools with Artificial Itelligece, 1999, pp.113-120. [6]Y. L. Tia, T. Kaade, J. f. Coh. Recogizig Actio Uits for Facial Expressio Aalysis. IEEE Tras. Patter Aalysis ad Machie Itelligece, 23(2), 2001, pp.97-115. [7] X. X. Yua, W. Jiag, L. Zhag. Facial expressio recogitio method based o wavelet eergy feature ad Support Vector Machies. Optical Techology, 34(2), 2008. [8] K. C. Di, D. Y. Li, D. R. Li. Cloud Theory ad Its Applicatios i Spatial Data Miig ad Kowledge Discovery. Joural of Image ad Graphics, 4(11), 1999. [9] H. J. Lv, Y. Wag, D. Y. Li. The Applicatio of Backward Cloud i Qualitative Evaluatio. Chiese Joural of Computers, 26(8), 2003. [10] D. R. Li, S. L. Wag, D. Y. Li. Spatial Data Miig Theories ad Applicatios. Sciece Press, 2006. [11] M. Qiao, Y. J. Che. Feature Extractio Methods o Facial Expressio Recogitio. Joural of Chogqig Istitute of Techology, 22(6), 2008. 7. ACKNOWLEDGEMENTS This paper is supported by Natioal 973 (2006CB701305, 2007CB310804), Natioal Natural Sciece Fud of Chia (60743001), Best Natioal Thesis Fud (2005047), ad Natioal New Cetury Excellet Talet Fud (NCET-06-0618). 8. REFERENCES [1] N. Zhag. Summarizatio for the facial expressio recogitio. Joural of Shadog Istitute of Light Idustry(Natural Sciece Editio), 21(4), 2007. 128