Performance Comparison of Visual and Thermal Signatures for Face Recognition

Transcription

1 Performance Comparison of Visual and Thermal Signatures for Face Recognition Besma Abidi The University of Tennessee The Biometric Consortium Conference 2003 September 22-24

2 OUTLINE Background Recognition Engine Test Database Information Fusion Data fusion Decision fusion Results Thermal Face Recognition Conclusions and Future work 2

3 Background Facial variations due to varying illumination, expression, and rotation degrade the performance of visually based face recognition systems. The use of thermal imagery has great advantages in poor illumination conditions, where visual face recognition systems often fail. Glasses block thermal emissions around the eyes and need to be taken into consideration. FaceIt, a face recognition commercial software package, highly ranked by the face recognition vendor test (FRVT), is used for the comparison of recognition rates using visual and thermal face images. 3

4 Recognition Engine FaceIt Create Gallery Select Subject Rank by Confidence Rates 4 Images are courtesy of Equinox Corporation

5 Test Database Database (Co-registered visual and LWIR images from Equinox) Total DB ( 1,622*2 = 3,244) - 90 individuals Eye alignment using FaceIt! Visual : Automatic(95%), manual (5%)! Thermal (LWIR): Eye coordinates from co-registered visual image! Fused image : Eye coordinates from visual image Size Glasses Lighting Expression Gallery 90 Off Overhead Neutral Probe Off Overhead Varying Probe Off Left Varying Probe Off Right Varying Probe On Overhead Varying Probe On Left Varying Probe On Right Database specifications with co-registered visual/thermal Varying 5

6 Information Fusion Data fusion (Low Level) Co-registered visual and thermal images Average of the 2 images used as a simple data fusion Weighted summation considered for a fully automated engine Decision Fusion (High Level) Visual and thermal recognition results from FaceIt Average and higher confidence rate used and compared 6

7 Sample Data Fusion 1 st row : Visual, 2 nd row: Thermal, and 3 rd row : Fused images Images courtesy of Equinox Corporation 7

8 Decision Fusion Decision fusion based on average confidence rates 8

9 Comparison of Visual and Thermal Recognition (glasses off) Thermal images give better performance than 9 visual images in all cases

10 Comparison of Fusion Methods (glasses off) Decision fusion (average, higher confidence) 10 gives the best results

11 Comparison of Fusion Methods (glasses on) Using FaceIt, recognition of individuals wearing glasses in thermal images results in poor performance 11

12 Radiometric Calibration Radiometric calibration establishes a direct relationship between the gray value response at a pixel and the thermal emissions. Radiometric calibration standardizes all thermal IR data, when taken in different environments. Helpful in differentiating between running people, indoor/outdoor conditions, and possibly nervousness, Both absolute and relative differences of thermal signatures are needed for a comprehensive and accurate solution. 12

13 Face Detection in Thermal Images (a) Input Image (b) Applying threshold (a) (b) (c) (c) Morphological Operation (d) Connected Components (e) Fitting Ellipses (f) Cropped Face region (d) (e) (f) 13

14 Thermal Face Normalization Recognition algorithms developed for visual face recognition can be used for thermal face recognition. The eyes, which are usually used for the normalization of visual faces, cannot be easily detected in thermal images. The nose is usually the coldest area, but not always. Edge detection, ellipse fitting, and multithresholding approaches used for face scaling and orientation. Use of co-registered visual images helps in the normalization process. x 14 x

15 Detecting Glasses in Thermal Images (a) (b) (c) (a) Input Image, (b) Thresholding and Connected Components Analysis, (c) Ellipse Fitting Classification : Size comparison, location, and symmetry of the ellipses are used to recognize the presence of glasses. Weighted summation applied to implement data fusion. More or less weights are given to thermal component as function of presence or absence of glasses. 15

16 Conclusions Thermal images give better performance than visual images when individuals are not wearing glasses. Both fusion of visual and thermal images (Data Fusion) or recognition results (Decision Fusion) increase the overall performance of face recognition systems. 16

17 Future Work Fusion scheme generalized to the case of individuals wearing glasses using weighted summation and/or neural network decision schemes. Feature level fusion methods will be investigated by considering variations in visual and thermal images. 17