123 CHAPTER 6 MULTIMODAL BIOMETRIC RECOGNITION SYSTEM Some of the challenges faced by the unimodal biometric systems are noisy data, lack of universal data, less performance improvement and vulnerable to spoof attacks. Hence, multimodal biometric system is preferred, which refers to the system that establishes identity based on combining two or more biometric traits, i.e., fuse multiple traits of an individual or extraction of multiple features and matching algorithms operating on the same biometric. The most compelling reason to combine different biometric traits is to improve the identification accuracy significantly and provide larger population coverage. This can be done when features extracted from different biometrics are statistically independent. 6.1 PROPOSED MULTIMODAL BIOMETRIC SYSTEM The block diagram of the proposed multimodal biometric recognition system is shown in Figure 6.1. The biometric images obtained from each modality are preprocessed, transformed using multi-resolution transforms such as Curvelet and Ridgelet, and then, features are extracted from these transformed output images. These extracted multi-resolution features are compared against the stored templates to generate the match scores.
124 DA Iris DA Fingerprint DA Face Preprocessing Preprocessing Preprocessing Multi-resolution Transforms Features Extraction Features Extraction Features Extraction Matching Score Matching Score Matching Score Score level Fusion Stored Templates Decision? Yes Genuine No Imposter DA Data Acquisition Figure 6.1 Block Diagram of Proposed Multimodal Biometric Recognition System A match score is a measure of similarity between the input and template biometric feature vectors. When match score output by different
125 biometric matchers are consolidated in order to arrive at a final recognition decision, fusion is said to be done at the match score level. This is also known as fusion at the measurement level or confidence level. After generating matching scores for each modality, the scores are normalized using various techniques such as min-max (MM), max (MX), median-mad (MAD), tanh (TH), double-sigmoid (DS), and z-score (ZS). 6.2 DIFFERENT LEVELS OF FUSION TECHNIQUES The various levels of fusion techniques are broadly categorized into two namely, pre-classification or fusion before matching, and postclassification or fusion after matching (Jain et al 2008). Different fusion techniques that can be accomplished at various levels in a biometric system are shown in Figure 6.2. Biometric Fusion Before matching After matching Sensor Level Feature Level Classifier Fusion Match-Score Level Rank Level Decision Level Figure 6.2 Different Levels of Fusion Techniques in a Biometric System
126 The sensor level and feature level techniques fall into the first category, while the second category fusion techniques are match-score level, rank level and decision level. 6.2.1 Sensor Level Fusion Technique It combines information at the raw data obtained using multiple sensors or multiple snapshots of a biometric using a single sensor. Mosaicing multiple samples of the same biometric is an example of this type of fusion. 6.2.2 Feature Level Fusion Technique It typically involves enhancing the feature vectors arising from multiple feature extractors and subjecting the fused feature vector to a feature transformation algorithm. This enables the system to compute discriminatory features that can enhance matching performance. 6.2.3 Match-Score Level Fusion Technique Each individual biometric process typically outputs a match score but possibly, multiple scores. The fusion process fuses these scores into a single score, which is then compared to the system acceptable threshold. If the classification approach to score fusion is allowed, then the output may be direct decisions instead of scores. Fusion methods at this level can be broadly classified into three categories namely, density-based, transformation-based and classifier-based schemes (Ross et al 2006). In the density-based scheme, a large number of training samples are necessary to reliably estimate the joint-density function especially if the dimensionality of the feature vector is large. In transformation-based scheme, a score normalization process is essential to transform the multiple match
127 scores generated by multiple matchers into a common domain. The goal of classifier-based scheme is to directly estimate the class rather than to compute an intermediate scalar value. This fusion scheme assumes the availability of a large representative number of genuine and impostor scores during the training phase of the classifier when its parameters are computed. 6.2.4 Rank Level Fusion Technique When a biometric system operates in the identification mode, the system output can be viewed as a ranking of the enrolled identities. In this case, the output indicates the set of possible matching identities sorted in decreasing order of confidence. The goal of rank level fusion scheme is to consolidate the rank output by the individual biometric subsystems in order to derive a consensus rank for each identity. This fusion technique does not need normalization techniques. 6.2.5 Decision Level Fusion Each individual biometric process outputs its Boolean result. The fusion process fuses them together by a combination algorithm such as AND, OR rules etc. Other decision level fusion technique includes majority voting, weighted majority voting, Bayesian decision fusion, etc. 6.3 PROPOSED SCORE LEVEL FUSION TECHNIQUES The matching scores at the output of the individual matchers may not be homogeneous. For example, one matcher may output a dissimilarity measure while another may output a similarity measure. Further, the scores of individual measures need not be on the same numerical scale and may follow different statistical distribution. Due to these reasons, score normalization is
128 essential to transform the scores of the individual matchers into a common domain prior to combining them. 6.3.1 Score Normalization Techniques Score normalization refers to changing the location and scale parameters of matching score distributions at the output of the individual matchers, so that the scores of different matchers are transformed into a common domain. When the parameters used for normalization are determined using a fixed training set, it is referred to as fixed score normalization. In such case, the matching score distribution of the training set is examined and a suitable model is chosen to fit the distribution. Based on the model, the normalization parameters are determined. In adaptive score normalization, the normalization parameters are estimated based on the current feature vector. This approach has the ability to adapt to variations in the input data such as the change in the length of the speech signal in speaker recognition systems. The score normalization techniques used in our work include min-max (MM), max (MX), median-mad (MAD), tanh (TH), double-sigmoid (DS), and Z-score (ZS) normalizations. Let M be the total number of matchers, X i be the set of all the raw output values (matching scores) of the corresponding matcher i and x i X i, i M, where x i is the output raw value of the matcher i. The corresponding normalized matching score is denoted as ns i is the set of normalized matching scores of the matcher i. NS i, where NS i Min-max normalization (MM): It is the simplest normalization technique that normalizes the numerical range of the scores to [0, 1] and also retains the shapes of the original distributions. The min-max normalization function is given by
129 where min(x i ) and max(x i ) are the minimum and maximum values of the matching scores. Max normalization (MX): This normalization technique is constructed from the above min-max normalization technique by assigning zero to the min value. The max normalization function is given by Median-MAD normalization (MAD): This normalization scheme would be relatively robust to the presence of noise in the training data and is given by The denominator value is Median Absolute Deviation (MAD), which is an estimate of the scale parameter of the feature value. Tanh-normalization (TH): It is similar to Z-score normalization, which is also computed using mean and standard deviation of the match scores. The tanh-normalization function is given as: where µ and respectively. denote the mean and standard deviation of the match scores
130 Double-sigmoid normalization (DS): This normalization scheme provides a linear transformation of the scores in the region of overlap, while the scores outside this region are transformed non-linearly. The double-sigmoid normalization function is given by where is the reference operating point and 1 and 2 denote the left and right edges of the region in which the function is linear. The function exhibits linear characteristics in the interval ( - 1, - 2). Z-score normalization (ZS): The most commonly used score normalization technique is the Z-score which is calculated using the arithmetic mean and standard deviation of the match scores. It is given by where µ i and i are the arithmetic mean and standard deviation respectively for the i th matcher. This scheme can be expected to perform well if the average and the variance of the score distributions of the matchers are available, if not known, they can be estimated based on the given training set. Table 6.1 shows the summary of the various matching score level normalization techniques as cited in Ross et al 2008. It states that the tanh and double sigmoid techniques are more robust and their efficiencies are high.
131 Table 6.1 Summary of Normalization Techniques Normalization techniques Robustness Efficiency Min-max No N/A Max No N/A Median-MAD Yes Moderate Tanh Yes High Double-sigmoid Yes High Z-score No High 6.3.2 Various Score-level Fusion Techniques Consider NS iris, NS fp, and NS face as the normalized scores of iris, fingerprint and face respectively. Let a, b and c be the weights assigned respectively to the biometric modalities iris, fingerprint and face. During the implementation of weighted fusion techniques, three test cases are considered. For the first test case, equal weights (0.33) are assigned to all the three modalities and for the remaining two test cases, more weights are assigned to iris, next less weight to fingerprint and least weight to the face modality. The chosen weights are based on the possibility of spoofing attacks. Table 6.2 Different Weights Assignment Strategy Weights assigned for Test case # Iris Fingerprint Face 1 0.33 0.33 0.33 2 0.4 0.3 0.3 3 0.5 0.3 0.2
132 Since, this system aims to reduce (or) decrease the vulnerability of spoofing attacks, falsifying iris trait is tough when compared to fingerprint and is still tough when compared to the face modality. Hence, the iris modality is assigned with the highest weight thus leading to better multimodal recognition rate. a takes the value either 0.33 or b takes either 0.3 or 0.33 c takes either 0.2 or 0.3 or 0.33, as listed in Table 6.2. The eight score level fusion techniques namely sum, mean, product, min, max, median, tanh and exponential methods are considered and are listed in Table 6.3. Here, NS final denotes the final normalized score after applying individual fusion techniques. Table 6.3 Various Score Level Fusion Techniques Fusion techniques Formula Weighted Sum Weighted Mean Weighted Min Weighted Max Weighted Median Weighted Product Weighted Tanh Weighted Exponential
133 6.3.3 Decision-Making Technique Threshold value is chosen adaptively by collecting the wrongly classified subjects minimum match scores and applying them in the corresponding fusion techniques. A user is considered to be genuine, if the match score value after fusion is less than the chosen threshold; otherwise as an imposter. 6.4 EXPERIMENTAL RESULTS AND DISCUSSION The proposed multimodal biometric recognition system is tested with the virtual multimodal database formed from the public biometric database. To demonstrate the effectiveness of the proposed methodologies on the virtual multimodal database, the public database considered here are CASIA-IrisV3-Interval database for Iris, FVC2004 DB1_A for fingerprint and ORL database for face. The subjects chosen from each biometric trait is forty with eight samples per subject, leading to 160 images for training and 160 for testing for each biometric modality. Table 6.4 Recognition Rate of the Public Database Modality No. of Subjects Recognition No. of images used for Rate (%) training testing CT RT Iris 40 160 160 76.625 93.625 Fingerprint 40 160 160 88.125 71.875 Face 40 160 160 96.875 79.375 The reason for choosing forty subjects is that the ORL database contains a maximum of forty subjects and reason for choosing eight samples per subject,
134 is that FVC2004 DB1_A database as a maximum of eight samples per subject. Table 6.4 illustrates the recognition rate of individual biometric traits from the public biometric database. It is seen from the Table 6.4 that the Ridgelet Transform performs better for iris images and Curvelet Transform performs better for fingerprint and face images. 6.4.1 Recognition Results for Virtual Multimodal Database Table 6.5 shows the multimodal biometric recognition rate using different normalization and fusion techniques for the virtual multimodal database by considering the above mentioned three test-cases. Eight different score fusion techniques and six different normalization techniques are considered for experimentation. The experimented results are shown in the Table 6.5. Each row in the table defines different fusion techniques and each column defines various normalization techniques. Table 6.5 Recognition Rate using Different Normalization and Fusion Techniques for Virtual Multimodal Database (a) Test Case-1 (b) Test Case-2 and (c) Test Case-3 Normalization techniques Fusion techniques MM MX MAD TH DS ZS Weighted Sum 98.13 93.13 76.25 95.00 91.88 98.13 Weighted Mean 94.38 93.75 68.13 96.25 92.50 95.63 Weighted Min 96.25 92.50 85.63 98.13 96.25 95.63 Weighted Max 83.75 70.00 56.88 78.75 61.25 78.75 Weighted Median 88.13 93.75 93.13 86.25 83.13 82.50 Weighted Product 95.00 93.13 91.25 95.00 92.50 98.75 Weighted Tanh 95.00 93.13 93.75 95.00 91.88 93.75 Weighted Exponential 95.00 93.13 91.25 95.00 92.50 93.75 (a)
135 Table 6.5 (Continued) Normalization techniques Fusion techniques MM MX MAD TH DS ZS Weighted Sum 95.00 93.75 68.13 96.25 92.50 95.63 Weighted Mean 96.25 94.38 76.25 96.25 93.75 95.63 Weighted Min 96.25 93.13 93.13 96.25 93.75 95.63 Weighted Max 96.25 96.25 57.50 97.50 93.13 92.50 Weighted Median 74.38 82.50 96.25 55.63 68.13 80.00 Weighted Product 93.13 94.38 89.38 96.25 93.75 94.38 Weighted Tanh 94.38 94.38 93.13 96.25 93.13 94.38 Weighted Exponential 95.00 93.13 60.63 96.25 91.88 90.00 (b) Fusion techniques Normalization techniques MM MX MAD TH DS ZS Weighted Sum 95.00 94.38 60.63 96.25 91.25 97.50 Weighted Mean 95.00 94.38 60.63 96.25 91.25 97.50 Weighted Min 98.13 96.25 93.13 96.88 93.75 98.75 Weighted Max 97.50 97.50 59.38 97.50 93.13 97.50 Weighted Median 76.88 73.13 95.00 66.25 76.88 81.25 Weighted Product 96.88 94.38 85.00 96.88 93.13 98.75 Weighted Tanh 96.25 94.38 90.63 96.25 93.13 98.75 Weighted Exponential 95.00 94.38 60.00 96.25 90.63 91.88 (c) In the case of virtual multimodal database recognition rates, from Table 6.5, it is clearly seen that for the test case-1, 98.75 % is obtained for the weighted-product fusion technique; for the test case-2, the maximum
136 recognition rate of 97.5 % is obtained for weighted-max fusion technique; for the test case-3, the recognition rate of 98.75 % is achieved for weighted-min, weighted-product and weighted-tanh fusion techniques. 6.4.2 Recognition Results for Self-built Multimodal Database Table 6.6 shows the multimodal biometric recognition rate for the self-built multimodal database for the above mentioned three test cases. Table 6.6 Recognition Rate using Different Normalization and Fusion Techniques for Self-Built Multimodal Database (a) Test Case-1 (b) Test Case-2 and (c) Test Case-3 Normalization techniques Fusion techniques MM MX MAD TH DS ZS Weighted Sum 100.00 100.00 100.00 98.50 100.00 100.00 Weighted Mean 100.00 100.00 100.00 98.50 100.00 100.00 Weighted Min 100.00 100.00 81.00 100.00 100.00 100.00 Weighted Max 98.50 98.50 98.50 97.00 97.50 94.50 Weighted Median 100.00 100.00 99.00 99.50 100.00 99.00 Weighted Product 100.00 100.00 99.50 98.50 100.00 100.00 Weighted Tanh 100.00 100.00 97.50 98.50 100.00 100.00 Weighted Exponential 100.00 100.00 99.50 98.50 100.00 99.50 (a)
137 Table 6.6 (Continued) Normalization techniques Fusion techniques MM MX MAD TH DS ZS Weighted Sum 99.00 100.00 100.00 98.50 100.00 99.00 Weighted Mean 99.00 100.00 100.00 98.50 100.00 99.00 Weighted Min 100.00 100.00 99.50 99.50 100.00 100.00 Weighted Max 99.50 97.00 98.50 93.00 94.50 94.00 Weighted Median 100.00 92.50 85.50 92.00 99.50 98.50 Weighted Product 88.00 100.00 98.50 98.50 100.00 100.00 Weighted Tanh 98.00 100.00 97.00 98.50 100.00 100.00 Weighted Exponential 99.00 100.00 99.50 98.50 100.00 96.50 (b) Normalization techniques Fusion techniques MM MX MAD TH DS ZS Weighted Sum 100.00 99.50 90.50 98.50 100.00 96.00 Weighted Mean 100.00 99.50 100.00 98.50 100.00 96.00 Weighted Min 100.00 100.00 100.00 99.50 100.00 100.00 Weighted Max 99.50 96.00 98.50 93.50 94.50 84.50 Weighted Median 99.00 98.50 80.00 98.00 99.00 98.50 Weighted Product 100.00 100.00 97.00 98.50 100.00 98.00 Weighted Tanh 100.00 99.50 96.00 98.50 100.00 99.00 Weighted Exponential 100.00 99.50 99.50 98.50 100.00 95.50 (c) From Table 6.6, it is observed for the self-built biometric data set that the 100 % recognition rate is achieved for many score normalization and fusion techniques. The test case-3, i.e., weight assignments of 0.5 for iris, 0.3 for fingerprint and 0.2 for face, results in better performance in an overall sense, for all fusion and normalization techniques while weighted-min fusion
138 technique outperforms other fusion techniques with 100 % recognition rate for almost all normalization techniques and for the three test cases, as evident from Table 6.6. 6.5 PERFORMANCE COMPARISON Liu et al 2009 used three biometric traits namely iris, fingerprint and face with forty subjects and obtained a recognition rate of 95.6% using score level fusion by employing the Gabor Wavelet and the Posterior Union Decision-Based Neural Network (PUDBNN) while our proposed work, achieves a recognition rate of 99.38% for forty subjects which is shown in Table 6.7. Genuine Acceptance Rate (GAR) obtained for the self-built multimodal database is 99.5%. Table 6.7 Performance Comparison with Liu et al (2009) (Against Score Level Fusion) Recognition Rate (in %) Modalities Liu et al (2009) method Proposed method Iris 87.60 93.63 Fingerprint 85.20 88.13 Face 83.50 96.88 Iris + Fingerprint + Face 95.60 99.38 Gan et al (2008) conducted experiments on ORL face database and CASIA iris database. They extracted the two features using Two-Dimensional Discrete Cosine Transform (2-D DCT) and used the Kernel Fisher Discriminant Analysis (KFDA) to produce the optimal discriminant feature at the feature level. They considered 7 frontal samples per subject with front 4 samples for each class as training samples, and the remainder 3
139 samples for each class as testing samples. Their obtained correct recognition rate based on appointed and random training and test samples were 96.67% and 94.17% respectively. Our experimental results done on the face and iris database produced the recognition rates of 97.5 % and 95.38 % at the feature level for the appointed and random training and test samples respectively. It is shown in Table 6.8. Table 6.8 Performance Comparison with Gan et al (2008) (Against Feature Level Fusion) RR of Gan (in %) method RR of proposed method (in %) 96.67 97.50 Al-Hijaili & AbdulAziz (2011) performed the fusion of two features obtained from the iris and face modalities at the matching score-level by assigning weights 0.8 and 0.2 for the iris and face respectively. Iris and face images were recognized using the method proposed by Masek (2003) and Turk & Pentland (1991) respectively. The recognition rates for iris and face images were 95.63% and 75 % for 40 subjects respectively. When the two scores were fused at the score-level, the obtained recognition rate after fusion was 98.75 % and the genuine acceptance rate (GAR) was 98 %. Our proposed method has provided a recognition rate of 100% at the score level, with a genuine acceptance rate (GAR) of 98%. The recognition rates (RR) of these works are tabulated in Table 6.9.
140 Table 6.9 Performance Comparison with Al-Hijaili & AbdulAziz (2011) (Against Score Level Fusion) RR of Al-Hijaili & AbdulAziz (2011) method (in %) RR of proposed method (in %) 98.75 100.00 The improvement in the performance is due to the extraction of statistical / co-occurrence features by deploying multi-resolution transforms such as Curvelet and Ridgelet. Also, the match-score level fusion technique with high weightage assigned to iris biometric modality leads to better recognition rate. 6.7 SUMMARY The proposed framework for multimodal biometric recognition system extracts the multi-resolution features from the Curvelet and Ridgelet transformed output images and the extracted features are matched at the matching score level. The performance of the multimodal biometric system is examined with different score normalization techniques and fusion methods by assigning different weights on both virtual and self-built multimodal database. The proposed weighted-min fusion technique performs well among all the normalization techniques. It is proved that the combination of iris, fingerprint and face biometric modalities for the proposed multimodal biometric recognition system has higher performance than each unimodal separately. The next chapter concludes this thesis work and addresses the scope for further enhancements in this work.