A COMPACT REPRESENTATION OF SENSOR FINGERPRINT FOR CAMERA IDENTIFICATION AND FINGERPRINT MATCHING. Ruizhe Li, Chang-Tsun Li, and Yu Guan

Transcription

1 A COMPACT REPRESENTATION OF SENSOR FINGERPRINT FOR CAMERA IDENTIFICATION AND FINGERPRINT MATCHING Ruizhe Li, Chang-Tsun Li, and Yu Guan Department of Computer Science, University of Warwick Coventry, CV4 7AL, UK ruizhe.li, c-t.li, ABSTRACT Sensor Pattern Noise (SPN) has been proved as an effective fingerprint of imaging devices to link pictures to the cameras that acquired them. SPN is usually extracted from large image block to improve the matching accuracy as the large image block contains more SPN information. However, fingerprint with high dimensionality will incur a costly computation in the matching phase, thus hindering many interesting applications which require an efficient real-time camera matching. To solve this problem, an effective feature extraction method based on PCA and LDA is proposed in this work to compress the camera fingerprint. Our experimental results show that the proposed feature extraction algorithm could greatly reduce the dimensionality of the original fingerprint and enhance the performance in term of Receiver Operating Characteristic (ROC) curve of several existing methods. Index Terms Digital forensics, Sensor pattern noise, Photo-response nonuniformity noise, PCA denoising. INTRODUCTION Sensor pattern noise, produced by imaging sensors, has been proved to be an effective way for digital camera identification. The photo-response nonuniformity (PRNU) noise is the deterministic component of SPN, which is primarily caused by the different sensitivity of individual sensor pixels to light. It is essentially the slight variations in the intensity of individual pixels and an unique pattern deposited in every image taken by a sensor. Lukas et al. [] first adopted a wavelet-based Wiener filter to extract the SPN from digital image. After that several works [ ] have been done to improve the performance of SPN-based source camera identification. Many of them have achieved nearly perfect identification rate when SPNs are extracted from the image blocks with high resolution. However, the complexity of computing correlation for fingerprint matching is proportional to the dimensionality of SPN. Therefore, the SPN extracted from large image block, which has a high dimensionality, will incur an expensive computational cost in the matching phase especially when there is a sizable database involved. For example, in the real practice investigators may face the applications of looking for a particular fingerprint in a large database which contains a huge number of fingerprint matching. To overcome this problem, an intuitive idea is to reduce the dimension of SPN. In [], Goljan et al. proposed a fingerprint digest as a possible solution. This fingerprint digest is primarily formed by only keeping a fixed number of the largest fingerprint values and their positions. In [3], Bayram et al. proposed to represent sensor fingerprint in binary-quantized form, which greatly reduce the size of fingerprint and speeds-up the correlation detection. In this paper, we employ the concept of PCA denoising [4] in the SPN-based source camera identification. An effective feature extraction algorithm based on this concept is first applied to extract a small number of principal components that will better represent the SPN signal. LDA is then utilized to take the advantage of the observed label information of training samples (reference fingerprints) to further reduce the dimension and improve the matching accuracy.. PROPOSED METHOD.. Feature extraction via PCA In digital camera identification, noise residual is usually extracted from large image block to improve the identification accuracy, since the large image block contains more SPN information. As a result, noise residual usually has a very high dimensionality (e.g. 4 4 pixels). Nevertheless, the high-dimensional feature is more likely to introduce redundancy and interfering components. For example, noise residual can be contaminated by color interpolation, JPEG compression, distortion introduced by denoising filter and other artifacts. Most of these artifacts are non-unique, redundant and less discriminant. Removing them will enhance the SPN signal in noise residual and improve the identification accuracy. However, they are mixed with the real SPN signal in noise residual and it is very hard to separate them. PCA [5] is a well-known decorrelation method which has been widely used for dimensionality reduction or redundancy removal. In this work, we attempt to find a PCA

2 transformed domain which can better separate the real SPN signal and these redundant features. By excluding these redundant features, we can extract a set of features which contain most of the discriminative information of SPN signal.... Training Sample Selection Assume there are n images {Ii }ni= taken by c cameras {Cj }cj= in a database, each camera has Lj images. We first extract noise residuals from the N N -pixels blocks cropped from the centre of these full-sized images and reshape them into a set of column vectors {xi RN }ni=. Several SPN extraction methods in the literature could be used here. As mentioned in [5], image content would seriously contaminate the real SPN, and their magnitude is far greater than that of the real SPN. Without removing these strong contaminations from the training set, PCA is more likely to find a subspace that better represent these noisy components rather than the SPN signal. Therefore, for training sample selection, we give the priority to the noise residual extracted from low-variation images (e.g., blue sky images). It is because such images are more close to the evenly lit scene and contain less scene details. Hence these images can better exhibit the changes caused by SPN in intensity between individual pixels. Such a training sample selection will capture the energy of true SPN in the training set and guide PCA to find a set of features that better represent the SPN signal rather than other noisy components.... PCA-based feature extractor n Let Ψ = n i= xi be the mean of the training set. Each xi differs from the mean Ψ by the vector Φi = xi Ψ. Then PCA is performed to seek a set of orthonormal vectors uk and their associated eigenvalues λk. The vectors uk and scalars λk are the eigenvectors and eigenvalues, respectively, of the covariance matrix S n S= Φi ΦTi = AAT () n i= n where A = [Φ, Φ,..., Φn ]. However as aforementioned, the dimensionality of SPN could extremely high (e.g., N =4 ). Thus directly solving the eigenvalue decomposition problem of S RN N would incurs a prohibitive computational cost (with a complexity O(N 6 )). To make PCA feasible for the high-dimensional SPN, we apply a fast method instead to compute these eigenvectors (when n N ). Assume uk are the unit eigenvectors of AT A Rn n with eigenvalue λ k. We could obtain AT Auk = λk uk. Multiplying both sides by A, we have AAT (Auk ) = λk (Auk ), where Auk are the eigenvectors of AAT = S with eigenvalues λk. Thus, instead of finding the eigenvectors uk of matrix S directly, we can calculate the eigenvectors uk of the smaller matrix AT A Rn n and obtain the objective eigenvector uk by uk = Auk. The obtained {uk }nk= are then normalized to the unit vectors and sorted in the descending order according to their associated Fig. : (a) An image taken by Canon Ixus7. (b) Clean reference SPN of Canon Ixus7. (c) The noise residual extracted from (a) via the Winer filter []. (d) The reconstructed version of (c) via the proposed method. (Note the intensity of Fig. (b), (c) and (d) has been scaled to the interval [,55] for visualization purpose.) eigenvalues λ λ...λn. Note that in practice the size of the training set tends to be much smaller than the dimensionality of SPN (n N ). Thus computing eigenvectors via this method (with a complexity O(n3 )) thus will be more effective than the traditional way. By excluding the eigenvectors with small eigenvalues while preserving the eigenvectors with large eigenvalues, we can obtain a small set of eigenvectors which contain most of the discriminative information of the SPN signal. In this work, the eigenvectors with the d largest eigenvalues are selected to construct the feature extractor Mpca = [u,..., ud ] RN d. We keep the top d eigenvectors corresponding to 99% of the variance as it could give us the best result. Based on this feature extractor M, we can obtain a new feature with much lower dimensionality by T x () y = Mpca where y Rd is the compact version of the original vector x. We call this obtained feature y as PCA feature in the rest of this paper. According to the PCA feature y and the feature extractor Mpca, we can easily obtain a reconstructed SPN in the spatial domain via the inverse PCA transformation x = Mpca y, where x is an approximation of x (when d < n). By preserving only the principal eigenvectors and conducting the inverse PCA transformation, we find that the impact of image content can be effectively suppressed in the reconstructed version. As shown in Fig. (c), the image content in Fig. (a) propagate through the Wiener filter into the noise residual, but these scene details have been significantly suppressed after the SPN reconstruction Fig. (d). The reason for this phenomenon is that, the objective of our PCA-based feature extractor is to extract a set of features that will better represent the primary signal in the training set, thus the aforementioned training sample selection will guide it to extract features that represent the SPN signal rather than other contaminated

3 signals. Moreover, since the magnitude of image content is not similar with that of true SPN at all [5], our PCAbased feature extractor can easily distinguish and separate them. To further prove this concept, we calculate the signalto-noise ratio (SNR) between the signal of interest (SPN) and the contaminants in the noise residual (Fig. (c)) and reconstructed SPN (Fig. (d)), respectively. Since we have the clean SPN ˆx (Fig. (b)), the contaminations Ξ in the noise residual or reconstructed SPN can be approximately estimated by subtracting the clean SPN from the observed var(ˆx) data. Thus SNR could be obtained as log var(ξ). The results show that the reconstructed SPN has a higher average SNR (6. db) than the noise residual (-9.8 db)... Feature Extraction via LDA Since the training set is labeled, LDA can use this information to further reduce the dimensionality. The purpose of LDA is to find a transformation matrix M lda which can best separate different classes. This optimal matrix can be obtained by maximizing the ratio of the determinant of the between-class scatter matrix S b to the determinant of the within-class scatter matrix S w. M lda = arg max J J T S b J J T S w J (3) The within-calss scatter matrix S w is defined as S w = c Lj j= i= (y i m j)(y i m j ) T where y i is the i th sample of class j, m j is the mean of class j, c is number of classes, L j is the number of samples in class j. The between-class scatter matrix S b is defined as S b = c j= L(m j m)(m j m) T where m is the global mean of all samples. Having the matrix M lda, a (c ) dimensional vector z can be obtained by z = M T lday = M T ldam T pcax = T x (4) where z can be seemed as another compact representation of the original noise residual x, we call this new feature as LDA feature in the rest of this paper. And T is the PCA+LDAbased feature extractor..3. Reference Estimation and Detection Statistics By performing the LDA-based feature extraction on the tracing set {x i } n i=, we can obtain a set of LDA features {z i } n i=. The reference feature z j for each camera C j then can be estimated by averaging all the L j features belong to that camera as z j = L j i= z i/l j, j =,,..., c. The Normalized Cross-Correlation (NCC) is adopted as the detection statistics to measure the similarity between the query z and the camera reference z corr (z, z ) = (z z) (z z ) z z z z (5) where z and z are the means of z and z, respectively. Notice that the complexity of computing correlation is proportional to the dimensionality of the features. Since the dimensionality of LDA feature z R (c ) is much lower than that of the original noise residual x R N (c N ), using the feature z as a compact representation of x in the matching phase could greatly reduce the computational complexity. 3. EXPERIMENTS In this work, the noise residuals extracted by the methods in [] (Lukas), [5] (Li s Model 3) and [9] (Kang) are used as the original features. To testify the feasibility of the proposed method, the performance of these original features combined with and without the proposed scheme are compared. Our experiments are conducted over the Dresden Image database [6]. A total of images from cameras are involved in the first experiment, each responsible for. These camera devices belong to 4 camera models, each camera model has 3 different devices. For each camera, we have 5 low-variation images for training and 5 images with scene details for testing. Thus there are 5 intraclass and 35 interclass correlation values in total. We extract all the SPNs from the luminance channel as the luminance channel contains information of all the three channels. Fig. shows the distribution with respect to randomly selected 5 intraclass and 5 interclass correlation values according to different features, where Original feature indicates the noise residual extracted by Kang s algorithm. It can be seen that using the LDA feature can better separate the intraclass and interclass samples than the original feature or PCA feature. To further testify the accuracy, all the 5 intraclass and 35 interclass samples are used to draw the overall ROC curves [9]. As shown in Fig. 3, the black, blue and red lines indicate the performance of the original features, PCA features and LDA features, respectively. It can be concluded from Table and Fig. 3 that our PCAbased feature extraction could enhance the performance of the original SPN extraction method, and PCA+LDA-based feature extraction could further improve it. Based on Table, we can obtain the detection threshold for each features when they have the low False Positive Rate (FPR) ( 3 ). Table : The TPR of different features at the FPR of 3 Methods Original SPN PCA PCA+LDA Lukas [] 83.65% 95.55% 99.% Li s Model 3 [5] 85.97% 96.3% 99.4% Kang [9] 93.% 98.87% 99.7% In practice, forensic investigators may face the problem that the testing images are not taken by the cameras in the database. Therefore, we use 5 6 testing samples taken by another 6 cameras from the Dresden database to further testify the performance of our proposed method. We calculate the NCC between these 5 6 testing samples and the aforementioned camera references. By comparing these 5 6 interclass samples with the threshold obtained from Table, we can get the FPR of different features in Table. From Table we can see that both PCA feature and LDA

4 .3.5 Intraclass Interclass (a) Original feature (b) PCA feature (c) LDA feature Fig. : Distribution of the correlation values obtained based on different features, 5 5 pixels Lukas Lukas+PCA Lukas+PCA+LDA Model 3 Model 3+PCA Model 3+PCA+LDA Kang Kang+PCA Kang+PCA+LDA 3 Fig. 3: The overall ROC comparisons among original features, PCA features and LDA features, 5 5 pixels. Table : The FPR of different features Methods Original SPN PCA PCA+LDA Lukas [].7%.3%.% Li s Model 3 [5] % %.4% Kang [9].% %.5% Original feature PCA feature LDA feature feature are able to make out that these testing samples are not taken by the aforementioned camera, even though their fingerprints are not involved in the training set. Fig. 4 presents the time cost for different features of matching a fingerprint against a simulate database with 3 cameras, where SPN extraction indicates the time cost of extracting SPN from a query image via Kang s method, Projection represents the time cost of feature extraction via PCA and LDA, Matching is the time cost of matching a query fingerprint against 3 cameras and the overall time is evaluated by Total. Notice that the time cost of training is not counted in this figure as it could be performed at offline. All the experiments are conducted on the same PC with Intel(R) Core(TM) i G RAM. It can be observed from Fig. 4 that although using PCA and LDA will incur an extra time cost in feature extraction, they dramatically reduce the time costs of matching process. Based on the overall time cost, we can conclude that the proposed method could greatly reduce the time cost of fingerprint matching with a large camera database. However, there is a potential limitation of LDA feature. Since the dimensionality of LDA feature is c, the computational cost of matching process based on LDA feature might be also expensive when the number of camera in the database c is huge. But for this case, the dimensionality of PCA feature will still stay at a constant low level as it only depends on the dominance of the 4 SPN extraction Projection Matching Total Fig. 4: Time cost of different features, 5 5 pixels. primary signal in the training set. 4. CONCLUSION In this work, we evaluate the concept of PCA denoising in source camera identification. A feature extraction algorithm based on this concept is proposed to extract a feature set with much lower dimensionality from the original noise residual. LDA is then utilized to take the advantage of the observed label information of the training data to further reduce the dimensionality and find a space could better separate different classes. By applying the proposed feature extractor on the noise residual, the SPN components could be well preserved while the dimensionality being effectively reduced. The experimental results show that the proposed feature extraction method could greatly improve the matching efficiency and further enhance the performance of several existing methods. However, this method is hard to address the problem when new cameras are sequentially added into the database. It is because that repeating an entire training that includes these new cameras would incur a costly re-computation. To solve this issue will be the focus of our future research.

5 5. REFERENCES [] J. Lukas, J. Fridrich, and M. Goljan, Digital camera identification from sensor pattern noise, IEEE Trans. on Information Forensics and Security, vol., no., pp. 5 4, 6. [] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3d transform-domain collaborative filtering, IEEE Trans. on Image Processing, vol. 6, no. 8, pp. 8 95, 7. [3] M. Chen, J. Fridrich, M. Goljan, and J. Lukas, Determining image origin and integrity using sensor noise, IEEE Trans. on Information Forensics and Security, vol. 3, no., pp. 74 9, 8. [4] M. Goljan, Digital camera identification from images - estimating false acceptance probability, in Proc. of International Workshop on Digital-forensics and Watermarking, pp [5] C.-T. Li, Source camera identification using enhanced sensor pattern noise, IEEE Trans. on Information Forensics and Security, vol. 5, no., pp. 8 87,. [6] A. Cortiana, V. Conotter, G. Boato, and F. G. B. De Natale, Performance comparison of denoising filters for source camera identification, in Proc. SPIE, Media Watermarking, Security, and Forensics III, San Francisco, CA, USA, Jan. 3-6,, vol. 788, pp. 7. Information Processing, Xi an, China, Jul. 9-3, 4, pp [] M. Goljan, J. Fridrich, and T. Filler, Managing a large database of camera fingerprints, in Proc. SPIE, Electronic Imaging, Media Forensics and Security XII, San Jose, CA, USA, Jan. 7-,, vol. 754, pp. 8. [3] S. Bayram, H. Sencar, and N. Memon, Efficient sensor fingerprint matching through fingerprint binarization, IEEE Trans. on Information Forensics and Security, vol. 7, no. 4, pp ,. [4] D. Zhang, R. Lukac, X. Wu, and D. Zhang, PCAbased spatially adaptive denoising of CFA images for single-sensor digital cameras, IEEE Trans. on Image Processing, vol. 8, no. 4, pp , 9. [5] K. Fukunaga, Introduction to Statistical Pattern Recognition, nd ed. New York: Academic, 99. [6] T. Gloe and R. Böhme, The dresden image database for benchmarking digital image forensics, Journal of Digital Forensic Practice, vol. 3, no. -4, pp. 5 59,. [7] C.-T. Li and Y. Li, Color-decoupled photo response non-uniformity for digital image forensics, IEEE Trans. on Circuits and Systems for Video Technology, vol., no., pp. 6 7,. [8] X. Kang, Y. Li, Z. Qu, and J. Huang, Enhancing source camera identification performance with a camera reference phase sensor pattern noise, IEEE Trans. on Information Forensics and Security, vol. 7, no., pp ,. [9] X. Kang, J. Chen, K. Lin, and A. Peng, A contextadaptive spn predictor for trustworthy source camera identification, EURASIP Journal on Image and Video Processing, vol. 4, no., pp., 4. [] R. Li, C.-T. Li, and Y. Guan, A reference estimator based on composite sensor pattern noise for source device identification, in Proc. SPIE, Electronic Imaging, Media Watermarking, Security, and Forensics 4, San Francisco, CA, USA, Feb. -6, 4, vol. 98. [] R. Li, Y. Guan, and C.-T. Li, PCA-based denoising of sensor pattern noise for source camera identification, in IEEE China Summit & Int. Conf. on Signal and