Person Re-identification Algorithms: A Survey

Transcription

1 THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS TECHNICAL REPORT OF IEICE joyport@ari.ncl.omron.co.jp, {kawanishi,minoh}@mm.media.kyoto-u.ac.jp, murase@is.nagoya-u.ac.jp Person Re-identification Algorithms: A Survey Yoshihisa IJIIR, Yasutomo KAWANISHI, Michihiko MINOH, and Hiroshi MURASE Corporate R&D, OMRON Corporation 9-1 Kizugawadai, Kizugawa, Kyoto, Japan Academic Center for Computing and Media Studies, Kyoto University Yoshida Nihonmatsucho, Sakyoku, Kyoto, Japan Graduate School of Information Science, Nagoya University Furocho, Chigusa, Nagoya, Japan joyport@ari.ncl.omron.co.jp, {kawanishi,minoh}@mm.media.kyoto-u.ac.jp, murase@is.nagoya-u.ac.jp Abstract In human surveillance, tracking of people is one of essential tasks. Since a field of view in a camera is limited, human tracking across cameras with non-overlapping field of views, i.e., human re-identification have been studied. In this paper, we survey existing work in the field of human re-identification, and summarize evaluation dataset and methodologies as well. Key words survey, human re-identification, human tracking across cameras 1. (Human re-identification or Human tracking across cameras) : 1 1 1

2 (a) (b) (c) (d) (e) (f) (g) (h) 1 Examples of VIPeR DB: (a)(b)are from ID19. Similarly, (c)(d)from 302, (e)(f)from 188, and (g)(h)from [1] [2] [3] (i) (ii) (iii) (iv) (i)-(iv) Nakajima [4] RGB RGB RG one vs. all SVM [5] [6] RGB Bird [7] HSL 10 Lin [8] RGB RGSS=(R+G+B)/3 Color Rank Cheng [9] Major Color (Major Color Spectrum Histogram) RGB Cai [10] HSV (hue) opponent color opponent color angle( ) ( ) Hamdoun [12] Camellia SURF Bak [13] Haar-like Adaboost Wang [14] Log RGB Histogram of Oriented Gradients: HOG Lab HOG shape context Shape and Appearance context 2

3 Alahi [15] (Alahi ) x, y / x/y / Bak [16] RGB [17] RGB RGB Gheissari [11] HSV HS watershed Hessian affine decomposable triangulated graph Berdugo [18] Lin [8] ranked color ratioranked oriented gradientsranked saliency map ranked color ratio Farenzena [19] HSV Maximally Stable Color Region (MSCR) Recurrent High-Structured Patches (RHSP) () () Symmetric Drive Accumulation of Local Features (SDALF) Bazzani [20] HSV (epitome) Gray [21] RGBYCbCrHSV Schmid Gabor Adaboost (Ensemble of Localized Features: ELF) HSV H() Schwartz [22] Partial Least Square: PLS HOG RGB Color Rank Kuo [23] HOG ( ) positive bag negative bag Multiple Instance Learning (MIL) [24] ST-HOG Zheng [25] RGB SIFT(Scale-Invariant Feature Transform) RGB L1 Group Cai [26] opponent color [27] () 3

4 .1 (S) (V) (Y) (N) 3. () Bhattacharyya 1 (Person-specific model) (Global/Generic discriminative model) Nakajima [4] SVM Schwartz [22] PLSKuo [23] MIL Bak [13] Adaboost Hirzer [28] positive negative Haar-like RGB adaboost one vs. all Bird [7] (FLD) FLD [6] Gray [21] Adaboost () vs. () Adaboost Prosser [29] RankSVM Ranking Ranking RankBoost RankSVM RankBoost RankSVM PRSVM [30] Kernel Large Margine Component Analysis Jensen-Shannon Zheng [31] RankSVM 4. 4

5 1 Variations of features, distance metric, classifiers described in articles / DB Nakajima [4] S 3 Y RGB RG SVM Own DB Bird [7] S Y HSL 10 HSL Euclidean Own DB Gheissari [11] S N HSV 5 HS Hist. Intersec. Own DB Cheng [9] S/V N RGB Major Color Own DB Wang [14] S N Log-RGB, Lab HOG, shape context L1 Gheissari DB Lin [8] S Y 2 RGB(RGS) Color Rank KL divergence Honeywell DB Gray [21] S Y RGB, YCbCr, HSV Schmid/Gabor Adaboost VIPeR Hamdoun [12] V N RGB (Camellia lib )SURF SADKD-tree CAVIAR Schwartz [22] S Y RGB, HOG, ColorRankHist. PLS ETHZ Alahi [15] S N (,, ) VIPeR Bak [16] S N RGB ilids V N RGB(RGS) ColorRank, RankedColorRatio, KL-divergence RankedOrientedGradients, RankedSaliencyMap Berdugo [18] Own DB Kuo [23] V Y RGB? HOG MIL Own DB Farenzena [19] S/V N HSV MSCRRHSP Bhat. VIPeR, ilids, ETHZ Bazzani [20] V N HSV Epitome Bhat. ETHZ, ilids Bak [13] S 3 Y RGB? Haar-like Adaboost CAVIAR, TRECVID Cai [10] S Y 4 HSV 5 Hue/OpponentAngle χ 2 ETHZ Zheng [25] S Y 4 RGB SIFT RGB L1 ilids Cai [26] S N opponent color CASIA, OULU DB Hirzer [28] S/V Y Lab, RGB Haar-like Aadboost VIPeR, Own DB *1 *2 KL *3 *4 *5 HSV 1 Cheng [9] Madden [32] Incremental Major Color Spectrum Histogram Representation(IMCSHR) (Brightness Transfer Function: BTF) BTF BTF Porikli [33] BTF Javed [34] [35] BTF 2 B i B j f ij O i B m H i (B m ) = m k=1 I i(b k ) O i O j 5

6 O i B i O j B j H i (B i ) = H j (B j ) = H j (f ij (B i )) BTF f ij (B i ) = H 1 j (H i (B i )) H 1 i 256 BTF 256 BTF BTF BTF BTF BTF Prosser [36] (Cumulative BTF: CBTF) BTF BTF N I 1, I 2,..., I N Ĥi(B m ) = 1 N m k=1 N L=1 I L(B k ) CBTF cf ij (B i ) = Ĥ 1 j (Ĥi(B i )) BTF Prosser [37] CBTF BTF BTF Gilbert [38] 5. (spatio-temporal transition probability) Kettnaker [39] Javed [40] [35] Parzen Porikli [41] Dick [42] Ellis [43]Makris [44] Tieu [45] [46] [47] [48] [49] [50] Chen [51] 6

7 CAVIAR [52], VIPeR [53], ilids [54], TRECVID [55], ETHZ [56] VIPeRCAVIARETHZ ETHZ / DB VIPeR VIPeR 20% DB ilids TRECVID (UK Home Office) DB HDD 6. 2 (Cumulative Match Characteristics: CMC) [57]CMC n CMC(n) ID(g i ) = ID(p j ), ID(g l ) = ID(p j ), s ij = f(g i, p j ), CMC(n) = pj; g l; s lj > s ij < = n g i ; ID(g i ) = ID(p j ). ID( ) f( ) s ij g i p j s lj CMC p j s ij g l ( ) n p j CMC.1 s lj > s ij s lj < s ij (1) CMC(1) CMC(10) 7. NEDO [1],, (PRMU), pp (2010). [2] Y. Ijiri, S. Lao, T. X. Han and H. Murase: Efficient facial attribute recognition with a spatial codebook, ICPR, pp (2010). [3] H. Murase and R. Sakai: Moving object recognition in eigenspace representation: gait analysis and lip reading, Pattern Recognition Lettetters, 17, pp (1996). [4] C. Nakajima, M. Pontil, B. Heisele and T. Poggio: Fullbody person recognition system, Pattern Recognition, 36, 9, pp (2003). [5],,,,, 2006, 2, p. 198 ( ). [6],,,,,, p. 225 ( ). [7] N. D. Bird, O. Masoud, N. P. Papanikolopoulos and A. Isaacs: Detection of Loitering Individuals in Public Transportation Areas, IEEE Trans. on ITS, 6, 2, pp (2005). [8] Z. Lin and L. S. Davis: Learning Pairwise Dissimilarity Profiles for Appearance Recognition in Visual Surveillance, Proc. of ISVC, pp (2008). [9] E. D. Cheng and M. Piccardi: Matching of Objects Moving Across Disjoint Cameras, Proc. of ICIP, pp (2006). [10] Y. Cai and M. Pietik: Person Re-identification Based on Global Color Context, The Tenth International Workshop on Visual Surveillance (2010). [11] N. Gheissari, T. B. Sebastian and R. Hartley: Person Reidentification Using Spatiotemporal Appearance, Proc. of CVPR, pp (2006). [12] O. Hamdoun, F. Moutarde, B. Stanciulescu and B. Steux: Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences, Proc. of ICDSC, pp. 1 6 (2008). [13] S. Bak, E. Corvee, F. Brémond and M. Thonnat: Person Re-identification Using Haar-based and DCD-based Signature, AMCSS 2010, pp. 1 8 (2010). [14] X. Wang, G. Doretto, T. Sebastian, J. Rittscher and P. Tu: Shape and Appearance Context Modeling, Proc. of ICCV, 7

8 pp. 1 8 (2007). [15] A. Alahi, P. Vandergheynst, M. Bierlaire and M. Kunt: Cascade of descriptors to detect and track objects across any network of cameras, CVIU, 114, 6, pp (2010). [16] S. Bak, E. Corvee, F. Brémond and M. Thonnat: Person Re-identification Using Spatial Covariance Regions of Human Body Parts, Proc. of AVSS, pp (2010). [17],,,,, (PRMU), 103, 659, pp ( ). [18] G. Berdugo, O. Soceanu, Y. Moshe, D. Rudoy and I. Dvir: Object Reidentification in Real World Scenarios Across Multiple Non-overlapping Cameras, Proc. of Euro. Sig. Proc. Conf., pp (2010). [19] M. Farenzena, L. Bazzani, A. Perina, V. Murino and M. Cristani: Person Re-Identification by Symmetry- Driven Accumulation of Local Features, Proc. of CVPR, pp (2010). [20] L. Bazzani, M. Cristani, A. Perina, M. Farezena and V. Murino: Multiple-shot Person Re-identification by HPE signature, Proc. of ICPR, pp (2010). [21] D. Gray and H. Tao: Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features, Proc. of ECCV, pp (2008). [22] W. R. Schwartz and L. S. Davis: Learning Discriminative Appearance-Based Models Using Partial Least Squares, Proc. of Brazil. Symp. on Comp. Graph. and Image Proc., pp (2009). [23] C.-H. Kuo, C. Huang and R. Nevatia: Inter-camera Association of Multi-target Tracks by On-Line Learned Appearance Affinity Models, Proc. of ECCV, pp (2010). [24],, Sthog, (CVIM), 2011, 10, pp. 1 8 ( ). [25] W.-S. Zheng, S. Gong and T. Xiang: Associating Groups of People, Proc. of BMVC (2009). [26] Y. Cai, V. Takala and M. Pietikainen: Matching groups of people by covariance descriptor, ICPR, pp (2010). [27],,, PRMU , pp (2011). [28] M. Hirzer, C. Beleznai, P. M. Roth and H. Bischof: Person Re-identification by Descriptive and Discriminative Classification, Scandinavian Conference on Image Analysis, pp (2011). [29] B. Prosser, S. Zheng, S. Gond and T. Xiang: Person Re- Identification by Support Vector Ranking, Proc. of BMVC, pp (2010). [30], S. Lao,, (PRMU), pp (2011). [31] W.-S. Zheng, S. Gong and T. Xiang: Person Reidentification by Probabilistic Relative Distance Comparison, Proc. of CVPR (2011). [32] C. Madden, E. D. Cheng and M. Piccardi: Tracking people across disjoint camera views by an illumination-tolerant appearance representation, Machine Vision and Applications, 18, 3-4, pp (2007). [33] F. Porikli: Inter-camera color calibration by correlation model function, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429), IEEE, pp. II (2003). [34] O. Javed, K. Shafique and M. Shah: Appearance Modeling for Tracking in Multiple Non-Overlapping Cameras, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 05), Ieee, pp (2005). [35] O. Javed, K. Shafique, Z. Rasheed and M. Shah: Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views, CVIU, 109, 2, pp (2008). [36] B. Prosser, S. Gong and T. Xiang: Multi-camera Matching using Bi-Directional Cumulative Brightness Transfer Functions, BMVC (2008). [37] B. Prosser, S. Gong and T. Xiang: Multi-camera Matching under Illumination Change Over Time, Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications, pp (2008). [38] A. Gilbert and R. Bowden: Tracking Objects Across Cameras by Incrementally Learning Inter-camera Colour Calibration and Patterns of Activity, 9th European Conference on Computer Vision ECCV, Vol. 3952, Springer, pp (2006). [39] V. Kettnaker and R. Zabih: Bayesian multi-camera surveillance, Proc. of CVPR, pp (1999). [40] O. Javed, Z. Rasheed, K. Shafique and M. Shah: Tracking across multiple cameras with disjoint views, Proc. of ICCV, Vol. 2, Ieee, pp (2003). [41] F. Porikli and A. Divakaran: Multi-camera calibration, object tracking and query generation, International Conference on Multimedia and Expo. ICME 03. Proceedings (Cat. No.03TH8698), pp. I 653 (2003). [42] A. R. Dick and M. J. Brooks: A Stochastic Approach to Tracking Objects Across Multiple Cameras, Australian Conf. on Art. Intel., pp (2004). [43] T. J. Ellis, D. Makris and J. K. Black: Learning a multicamera topology, Proc. of VSPETS, pp (2003). [44] D. Makris, T. Ellis and J. Black: Bridging the gaps between cameras, Proc. of CVPR, pp (2004). [45] K. Tieu, G. Dalley and W. L. Grimson: Inference of Non- Overlapping Camera Network Topology by Measuring Statistical Dependence, Proc. of ICCV, Ieee, pp (2005). [46], D, J89-D, 7, pp (2006). [47] a. Gilbert and R. Bowden: Incremental, scalable tracking of objects inter camera, Computer Vision and Image Understanding, 111, 1, pp (2008). [48] K. Shafique, A. Hakeem, O. Javed and N. Haering: Self Calibrating Visual Sensor Networks pro fashion. at Tentry sstem 3 Recovering Topology of Overlap-, IEEE Workshop Applications of Computer Vision, pp. 1 6 (2008). [49] X. Wang, K. Tieu and W. E. L. Grimson: Correspondencefree activity analysis and scene modeling in multiple camera views, IEEE trans. on PAMI, 32, 1, pp (2010). [50],,,, (2007). [51] K.-W. Chen, C.-C. Lai, P.-J. Lee, C.-S. Chen and Y.- P. Hung: Adaptive Learning for Target Tracking and True Linking Discovering Across Multiple Non-Overlapping Cameras, IEEE Trans. on Multimedia, 13, 4, pp (2011). [52] CAVIAR DB ( CAVIARDATA1/). [53] VIPeR DB ( v1.0.zip). [54] ilids DB ( /scienceandresearch.homeoffice.gov.uk/ hosdb/cctv-imaging-technology/i-lids/index.html). [55] ilids DB ( trecvid/2010/index.html#eval_data). [56] ETHZ DB ( 2007/). [57] D. Gray, S. Brennan and H. Tao: Evaluating Appearance Models for Recognition, Reacquisition, and Tracking, Proc. of PETS, Vol. 3, pp (2007). 8