Image as a Feature Vector. Eigenfaces: linear projection. Announcements

Transcription

1 Announcements Face Recognton: Dmensonalty Reducton & Unconstraned Face Rec Bometrcs CSE 190 Lecture 14 H3 assgned, due on hursday. Last H fngerprnt recognton to be assgned next week. hursday guest lecture Forensc DNA echnology Dr. Davd Kng, IntegenX, Executve Vce Presdent, Product Development Attendance s mandatory CSE190, Sprng 2014 CSE190, Sprng 2014 H3 Hand Recognton Challenge Hand outlne acquston Data: Your hand outlnes Goal: Buld two dfferent classfers Features: Manual Evaluaton: 1. Four-fold cross valdaton on tranng set 2. Unlabeled test set nner 10 pts extra credt on H.! he Hand Recognton Challenge You are the data You wll be gven 5 peces of paper. You wll be assgned an ID number On the upper rght corner, wrte <Your ID> <Your ID> - 5 Flp over pages, so ID Is on back race your left hand on four peces of paper. Now have your neghbor trace your left hand on the ffth peces of paper.! CSE190, Sprng 2014 CSE190, Sprng 2014 Image as a Feature Vector Egenfaces: lnear projecton x 2 An n-pxel mage x R d can be projected to a low-dmensonal feature space y R k by Example: Projectng from R 3 to R 2 x 1 x 3 y = x where s an n by m matrx. R k Consder an n-pxel mage to be a pont n an n-dmensonal space, x R n. Each pxel value s a coordnate of x. Recognton s performed usng nearest neghbor n R k. How do we choose a good? 1

2 Egenfaces: Prncpal Component Analyss () How do you construct Egenspace? [ ] [ ] [ x 1 x 2 x 3 x 4 x 5 ] Some detals: Use Sngular value decomposton, trck descrbed n text to compute bass when n<<d Construct data matrx by stackng vectorzed mages and then apply Sngular Value Decomposton (SVD) Matrx Decompostons Defnton: he factorzaton of a matrx M nto two or more matrces M 1, M 2,, M n, such that M = M 1 M 2 M n. Many decompostons exst QR Decomposton LU Decomposton LDU Decomposton Etc. Sngular Value Decomposton Any m by n matrx A may be factored such that A = UΣV [m x n] = [m x m][m x n][n x n] U: m by m, orthogonal matrx Columns of U are the egenvectors of AA V: n by n, orthogonal matrx, columns are the egenvectors of A A Σ: m by n, dagonal wth non-negatve entres (σ 1, σ 2,, σ s ) wth s=mn(m,n) are called the called the sngular values. SVD algorthm produces sorted sngular values: σ 1 σ 2 σ s Important property Sngular values are the square roots of Egenvalues of both AA and A A & Columns of U are correspondng Egenvectors!! SVD Propertes In Matlab [u s v] = svd(a), and you can verfy that: A=u*s*v r=rank(a) = # of non-zero sngular values. U, V gve an orthonormal bases for the subspaces of A: 1st r columns of U: Column space of A Last m - r columns of U: Left nullspace of A 1st r columns of V: Row space of A 1st n - r columns of V: Nullspace of A For some d where d r, the frst d column of U provde the best d-dmensonal bass for columns of A n least squares sense. Performng wth SVD Sngular values of A are the square roots of egenvalues of both AA and A A & Columns of U are correspondng Egenvectors And n a a = a 1 a 2! a n =1 Covarance matrx s: [ ][ a 1 a 2! a n ] = AA n Σ = 1 ( x! n! µ )( x!! µ ) =1 So gnorng 1/n, subtract mean mage µ from each nput mage, create a d x n data matrx, and perform thn SVD on the data matrx. D=[x 1 -µ x 2 -µ x n -µ ] 2

3 hn SVD Any m by n matrx A may be factored such that A = UΣV [m x n] = [m x m][m x n][n x n] If m>n, then one can vew Σ as: (.e., more pxels than mages) Frst Prncpal Component Drecton of Maxmum Varance ' 0 here Σ =dag(σ1, σ2,, σs) wth s=mn(m,n), and lower matrx s (n-m by m) of zeros. Mean hs s what you should use!! Alternatvely, you can wrte: A = U Σ V In Matlab, thn SVD s:[u S V] = svd(a, econ ) Egenfaces Modelng Egenfaces: ranng Images [urk, Pentland 91] 1. Gven a collecton of n labeled tranng mages, 2. Compute mean mage and covarance matrx. 3. Compute k Egenvectors (note that these are mages) of covarance matrx correspondng to k largest Egenvalues. 4. Project the tranng mages to the k-dmensonal Egenspace. Recognton 1. Gven a test mage, project to Egenspace. 2. Perform classfcaton to the projected tranng mages. [ urk, Pentland 91] Egenfaces Mean Image Varable Lghtng Bass Images 3

4 Reconstructon usng Egenfaces Gven mage on left, project to Egenspace, then reconstruct an mage (rght). Underlyng assumptons Background s not cluttered (or else only lookng at nteror of object Lghtng n test mage s smlar to that n tranng mage. No occluson Sze of tranng mage (wndow) same as wndow n test mage. Dffcultes wth Projecton may suppress mportant detal smallest varance drectons may not be unmportant Does not generalze well to unseen condtons Method does not take dscrmnatve task nto account typcally, we wsh to compute features that allow good dscrmnaton not the same as largest varance Illumnaton Varablty Fsherfaces: Class Specfc Lnear Projecton P. Belhumeur, J. Hespanha, D. Kregman, Egenfaces vs. Fsherfaces: Recognton Usng Class Specfc Lnear Projecton, PAMI, July 1997, pp An n-pxel mage x R n can be projected to a low-dmensonal feature space y R m by y = x where s an n by m matrx. he varatons between the mages of the same face due to llumnaton and vewng drecton are almost always larger than mage varatons due to change n face dentty. -- Moses, Adn, Ullman, ECCV 94 Recognton s performed usng nearest neghbor n R m. How do we choose a good? 4

5 Covarance of projected samples & Fsher s Lnear Dscrmnant Let Σ x be a covarance matrx be an n by m matrx defnng a lnear projecton y=x χ χ 2 1 (Egenfaces) = arg max S Maxmzes projected total scatter he mean of y s gven by µ y = µ x hen the covarance of y s gven by: Σ y = Σ x FLD Fsher s Lnear Dscrmnant SB fld = arg max S Maxmzes rato of projected between-class to projected wthn-class scatter & Fsher s Lnear Dscrmnant & Fsher s Lnear Dscrmnant Between-class scatter thn-class scatter c S = ( xk µ )( µ k otal scatter c µ S = 1 (x k µ)(x k µ) = S B + S here c S B = χ ( µ µ )( µ µ ) =1 = 1 = 1 xk χ x k χ c s the number of classes µ s the mean of class χ χ s number of samples of χ.. µ ) χ 1 χ 2 µ µ 2 χ χ 2 1 FLD (Egenfaces) = arg max S Maxmzes projected total scatter Fsher s Lnear Dscrmnant SB fld = arg max S Maxmzes rato of projected between-class to projected wthn-class scatter Computng the Fsher Projecton Matrx he w s are orthonormal here are at most c-1 non-zero generalzed Egenvalues, so m c-1 Can be computed wth eg n Matlab fld = arg max Fsherfaces = fld Snce S s rank N-c, project tranng set to subspace = arg max S spanned by frst N-c prncpal components of the tranng set. Apply FLD to N-c S B dmensonal subspace yeldng c-1 dmensonal feature space. S Fsher s Lnear Dscrmnant projects away the wthn-class varaton (lghtng, expressons) found n tranng set. Fsher s Lnear Dscrmnant preserves the separablty of the classes. 5

6 vs. FLD Expermental Results - 1 Varaton n Facal Expresson, Eyewear, and Lghtng Input: 160 mages of 16 people ran: 159 mages est: 1 mage th glasses thout glasses 3 Lghtng condtons 5 expressons FLD For Glasses/No Glasses Recognton Effect of Croppng Expermental Results - 2 Harvard Face Database 15 o 30 o 45 o 10 ndvduals 66 mages per person ran on 6 mages at 15 o est on remanng mages 60 o 6

7 Recognton Results: Lghtng Extrapolaton Error Rate degrees 30 degrees 45 degrees Lght Drecton Correlaton Egenfaces Egenfaces (w/o 1st 3) Fsherface FACE RECOGNIION: SAE OF HE AR? From Executve Summary Usng the most accurate face recognton algorthm, the chance of dentfyng the unknown subject (at rank 1) n a database of 1.6 mllon crmnal records s about 92%. Facal recognton algorthms are more accurate on the vsa mages than the mug shot mages. he vsa mages are collected wth careful cooperaton of the subject, actve complance by the photographer to the mage collecton specfcaton, and a yes/no revew by an offcal. UNCONSRAINED FACE RECOGNIION Pose Illumnaton Expresson Agng Dstress Glasses & sunglasses Facal har Headgear Occluson Blur Resoluton Low qualty A PROGERSSION OF BENCHMARKS FERE - - ell controlled mages Yale Face Database B - - Systema<c vara<on of pose & lgh<ng Mul<- PIE - - Systema<c vara<on n pose, lgh<ng, expresson, mul<ple sessons NIS Mul<ple Bometrc Evalua<on - - Large scale, mostly well- controlled Labeled Faces n the ld (LF) - - oward unconstraned 7

8 LABELED FACES IN HE ILD: DE FACO SANDARD FOR EVALUAION 13,233 mages of 5,749 ndvduals Acqured from Yahoo News n 2002 Results reported for 38 commercal and academc methods oward unconstraned face recogn<on AVrbute and Smle Classfiers for Face Verfica<on ICCV 2009 Neeraj Kumar Alexander C. Berg Peter N. Belhumeur Shree K. Nayar 3000 same pars Columba Unversty 3000 dfferent pars G. Huang, M. Ramesh,. Berg, E. Learned- Mller, 2007 AVrbutes can define categores Female Eyeglasses Mddle-aged Dark har AVrbutes can define categores Caucasan eeth showng Outsde lted head Are these mages of the same person? hvp://mughunt.securcs.com/ 8

9 Low-level features HOG HOG LBP LBP SIF SIF Dfferent HOG - HOG LBP SIF SIF Verfcaton + LBP PubFg dataset & benchmark Attrbutes Round Jaw Images Dark har Verfcaton Male Images Our approach: attrbutes Asan Pror approaches Low-level features Dfferent + - Amazon Mechancal urk Publc fgures: Poltcans Celebrtes Larger & deeper: 60,000 Images 200 People 300 Images per person Subsets: Pose Illumnaton Expresson 500,000 Attrbute Labels = $5, month See also [Deng, et al., 2009] [Vjayanarasmhan & Grauman, 2009] Learnng an attrbute classfer Usng attrbutes to perform verfcaton ranng mages Low-level features HoG HSV Feature selecton, Nose HoG, Eyes HSV, Har Males HoG HSV ran classfer Edges, Mouth Gender classfer Male Verfcaton classfer 0.87 Females 9

10 Prevous state-of-the-art on LF Atrbutes for Recognton on LF 85.29% Accuracy (31.68% Drop n error rates) as of May 2009 as of May 2009 HUMAN FACE VERIFICAION PERFORMANNCE AND CONEX LF Results May 2014 Orgnal 99.20% [Kumar, Ber Belhumeur, IC Cropped 97.53% Inverse Cropped 94.27% 10