Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)

Transcription

1 Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control Bometrc Authentcaton

2 Applcaton: Vdeo Survellance Applcaton: Autotaggng Photos n Facebook, Flckr, Pcasa, Photo, Face Scan at Arports Photo Can be traned to recognze pets!

3 Photo Thngs Photo thnks are faces Recognton should be Invarant to Lghtng varaton Head pose varaton Dfferent expressons Beards, dsguses Glasses, occluson Agng, weght gan Why s Face Recognton Hard? The many faces of Madonna Intra-class Varablty Faces wth ntra-subject varatons n pose, llumnaton, expresson, accessores, color, occlusons, and brghtness

4 Inter-class Smlarty Dfferent people may have very smlar appearance Face Detecton n Humans There are processes for face detecton and recognton n the bran Twns news.bbc.co.uk/h/englsh/n_depth/amercas/2000/ us_electons Father and son Blurred Faces are Recognzable Blurred Faces are Recognzable Mchael Jordan, Woody Allen, Golde Hawn, Bll Clnton, Tom Hanks, Saddam Hussen, Elvs Presley, Jay Leno, Dustn Hoffman, Prnce Charles, Cher, and Rchard Nxon. The average recognton rate at ths resoluton s one-half.

5 Upsde-Down Faces are Recognzable Context s Important The Margaret Thatcher Illuson, by Peter Thompson P. Snha and T. Poggo, I thnk I know that face, Nature 384, 1996, 404. Face Recognton Archtecture Image as a Feature Vector x 2 Image wndow Feature Extracton Feature Vector Classfcaton Face Identty x 1 x 3 Consder an n-pxel mage to be a pont n an n- dmensonal mage space, x R n Each pxel value s a coordnate of x Preprocess mages so faces are cropped and roughly algned (poston, orentaton, and scale)

6 Nearest Neghbor Classfer { R j } s a set of tranng mages of frontal faces ID( I) = arg mn dst( R, I) j j I x 2 R 1 x 1 x 3 R 2 Key Idea Expensve to compute nearest neghbor when each mage s bg (n dmensonal space) Not all mages are very lkely especally when we know that every mage contans a face. That s, mages of faces are hghly correlated, so compress them nto a low-dmensonal, lnear subspace that retans the key appearance characterstcs Egenfaces (Turk and Pentland, 1991) The set of face mages s clustered n a subspace of the set of all mages Fnd best subspace to reduce the dmensonalty Transform all tranng mages nto the subspace Use nearest-neghbor classfer to label a test mage Lnear Subspaces convert x nto v 1, v 2 coordnates: What does the v 2 coordnate measure? Dstance to lne defned by v 1 What does the v 1 coordnate measure? Dstance from x x on lne defned by v 1

7 Dmensonalty Reducton Lnear Subspaces Suppose we have ponts n 2D and we take a lne through that space We can represent the orange ponts well wth only ther v 1 coordnates (snce v 2 coordnates are all 0) Ths makes t much cheaper to store and compare ponts A bgger deal for hgher dmensonal problems We can project each pont onto that 1D lne Represent pont by ts poston along the lne Lnear Subspaces Some lnes wll represent the data well and others not, dependng on how well the projecton separates the data ponts Prncpal Component Analyss (PCA) Problems arse when performng recognton n a hghdmensonal space ( curse of dmensonalty ) Sgnfcant mprovements can be acheved by frst mappng the data nto a lower-dmensonal subspace The goal of PCA s to reduce the dmensonalty of the data whle retanng the mportant varatons present n the orgnal data

8 Prncpal Component Analyss (PCA) Dmensonalty reducton mples nformaton loss How to determne the best lower dmensonal subspace? Maxmze nformaton content n the compressed data by fndng a set of k orthogonal vectors that account for as much of the data s varance as possble Best dmenson = drecton n n-d wth max varance 2 nd best dmenson = drecton orthogonal to frst wth max varance Prncpal Component Analyss (PCA) Geometrc nterpretaton PCA projects the data along the drectons where the data vares the most These drectons are determned by the egenvectors of the covarance matrx correspondng to the largest egenvalues The egenvectors defne a new coordnate system n whch to represent each tranng mage Prncpal Component Analyss (PCA) The best low-dmensonal space can be determned by the best egenvectors of the covarance matrx of the data,.e., the egenvectors correspondng to the largest egenvalues also called prncpal components Can be effcently computed usng Sngular Value Decomposton (SVD) Algorthm Each nput mage, X, s an nd column vector of all pxel values (n raster order) Compute average face mage from all M tranng mages of all people: M 1 A = X M = 1 Normalze each tranng mage, X, by subtractng the average face: Y = X A

9 Stack all tranng mages together n x M matrx Compute n x n Covarance Matrx C = YY Algorthm Y = [ YY2... Y 1 M = M T 1 ] Y Y T Algorthm Compute egenvalues and egenvectors of C by solvng λ u = Cu where the egenvalues are λ > λ >... > 1 2 λ n and the correspondng egenvectors are u 1, u 2,, u n Algorthm Each u s an n x 1 egenvector called an egenface (to be cute!) Each u s a vector/drecton n face space Y = w u + w u w u 1 n 1 X = wu + A =1 Image s exactly reconstructed by a lnear combnaton of all egenvectors 2 2 n n Algorthm Reduce dmensonalty by usng only the best k << n egenvectors (.e., the ones correspondng to the largest k egenvalues k X wu + A =1 Each mage X s approxmated by a set of k weghts [w 1, w 2,, w k ] = W where w j = u T j ( X A)

10 Egenface Representaton Egenface Representaton Each face mage s represented by a weghted combnaton of a small number of component or bass faces Usng Egenfaces Reconstructon of an mage of a face from a set of weghts Recognton of a person from a new face mage Face Image Reconstructon Face X n face space coordnates: = Reconstructon: = + ^ X = A + w 1 u 1 + w 2 u 2 + w 3 u 3 + w 4 u 4 +

11 Reconstructon The more egenfaces you use, the better the reconstructon, but even a small number gves good qualty for matchng Egenfaces Recognton Algorthm Modelng (Tranng Phase) 1. Gven a collecton of n labeled tranng mages 2. Compute mean mage, A 3. Compute k egenvectors, u 1,, u k, of covarance matrx correspondng to k largest egenvalues 4. Project each tranng mage, X, to a pont n k-dmensonal face space: for j = 1,..., k compute w j = u ( X T j X projects to W = [w 1, w 2,, w k ] A) Egenfaces Algorthm Recognton (Testng Phase) Choosng the Dmenson K egenvalues 1. Gven a test mage, G, project t nto face space for j = 1,..., k compute w = u ( G A) 2. Classfy t as the class (person) that s closest to t (as long as ts dstance to the closest person s close enough ) j T j = K How many egenfaces to use? Look at the decay of the egenvalues the egenvalue tells you the amount of varance n the drecton of that egenface gnore egenfaces wth low varance NM

12 Example: Tranng Images Egenfaces Note: Faces must be approxmately regstered (translaton, rotaton, sze, pose) [ Turk & Pentland, 2001] Average Image, A 7 egenface mages Tranng mages Example Example Top egenvectors: u 1, u k Average: A 98 99

13 Expermental Results Tranng set: 7,562 mages of approxmately 3,000 people k = 20 egenfaces computed from a sample of 128 mages Test set accuracy on 200 faces was 95% Lmtatons PCA assumes that the data has a Gaussan dstrbuton (mean µ, covarance matrx C) The shape of ths dataset s not well descrbed by ts prncpal components Lmtatons Background (de-emphasze the outsde of the face e.g., by multplyng the nput mage by a 2D Gaussan wndow centered on the face) Lghtng condtons (performance degrades wth lght changes) Scale (performance decreases quckly wth changes to head sze); possble solutons: mult-scale egenspaces scale nput mage to multple szes Orentaton (performance decreases but not as fast as wth scale changes) plane rotatons can be handled out-of-plane rotatons are more dffcult to handle Lmtatons Not robust to msalgnment 106