EECS 442 Computer vision. Fitting methods

Transcription

1 EECS 442 Computer vision Fitting methods - Problem formulation - Least square methods -RANSAC - Hough transforms - Multi-model fitting - Fitting helps matching! Reading: [HZ] Chapters: 4, 11 [FP] Chapters: 16 Some slides of this lectures are courtesy of profs. S. Lazebnik & K. Grauman

2 Fitting Goal: Choose a parametric model to fit a certain quantity from data -Lines -Curves - Homographic transformation - Fundamental matrix - Shape model

3 Example: Computing vanishing points

4 Example: Estimating an homographic transformation H

5 Example: Estimating F

6 Example: fitting an 2D shape template A

7 Example: fitting a 3D object model

8 Fitting Goal: Choose a parametric model to fit a certain quantity from data Critical issues: -noisy data - outliers - missing data

9 Critical issues: noisy data

10 Critical issues: noisy data (intra-class variability) A

11 Critical issues: outliers H

12 Critical issues: missing data (occlusions)

13 Fitting Goal: Choose a parametric model to fit a certain quantity from data Techniques: Least square methods RANSAC Hough transform EM (Expectation Maximization) [forthcoming lecture]

14 Least squares methods - fitting a line - Data: (x 1, y 1 ),, (x n, y n ) y=mx+b Line equation: y i = m x i + b Find (m, b) to minimize (x i, y i ) E = n i = 1 ( y i mx i b) 2

15 0 2 2 = = Y X XB X db de T T [ ] 2 2 n 1 n 1 n 1 i 2 i i XB Y b m 1 x 1 x y y b m 1 x y E = = = = M M M Normal equation = = n i i i b mx y E 1 2 ) ( Y X XB X T T = Least squares methods - fitting a line - ( ) Y X X X B T T 1 = (XB ) (XB ) Y 2(XB ) Y Y XB ) (Y XB ) (Y T T T T + = =

16 Least squares methods - fitting a line - Ax = b More equations than unknowns Look for solution which minimizes Ax-b = (Ax-b) T (Ax-b) T Solve ( Ax b) ( Ax b) = x i 0 LS solution x = ( A T A) 1 A T b

17 Least squares methods - fitting a line - Solving x = ( A t A) 1 A t b A + = t 1 (A A) A t = pseudo-inverse of A A = U t V = SVD decomposition of A A = V 1 1 U A + = V + U with + 1 equal to for all nonzero singular values and zero otherwise

18 Least squares methods - fitting a line - E = n i = 1 (yi mxi b) 2 y=mx+b B ( T ) 1 T m = X X X Y B = b (x i, y i ) Limitations Not rotation-invariant Fails completely for vertical lines

19 Least squares methods - fitting a line - Distance between point (x n, y n ) and line ax+by=d ax+by=d Find (a, b, d) to minimize the sum of squared perpendicular distances E = n i = 1 ( a x i + b y i d ) 2 (x i, y i ) (U T U) N = 0 data model parameters

20 Least squares methods - fitting a line - A h = 0 Minimize A h subject to h = 1 A = UDV T h = last column of V

21 Least squares methods - fitting an homography - y H y x x

22 Least squares methods - fitting an homography - From n>=4 corresponding points: A h = 0 h h M h 1,1 1,2 3,3 = 0

23 Least squares: Robustness to noise

24 Least squares: Robustness to noise outlier! Problem: squared error heavily penalizes outliers

25 Critical issues: outliers H CONCLUSION: Least square is not robust w.r.t. outliers

26 Least squares: Robust estimators General approach: minimize i ( u ( x, θ ) σ ) ρ ; i i n i = 1 (yi mxi b) u i (x i, θ) residual of i th point w.r.t. model parameters θ ρ robust function with scale parameter σ u = 2 The robust function ρ favors a configuration with small residuals Penalizes large residuals

27 Least squares: Robust estimators Good scale parameter σ The effect of the outlier is eliminated

28 Least squares: Robust estimators Bad scale parameter σ (too small!)

29 Least squares: Robust estimators Bad scale parameter σ (too large!) Same as standard LSQ CONCLUSION: Robust estimator useful if prior info about the distribution of points is known Robust fitting is a nonlinear optimization problem (iterative solution) Least squares solution provides good initial condition

30 Fitting Goal: Choose a parametric model to fit a certain quantity from data Techniques: Least square methods RANSAC Hough transform

31 Basic philosophy (voting scheme) Data elements are used to vote for one (or multiple) models Robust to outliers and missing data Assumption1: Noise features will not vote consistently for any single model ( few outliers) Assumption2: there are enough features to agree on a good model ( few missing data)

32 RANSAC (RANdom SAmple Consensus) : Learning technique to estimate parameters of a model by random sampling of observed data Fischler & Bolles in 81. δ f { P O} π : I, such that: (P, β ) < δ min π O f ( P, β ) = β Model parameters ( T ) 1 T P P P

33 RANSAC Sample set = set of points in 2D Algorithm: 1. Select random sample of minimum required size to fit model 2. Compute a putative model from sample set 3. Compute the set of inliers to this model from whole data set Repeat 1-3 until model with the most inliers over all samples is found

34 RANSAC Sample set = set of points in 2D Algorithm: 1. Select random sample of minimum required size to fit model [?] =[2] 2. Compute a putative model from sample set 3. Compute the set of inliers to this model from whole data set Repeat 1-3 until model with the most inliers over all samples is found

35 RANSAC Sample set = set of points in 2D Algorithm: 1. Select random sample of minimum required size to fit model [?] =[2] 2. Compute a putative model from sample set 3. Compute the set of inliers to this model from whole data set Repeat 1-3 until model with the most inliers over all samples is found

36 RANSAC δ Sample set = set of points in 2D Algorithm: O = Select random sample of minimum required size to fit model [?] =[2] 2. Compute a putative model from sample set 3. Compute the set of inliers to this model from whole data set Repeat 1-3 until model with the most inliers over all samples is found

37 RANSAC (RANdom SAmple Consensus) : Fischler & Bolles in 81. δ Algorithm: O = 6 1. Select random sample of minimum required size to fit model [?] 2. Compute a putative model from sample set 3. Compute the set of inliers to this model from whole data set Repeat 1-3 until model with the most inliers over all samples is found

38 How many samples? Number of samples N Choose N so that, with probability p, at least one random sample is free from outliers (e.g. p=0.99) (outlier ratio: e ) Initial number of points s Typically minimum number needed to fit the model Distance threshold δ Choose δ so probability for inlier is p (e.g. 0.95) Zero-mean Gaussian noise with std. dev. σ: t 2 =3.84σ 2 N = log 1 ( ) s ( p) / log 1 ( 1 e) proportion of outliers e s 5% 10% 20% 25% 30% 40% 50% Source: M. Pollefeys

39 Estimating H by RANSAC H 8 DOF Need 4 correspondences Sample set = set of matches between 2 images Algorithm: 1. Select a random sample of minimum required size [?] 2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model with the most inliers over all samples is found

40 Estimating F by RANSAC Outlier matches F 7 DOF Need 7 (8) correspondences Sample set = set of matches between 2 images Algorithm: 1. Select a random sample of minimum required size [?] 2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model with the most inliers over all samples is found

41 RANSAC - conclusions Good: Simple and easily implementable Successful in different contexts Bad: Many parameters to tune Trade-off accuracy-vs-time Cannot be use if ratio inliers/outliers is too small

42 Fitting Goal: Choose a parametric model to fit a certain quantity from data Techniques: Least square methods RANSAC Hough transform

43 Hough transform P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959 Given a set of, find the curve or line that explains the data points best y m x y = m x + n Hough space n

44 Hough transform y m x n y m x n

45 Hough transform P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959 Issue : parameter space [m,n] is unbounded Use a polar representation for the parameter space y ρ ρ θ x x cosθ + ysinθ = ρ Hough space θ

46 Hough transform - experiments features votes

47 Hough transform - experiments Noisy data features votes Issue: Grid size needs to be adjusted

48 Hough transform - experiments features votes Issue: spurious peaks due to uniform noise

49 Hough transform - conclusions Good: All points are processed independently, so can cope with occlusion/outliers Some robustness to noise: noise points unlikely to contribute consistently to any single bin Bad: Spurious peaks due to uniform noise Trade-off noise-grid size (hard to find sweet point)

50 Hough transform - experiments Courtesy of TKK Automation Technology Laboratory

51 Generalized Hough transform [more on forthcoming lectures] D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981 Identify a shape model by measuring the location of its parts and shape centroid Measurements: orientation theta, location of p Each measurement casts a vote in the Hough space: p + r(θ) a p θ r(θ)

52 Generalized Hough transform B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004

53 Fitting multiple models

54 Fitting multiple models I h H 1 I k H 2 Incremental fitting E.M. (probabilistic fitting) [forthcoming lecture] Hough transform

55 Incremental line fitting Scan data point sequentially (using locality constraints) Perform following loop: 1. Select N point and fit line to N points 2. Compute residual R N 3. Add a new point, re-fit line and re-compute R N+1 4. Continue while line fitting residual is small enough, When residual exceeds a threshold, start fitting new model (line)

56 Hough transform Courtesy of unknown Same cons and pros as before

57 Fitting helps matching! Suppose we need to align these 2 images Images courtesy of M. Brown

58 Fitting helps matching! Feature are matched (for instance, based on correlation)

59 Fitting helps matching! Image 1 Image 2 Matches bases on appearance only Red: good matches Green: bad matches Idea: Fitting an homography H (by RANSAC) mapping features from images 1 to 2 Bad matches will be labeled as outliers (hence rejected)!

60 Fitting helps matching!

61 Recognising Panoramas M. Brown and D. G. Lowe. Recognising Panoramas. In Proceedings of the 9th International Conference on Computer Vision -- ICCV2003

62 Images courtesy of Brandon Lloyd Fitting helps matching!

63

64