A QCQP Approach to Triangulation 1 Chris Aholt, Sameer Agarwal, and Rekha Thomas 1 University of Washington 2 Google, Inc. 2 1
The Triangulation Problem X Given: -n camera matrices P i R 3 4 -n noisy observations x i R 2 P i = a i b i X = X 1 x 1 x 1 x 2 x 3 x 3 P 1 x 2 P 2 P 3 X R 3 n i=1 s.t. x i = x i x i 2 a i b i
Previous Work First order optimality Linear initialization + non-linear refinement [Hartley, Seo 08] - Verify global optimality of local solution [Hartley, Sturm 97] - Two-view triangulation [Stewenius, et al. 05] - Groebner basis of local solution space Relaxations [Kahl, et al. 08] - Branch and bound [Kahl, Henrion 07] - SDP relaxations
Previous Work First order optimality Linear initialization + non-linear refinement No guarantees on optimality [Hartley, Seo 08] - Verify global optimality of local solution [Hartley, Sturm 97] - Two-view triangulation [Stewenius, et al. 05] - Groebner basis of local solution space Relaxations [Kahl, et al. 08] - Branch and bound [Kahl, Henrion 07] - SDP relaxations
Previous Work First order optimality Linear initialization + non-linear refinement No guarantees on optimality [Hartley, Seo 08] - Verify global optimality of local solution [Hartley, Sturm 97] - Two-view triangulation [Stewenius, et al. 05] - Groebner basis of local solution space Not extendable to higher n Relaxations [Kahl, et al. 08] - Branch and bound [Kahl, Henrion 07] - SDP relaxations
Previous Work First order optimality Linear initialization + non-linear refinement No guarantees on optimality [Hartley, Seo 08] - Verify global optimality of local solution [Hartley, Sturm 97] - Two-view triangulation [Stewenius, et al. 05] - Groebner basis of local solution space Not extendable to higher n Relaxations [Kahl, et al. 08] - Branch and bound [Kahl, Henrion 07] - SDP relaxations Exponential in precision or in n
Previous Work First order optimality Linear initialization + non-linear refinement No guarantees on optimality [Hartley, Seo 08] - Verify global optimality of local solution [Hartley, Sturm 97] - Two-view triangulation [Stewenius, et al. 05] - Groebner basis of local solution space Not extendable to higher n Relaxations [Kahl, et al. 08] - Branch and bound [Kahl, Henrion 07] - SDP relaxations Exponential in precision or in n Our Method SDP relaxation Pay attention to algebraic structure Polynomial in n
Unconstrained to Constrained X R 3 n i=1 s.t. x i = x i x i 2 a i b i
Unconstrained to Constrained X R 3 x i R 2 n i=1 s.t. x i = x i x i 2 a i b i
Unconstrained to Constrained X R 3 x i R 2 n i=1 s.t. x i = x i x i 2 a i b i x R 2n s.t. x x 2 x =(x 1,...,x n ) V V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i
Unconstrained to Constrained X R 3 x i R 2 n i=1 s.t. x i = x i x i 2 a i b i x R 2n s.t. x x 2 x =(x 1,...,x n ) V Finding the closest x point from to V. V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i
Unconstrained to Constrained X R 3 x i R 2 n i=1 s.t. x i = x i x i 2 a i b i X x 1 x 2 x n x R 2n s.t. x x 2 x =(x 1,...,x n ) V.. P 1 P 2 P n V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i
Unconstrained to Constrained X R 3 x i R 2 n i=1 s.t. x i = x i x i 2 a i b i X x 1 x 2 x n x R 2n s.t. x x 2 x =(x 1,...,x n ) V.. P 1 P 2 P n V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i Can we talk about V without reference to X?
The Constraint Set V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i n = 2 V = {(x 1,x 2 ) : x 1F 12 x 2 =0} epipolar equation (quadratic)
The Constraint Set V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i n = 2 V = {(x 1,x 2 ) : x 1F 12 x 2 =0} epipolar equation (quadratic) n > 2 [Heyden, Astrom 97] V = {(x 1,...,x n ) : x i F ij x j =0,T ijk x i x j x k =0} quadratic cubic
The Constraint Set V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i n = 2 V = {(x 1,x 2 ) : x 1F 12 x 2 =0} epipolar equation (quadratic) n > 2 [Heyden, Astrom 97] V = {(x 1,...,x n ) : x i F ij x j =0,T ijk x i x j x k =0} quadratic cubic W = {(x 1,...,x n ) : x i F ij x j =0} Relaxation of V
The Constraint Set V = {(x 1,...,x n ) : a X R 3 s.t. i x i = i } b i n = 2 V = {(x 1,x 2 ) : x 1F 12 x 2 =0} epipolar equation (quadratic) n > 2 [Heyden, Astrom 97] V = {(x 1,...,x n ) : x i F ij x j =0,T ijk x i x j x k =0} quadratic cubic W = {(x 1,...,x n ) : x i F ij x j =0} Relaxation of V [Heyden, Astrom 97] V = W when n = 2 or non-coplanar cameras.
From QCQP to SDP x s.t. x x 2 x W x W x x Finding the closest point from x to W.
From QCQP to SDP x s.t. x x 2 x W x Y s.t. G, Y F i,y = b i Y 0 rank(y )=1 Rank-constrained semidefinite program.
From QCQP to SDP x s.t. x x 2 x W x Y s.t. G, Y F i,y = b i Y 0 rank(y )=1 Y s.t. G, Y F i,y = b i Y 0 First SDP relaxation - standard! Solvable in time polynomial in n.
From QCQP to SDP x s.t. x x 2 x W x Y s.t. G, Y F i,y = b i Y 0 rank(y )=1 Y s.t. G, Y F i,y = b i Y 0 Y = y (y ) Candidate solution for QCQP
When Does QCQP = SDP? x W x
When Does QCQP = SDP? x W x Perpendicular to W
When Does QCQP = SDP? R 2n S 2n W x x I Y
When Does QCQP = SDP? R 2n S 2n W x x I Y Positive semidefinite cone
When Does QCQP = SDP? W x x I Y Theorem (A, Agarwal, Thomas) QCQP = SDP if and only if Y Can be formulated for general QCQPs with equality constraints. Extends previous results in special cases of QCQPs. Polynomial time test for optimality.
When Does QCQP = SDP? W x x I Y Theorem (A, Agarwal, Thomas) Always works in the case of small noise.
When Does QCQP = SDP? W x x I Y Theorem (A, Agarwal, Thomas) Always works in the case of small noise. Bonus: Always works for two-view triangulation. [Moré 93]
Synthetic - Cameras on Sphere 1 0.9 Fraction guaranteed optimal 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 2 views 3 views 5 views 7 views 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Standard deviation
Synthetic - Coplanar Cameras 1 0.9 Fraction guaranteed optimal 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 2 views 3 views 5 views 7 views 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Standard deviation
Synthetic - Collinear Cameras 1 0.9 0.8 2 views 3 views 4 views Fraction guaranteed optimal 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Standard deviation
Quadratic Optimization Distance Problems Real Data Real Data Data set # images # points Optimal Time (sec) Model House 10 672 100% 143 Corridor 11 737 99.86% 193 Dinosaur 36 4983 100% 960 Notre Dame 48 16,288 98.4% 7200 All the camera configurations are accounted for: Model House Dinosaur Notre Dame Corridor 21/22
Summary of Our Contributions Nearly polynomial time algorithm for triangulation. Geometric understanding from constraints. General theorems for SDP relaxations of QCQPs.