Direct Camera Calibration

Transcription

1 Direct Camera Calibration

2 Solve for camera parameters (intrinsic, extrinsic, or both) from a set of known 3-D points (known geometry) and their 2-D projections on the image P=[X, Y, Z] in 3-D world coordinate system A = [x, y] image pixel Direct Calibration or from Projection Matrix Intrinsic parameters: principal point [o x, o y ] (image center) scale factors s x, s y (pixel dimension: e.g., pixels/mm) aspect ratio α (s y /s x ) focal length f (mm) lens distortions k 1, k 2, k 3, p 1, p 2 2

3 Typical Calibration Pattern Calibration pattern with known 3-D points In what precision the pattern should be built? Is calibration necessary? 3

4 Direct Calibration Input is a set of known 3-D points in world coordinate system. How to establish the world coordinate system? Three mutually orthogonal directions 3 D points in world coordinate system = [ X w Y w Z w ] The same 3 - Dpoints in the camera coordinate system = [ X c Y c Z c ] 4

5 y c C x c y f c x f z w I F z o r W y w z c y o x o x w The position and orientation of the camera frame is unknown The origin of the camera frame is the center of projection 5

6 X c X w Y c = R Y Z c w +T Rotation first and then translation Z w R and T (extrinsic parameters) are not known X c = r 11 X w + r 12 Y w + r 13 Z w +T x Y c = r 21 X w + r 22 Y w + r 23 Z w +T y Z c = r 31 X w + r 32 Y w + r 33 Z w +T z 6

7 X c Y c Z c = R X w Y w Z w x i = s x f X c y i = s y f Y c + T Z c + o x Z c + o y Let f x = s x f and α = s x /s y Solving for five parameters f, s x,s y,o x,and o y becomes solving for four parameters f x,α,o x, and o y. All four parameters are independent. f x is the focal length expressed in horizontal pixels. 7

8 x i o x = f x r 11 X w + r 12 Y w + r 13 Z w + T x r 31 X w + r 32 Y w + r 33 Z w + T z y i o y = f y r 21 X w + r 22 Y w + r 23 Z w + T y r 31 X w + r 32 Y w + r 33 Z w + T z If we know a sufficient number of 3-D points and image data points (x, y), then all parameters can be calculated. 8

9 Problem Statement x i o x = f x r 11 X w + r 12 Y w + r 13 Z w + T x r 31 X w + r 32 Y w + r 33 Z w + T z y i o y = f y r 21 X w + r 22 Y w + r 23 Z w + T y r 31 X w + r 32 Y w + r 33 Z w + T z f x,α,r, and T can be estimated from known world points [X i w,y i w,z i w ] T and their projected image points (x i, y i ) and i = 1,2,...N, assuming o x and o y are known (the origin of the image reference frame) 9

10 10

11 x i (r 21 X w + r 22 Y w + r 23 Z w + T y ) y i α(r 11 X w + r 12 Y w + r 13 Z w + T x ) = 0 A [r 21 r 22 r 23 T y αr 11 αr 12 αr 13 αt x ] T = Av = 0 Using SVD (A = UDV T ) to solve for v as the column of V corresponding to the only null or the smallest (noise) singular value (eigenvalue) along the diagonal of D 11

12 Av = 0 A is a N 8 matrix. If N 7 and the N points are not coplanar, A has rank 7 and the homogeneous system has a nontrivial solution (unique up to an unknow scale factor). but becasue of image noise and inaccuracy in locating 2- D points. A could have the maximum rank (8). In this case, there will be no null singular value along the diagonal of D and the system would only have a trivial solution. In this case, we use the eigenvector corresponding to the smallest eigenvalue. 12

13 Solving Av = 0 to get the solution upto an unknow scale factor v = γ[r 21 r 22 r 23 T y αr 11 αr 12 αr 13 αt x ], where γ is the scale factor. Since r r r 2 23 = 1 (why?), γ 2 (r r r 2 23 ) = γ Similarly, r r r 2 13 = 1 and α > 0, γ 2 α 2 (r r r 2 13 ) = αγ Solve these two equations for ϒ and α. 13

14 Will the sign of the third row change if the sign of scale factor is reversed? 14

15 15

16 What parameters are missing? x o x = f x r 11 X w + r 12 Y w + r 13 Z w + T x r 31 X w + r 32 Y w + r 33 Z w + T z y o y = f y r 21 X w + r 22 Y w + r 23 Z w + T y r 31 X w + r 32 Y w + r 33 Z w + T z Determine the unknow sign for scale factor γ. Z c > 0 x(r 11 X w + r 12 Y w + r 13 Z w + T x ) > 0 and y(r 21 X w + r 22 Y w + r 23 Z w + T y ) > 0 This is true if virtual image plane is used, otherwise the sign has to be reversed. 16

17 x = f x r 11 X w + r 12 Y w + r 13 Z w + T x r 31 X w + r 32 Y w + r 33 Z w + T z x(r 31 X w + r 32 Y w + r 33 Z w + T z ) = f x (r 11 X w + r 12 Y w + r 13 Z w + T x ) xt z + f x (r 11 X w + r 12 Y w + r 13 Z w + T x ) = x(r 31 X w + r 32 Y w + r 33 Z w ) 17

18 [ x (r 11 X w + r 12 Y w + r 13 Z w + T x )] T z f x f x = x(r 31 X w + r 32 Y w + r 33 Z w ) A T z = b T z f x = (A T A) 1 A T b 18

19 x 1 (r 11 X W 1 + r 12 Y W 1 + r 13 Z W 1 + T x ) x A = 2 (r 11 X W 2 + r 12 Y W 2 + r 13 Z W 2 + T x ). x n (r 11 X W n + r 12 Y W n + r 13 Z W n + T x ) b = x 1 (r 31 X W 1 + r 32 Y W 1 + r 33 Z W 1 ) x 2 (r 31 X W 2 + r 32 Y W 2 + r 33 Z W 2 ) x n (r 31 X W n + r 32 Y W n + r 33 Z W n ) 19

20 Locate three vanishing points from three sets of mutually orthogonal parallel lines in space The image center is the orthocenter of the plane defined by the three vanishing points Position of the calibration pattern is critical One of the three mutually orthogonal directions can be nearly parallel to the image plane, which should be avoided Noise or inaccuracy of parallel lines cause problem Average results from several views of the calibration pattern 20

21 21

22 22

23 Camera Parameters from Projection Matrix 23

24 24

25 25

26 Again, A = UDV T and m is the column of V corresponding to the null singular value upto an unknown scale factor 26

27 Am = 0 A is a 2N 12 matrix. N 6 (6 pairs of x and y = 12) If N 11 and the N points are not coplanar, A has rank 11 and the homogeneous system has a nontrivial solution (unique up to an unknow scale factor). but becasue of image noise and inaccuracy in locating 2- D points. A could have the maximum rank (12). In this case, there will be no null singular value along the diagonal of D and the system would only have a trivial solution. In this case, we use the eigenvector corresponding to the smallest eigenvalue. 27

28 x o x = f x r 11 X w + r 12 Y w + r 13 Z w + T x r 31 X w + r 32 Y w + r 33 Z w + T z y o y = f y r 21 X w + r 22 Y w + r 23 Z w + T y r 31 X w + r 32 Y w + r 33 Z w + T z 28

29 f x r 11 + o x r 31 f x r 12 + o x r 32 f x r 13 + o x r 33 f x T x + o x T z M = f y r 21 + o y r 31 f y r 22 + o y r 32 f y r 23 + o y r 33 f y T y + o y T z r 31 r 32 r 33 T z 29

30 f x r 11 + o x r 31 f x r 12 + o x r 32 f x r 13 + o x r 33 f x T x + o x T z M = f y r 21 + o y r 31 f y r 22 + o y r 32 f y r 23 + o y r 33 f y T y + o y T z r 31 r 32 r 33 T z 30

31 31

32 [ f x r 11 + o x r 31 f x r 12 + o x r 32 f x r 13 + o x r 33 ][ r 31 r 32 r 33 ] T = o x [ f y r 21 + o y r 31 f y r 22 + o y r 32 f y r 23 + o y r ] 33 [ r 31 r 32 r 33 ] T = o y Why? Orthogonality. o x = [ m ˆ 11 m ˆ 12 m ˆ 13 ][ m ˆ 31 m ˆ 32 m ˆ 33 ] T o y = [ m ˆ 21 m ˆ 22 m ˆ 23 ][ m ˆ 31 m ˆ 32 m ˆ 33 ] T 32

33 q 1 = [ f x r 11 + o x r 31 f x r 12 + o x r 32 f x r 13 + o x r 33 ] q 2 = [ f y r 21 + o y r 31 f y r 22 + o y r 32 f y r 23 + o y r ] 33 f x = q 1 q T 1 o 2 x = [ m ˆ 11 m ˆ 12 m ˆ 13 ] T 2 [ m ˆ 11 m ˆ 12 m ˆ 13 ] o x f y = q 2 q T 2 o 2 y = [ m ˆ 21 m ˆ 22 m ˆ 23 ] T 2 [ m ˆ 21 m ˆ 22 m ˆ 23 ] o y 33

34 f x T x + o x T z = ˆ m 14 f y T x + o y T z = ˆ m 24 34