Camera Models and Imaging

Transcription

1 Leow Wee Kheng CS4243 Computer Vision and Pattern Recognition Camera Models and Imaging CS4243 Camera Models and Imaging 1

2 Through our eyes Through our eyes we see the world. CS4243 Camera Models and Imaging 2

3 cornea lens focus light retina captures image iris pupil control amount of light entering the eye transmit image optic nerves CS4243 Camera Models and Imaging 3

4 Through their eyes Camera is computer s eye. CS4243 Camera Models and Imaging 4

5 Camera is structurally the same the eye. sensor retina lens lens cornea CS4243 Camera Models and Imaging 5

6 lens aperture pupil lens cornea CS4243 Camera Models and Imaging 6

7 aperture plates iris aperture pupil CS4243 Camera Models and Imaging 7

8 Imaging Images are 2D projections of real world scene. Images capture two kinds of information: Geometric: positions, points, lines, curves, etc. Photometric: intensity, colour. Complex 3D-2D relationships. Camera models approximate relationships. CS4243 Camera Models and Imaging 8

9 Camera Models Pinhole camera model Orthographic projection Scaled orthographic projection Paraperspective projection Perspective projection CS4243 Camera Models and Imaging 9

10 Pinhole Camera Model image plane 2D image pinhole virtual image plane 3D object focal length CS4243 Camera Models and Imaging 10

11 Math representation world frame / camera frame image frame projection line camera centre image centre optical axis CS4243 Camera Models and Imaging 11

12 By default, use right-handed coordinate system. CS4243 Camera Models and Imaging 12

13 3D point P = (X, Y, Z) T projects to 2D image point p = (x, y) T. By symmetry, i.e., Simplest form of perspective projection. CS4243 Camera Models and Imaging 13

14 Orthographic Projection 3D scene is at infinite distance from camera. All projection lines are parallel to optical axis. So, CS4243 Camera Models and Imaging 14

15 Scaled Orthographic Projection Scene depth << distance to camera. Z is the same for all scene points, say Z 0 CS4243 Camera Models and Imaging 15

16 Perspective Projection camera frame world frame image frame is parallel to camera s X-Y plane CS4243 Camera Models and Imaging 16

17 Intrinsic Parameters Pinhole camera model Camera sensor s pixels not exactly square x, y : coordinates (pixels) k, l : scale parameters (pixels/m) f : focal length (m or mm) CS4243 Camera Models and Imaging 17

18 f, k, l are not independent. Can rewrite as follows (in pixels): So, Image centre or principal point c may not be at origin. Denote location of c in image plane as c x, c y. Then, Along optical axis, X = Y = 0, and x = c x, y = c y. CS4243 Camera Models and Imaging 18

19 Image frame may not be exactly rectangular. Let θ denote skew angle between x- and y-axis. Then, Combine all parameters yield Intrinsic parameter matrix homogeneous coordinates CS4243 Camera Models and Imaging 19

20 Denoting ρ = Z, we get Some books and papers absorb ρ into : giving Be careful. A simpler form of K uses skew parameter s: CS4243 Camera Models and Imaging 20

21 Extrinsic Parameters Camera frame is not aligned with world frame. Rigid transformation between them: frame object Coordinates of 3D scene point in camera frame. Coordinates of 3D scene point in world frame. Rotation matrix of world frame in camera frame. Position of world frame s origin in camera frame. CS4243 Camera Models and Imaging 21

22 Translation matrix Rotation matrix CS4243 Camera Models and Imaging 22

23 Rotation matrix is orthonormal: So,. In particular, let Then, CS4243 Camera Models and Imaging 23

24 Combine intrinsic and extrinsic parameters: Simpler notation: CS4243 Camera Models and Imaging 24

25 Lens Distortion Lens can distort images, especially at short focal length. barrel pin-cushion fisheye CS4243 Camera Models and Imaging 25

26 Radial distortion is modelled as distorted coordinates undistorted coordinates distortion parameters with. Actual image coordinates CS4243 Camera Models and Imaging 26

27 Camera Calibration Compute intrinsic / extrinsic parameters. Capture calibration pattern from various views. CS4243 Camera Models and Imaging 27

28 Detect inner corners in images. Run calibration program (available in OpenCV). CS4243 Camera Models and Imaging 28

29 Homography A transformation between projective planes. Maps straight lines to straight lines. CS4243 Camera Models and Imaging 29

30 Homography equation: image point homography image point Affine is a special case of homography: CS4243 Camera Models and Imaging 30

31 Scaling H by s does not change equation: Homography is defined up to unspecified scale. So, can set h 33 = 1. Expanding homography equation gives CS4243 Camera Models and Imaging 31

32 Assembling equations over all points yields System of linear equations. Easy to solve. CS4243 Camera Models and Imaging 32

33 Example Circles: input corresponding points. Black dots: computed points, match input points. CS4243 Camera Models and Imaging 33

34 Before homographic transform CS4243 Camera Models and Imaging 34

35 After homographic transform CS4243 Camera Models and Imaging 35

36 Summary Simplest camera model: pinhole model. Most commonly used model: perspective model. Intrinsic parameters: Focal length, principal point. Extrinsic parameters: Camera rotation and translation. Lens distortion Homography: maps lines to lines. CS4243 Camera Models and Imaging 36

37 Further Reading Camera models: [Sze10] Section 2.1.5, [FP03] Section 1.1, 1.2, 2.2, 2.3. Lens distortion: [Sze10] Section 2.1.6, [BK08] Chapter 11. Camera calibration: [BK08] Chapter 11, [Zha00]. Image undistortion: [BK08] Chapter 11. CS4243 Camera Models and Imaging 37

38 References Bradski and Kaehler. Learning OpenCV. O Reilly, D. A. Forsyth and J. Ponce. Computer Vision: A Modern Approach. Pearson Education, R. Szeliski. Computer Vision: Algorithms and Applications. Springer, Z. Zhang. A flexible new technique for camera calibration. IEEE Trans. PAMI, 22(11): , CS4243 Camera Models and Imaging 38