Announcements! First homework is on the wiki. Due by Tuesday (Monday is a holiday).!

Transcription

1 Cameras!

2 Announcements! First homework is on the wiki. Due by Tuesday (Monday is a holiday).! Hand in at start of class or to TA (andrea@cds.caltech.edu)! Will take a poll on-line to decide TA office hours. URL for the poll will be announced via the class mailing list -> make sure youʼve signed up.! lhm - 2

3 Heat Transfer Characteristics: Thermal Inertia! (U. Arizona)! lhm - 3

4 Open Problem: Terrain Classification for Mars Rovers! Rovers sometimes get stuck in the ripples; easily cross the bedrock!! How to discriminate the two reliably at low cost for sensors and processing?! lhm - 4

5 Surface Irradiance for Various Lighting Conditions! Point source at infinity (e.g. sun)! E, W/m 2!! n Point source at range r! I, W/sr!! n! i! i da s E i = E cos! i d! i = E i da s Extended source at range r! da i d! i L i, W/sr/m 2!! i! n d! s da s d! s = da s cos" i r 2 d! i = I d! s d! s da s E i = I cos" i r 2 d! s = da cos" s i, d! r 2 i = da i r 2 d! i = L i d! s da i = L i d! i da s cos" i E i = d! i = L i cos" i d! i da s Same for extended source at infinity (e.g. sky)! lhm - 5

6 Recap and Where Weʼre Going! Goal: be able to model the light reaching a pixel in the image or to invert this to estimate scene properties from measured image brightness! lhm - 6

7 This Lecture! Basic optics:! Pinhole camera model! Thin lens! Thick lens! Camera electronics! Focal planes: CCD vs CMOS, interline vs frame transfer, etc.! Noise models! Image processing pipeline within cameras! Color image acquisition and representation! Cameras outside the visible spectrum! lhm - 7

8 In Context with Coming Lectures! Today: cameras: optics, detectors! Thursday: geometric camera modeling and calibration (Adnan Ansar)! Jan 18: feature detection and matching (Roland Brockers)! Jan 20: feature quality assessment outlier detection (Yang Cheng)! Reading material:! Today: Szeliski sec , 2.3, Forsyth ch. 1! Thursday: Szeliski sec. 2.1, 6.3; optional: Forsyth chs. 2, 3! Additional reference material:! E. Hecht, Optics! J. Nakamura, Image Sensors and Signal Processing for Digital Still Cameras! lhm - 8

9 Pinhole Camera! Center of projection! lhm - 9

10 Equivalent Model with Virtual Image Plane! Center of projection! lhm - 10

11 Basic Geometric Properties: (1) Distant Objects are Smaller! lhm - 11

12 Basic Geometric Properties: (2) Lines Project to Lines and Parallel Lines Meet! Vanishing point! Virtual image plane! Center of projection! lhm - 12

13 Basic Geometric Properties: (3) Multiple Parallel Lines form a Vanishing Line! Vanishing line (horizon)! Virtual image plane! Center of projection! lhm - 13

14 Why Care about Vanishing Points and Lines? Example: Urban Localization from Building Facades! lhm - 14

15 Perspective Projection! lhm - 15

16 Weak Perspective (Affine Projection)! lhm - 16

17 Weak Perspective (Affine Projection)! lhm - 17

18 Weak Perspective! Strong perspective:! Angles are not preserved! The projections of parallel lines intersect at one point!! 4-Jan-2011 Weak perspective:! Angles are better preserved! The projections of parallel lines are (almost) parallel!! ME/CS 132 lhm - 18

19 Perspective Projection! lhm - 19

20 Orthographic Projection! lhm - 20

21 Finite Field of View! lhm - 21

22 Homogeneous Coordinates in 2-D! Physical point! p = " x y $ % represented by three coordinates!! " u v w $ % x = u / w y = y / w Two sets of homogeneous coordinate vectors are equivalent if they are proportional to each other:!! " u v w $! ' % " u' v' w' $ % (! " u v w $! '! % " u' v' w' $ %! ) 0 lhm - 22

23 p = x y z! " $ % Homogeneous Coordinates in 3-D! 4-Jan-2011 lhm - 23 ME/CS 132 Physical point represented by three coordinates! Two sets of homogeneous coordinate vectors are equivalent if they are proportional to each other:! u v w t! " $ % x = u / t y = v / t z = w / t u v w t! " $ % ' u' v' w' t '! " $ % ( u v w t! " $ % '! u' v' w' t '! " $ %! ) 0

24 Matrix Representation of Perspective Projection!! p = " u v w $! = % " x y z / f $! = % " / f 1! $ % " x y z 1 $ = P % x' y' = u / w v / w P C! R,T! p = C R T [ ]P B P B! lhm - 24

25 Limitations of Pinhole Cameras/Models! lhm - 25

26 Limitations of Pinhole Cameras/Models! From Hecht, Optics! lhm - 26

27 Snellʼs Law, Spherical Surfaces, and Paraxial Optics! n 1 sin! 1 = n 2 sin! 2 n 1! 1 = n 2! 2 lhm - 27

28 The Thin Lens Model! 1 z' - 1 z = 1 f lhm - 28

29 Depth of Field and the Effect of Aperture! lhm - 29

30 The Thick Lens Model! Thin lens equation still applies if z and zʼ are relative to H and Hʼ! lhm - 30

31 Aberrations! Spherical! (among others)! Distortion! (weʼll model it)! Chromatic! lhm - 31

32 Mechanical Vignetting! lhm - 32

33 Natural Vignetting! lhm - 33

34 lhm - 34

35 Recap and Where Weʼre Going! Goal: be able to model the light reaching a pixel in the image or to invert this to estimate scene properties from measured image brightness! lhm - 35

36 Focal Plane Architectures: CCD Imagers! Frame transfer! Interline transfer! J. Maki, et al., Mars Exploration Rover Engineering Cameras, Journal of Geophysical Research, vol. 108, no. E12, 2003! lhm - 36

37 Focal Plane Architectures: CMOS Imagers! lhm - 37

38 Fill Factor and Quantum Efficiency! Full well: maximum charge that can be stored in a pixel!! Front vs back illumination! lhm - 38

39 Color Camera Architectures! Color mosaic filters (usually 3)! Beam-splitters (2, 3, or 4 way)! Stacked response! Line-scan spectrometers! Tunable filters! lhm - 39

40 Noise Sources in Digital Imagers! Photon shot noise:! Photon arrivals have a Poisson distribution. Variance is equal to the mean.! Dark current shot noise:! Charge accumulates even without incident light. Proportion to integration time; strongly temperature dependent (terms with T 3/2 and T3)! Read noise: from the readout amplifier! Spatial non-uniformities (fixed pattern noise)! Quantization! Smear, blooming! Defects (bad pixels)!! lhm - 40

41 Digital Camera Architecture! lhm - 41

42 Color Response in Cameras! S(λ)! ρ(λ)! σ(λ)! lhm - 42

43 Infrared Detectors! lhm - 43