# Epipolar Geometry and Stereo Vision

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 04/12/11 Epipolar Geometry and Stereo Vision Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverese, Steve Seitz, many figures from Hartley & Zisserman

2 HW 3 is back Stats HW1: mean= 93, quartile= 91, median= 97 HW2: mean= 89, quartile= 86, median= 96 HW3: mean= 94, quartile= 89, median= 99 Summary Most homeworks were basically correct Problem 1b: u2, v2 need to account for scale and orientation Problem 1c: main causes of error were perspective and multiple objects Problem 3: some extra credit possible Problem 4: sometimes wanted more detail

3 This class: Two-View Geometry Epipolar geometry Relates cameras from two views Stereo depth estimation Recover depth from two images

4 Depth from Stereo Goal: recover depth by finding image coordinate x that corresponds to x X X x x x z x' f f C Baseline B C

5 Depth from Stereo Goal: recover depth by finding image coordinate x that corresponds to x Problems Calibration: How do we recover the relation of the cameras (if not already known)? Correspondence: How do we search for the matching point x? X X x x x z x' f f C Baseline C B

6 Correspondence Problem We have two images taken from cameras at different positions How do we match a point in the first image to a point in the second? What constraints do we have?

7 Key idea: Epipolar constraint X X X x x x x Potential matches for x have to lie on the corresponding line l. Potential matches for x have to lie on the corresponding line l.

8 Epipolar geometry: notation X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the other camera center

9 Epipolar geometry: notation X x x Baseline line connecting the two camera centers Epipolar Plane plane containing baseline (1D family) Epipoles = intersections of baseline with image planes = projections of the other camera center Epipolar Lines - intersections of epipolar plane with image planes (always come in corresponding pairs)

10 Example: Converging cameras

11 Example: Motion parallel to image plane

12 Example: Forward motion What would the epipolar lines look like if the camera moves directly forward?

13 Example: Forward motion e e Epipole has same coordinates in both images. Points move along lines radiating from e: Focus of expansion

14 Epipolar constraint: Calibrated case X x x Suppose that we know the intrinsic and extrinsic parameters of the cameras. Then we can 1. Convert to normalized coordinates by pre-multiplying all points with the inverse of the calibration matrix 2. Set the first camera s coordinate system as world coordinates and define R and t that map from X to X Here, x is in homogeneous coordinates but X is in inhomogeneous coordinates xˆ K 1 x X X xˆ K RX t x X 1

15 Epipolar constraint: Calibrated case X Rxˆ t x x t R xˆ K 1 x X X The vectors xˆ, t, and xˆ K RX t Rˆ x x X 1 are coplanar

16 Essential matrix X x x xˆ [ t ( Rxˆ )] 0 ˆ x T Exˆ 0 with E t R Essential Matrix (Longuet-Higgins, 1981) The vectors xˆ, t, and Rx are coplanar

17 Properties of the Essential matrix X x x xˆ [ t ( Rxˆ )] 0 ˆ x T Exˆ 0 with E t R Drop ^ below to simplify notation E x is the epipolar line associated with x (l = E x ) E T x is the epipolar line associated with x (l = E T x) E e = 0 and E T e = 0 E is singular (rank two) E has five degrees of freedom (3 for R, 2 for t because it s up to a scale)

18 Epipolar constraint: Uncalibrated case X x x If we don t know K and K, then we can write the epipolar constraint in terms of unknown normalized coordinates: ˆ x T Ex ˆ 0 x K xˆ, x Kx ˆ

19 The Fundamental Matrix X x x xˆ T Ex ˆ 0 x T F x 0 with F K EK T 1 x x K xˆ Kxˆ Fundamental Matrix (Faugeras and Luong, 1992)

20 Properties of the Fundamental matrix X x x x F x 0 with F x is the epipolar line associated with x (l = F x ) F T x is the epipolar line associated with x (l = F T x) F e = 0 and F T e = 0 F is singular (rank two): det(f)=0 F has seven degrees of freedom T F K EK T 1

21 Estimating the Fundamental Matrix 8-point algorithm Least squares solution using SVD on equations from 8 pairs of correspondences Enforce det(f)=0 constraint using SVD on F 7-point algorithm Use least squares to solve for null space (two vectors) using SVD and 7 pairs of correspondences Solve for linear combination of null space vectors that satisfies det(f)=0 Minimize reprojection error Non-linear least squares

22 8-point algorithm 1. Solve a system of homogeneous linear equations a. Write down the system of equations x T Fx 0

23 8-point algorithm 1. Solve a system of homogeneous linear equations a. Write down the system of equations b. Solve f from Af=0 using SVD Matlab: [U, S, V] = svd(a); f = V(:, end); F = reshape(f, [3 3]) ;

24 Need to enforce singularity constraint

25 8-point algorithm 1. Solve a system of homogeneous linear equations a. Write down the system of equations b. Solve f from Af=0 using SVD Matlab: [U, S, V] = svd(a); f = V(:, end); F = reshape(f, [3 3]) ; 2. Resolve det(f) = 0 constraint by SVD Matlab: [U, S, V] = svd(f); S(3,3) = 0; F = U*S*V ;

26 8-point algorithm 1. Solve a system of homogeneous linear equations a. Write down the system of equations b. Solve f from Af=0 using SVD 2. Resolve det(f) = 0 constraint by SVD Notes: Use RANSAC to deal with outliers (sample 8 points) Solve in normalized coordinates mean=0 RMS distance = (1,1,1) just like with estimating the homography for stitching

27 Comparison of homography estimation and the 8-point algorithm Assume we have matched points x x with outliers Homography (No Translation) Fundamental Matrix (Translation)

28 Comparison of homography estimation and the 8-point algorithm Assume we have matched points x x with outliers Homography (No Translation) Correspondence Relation x' Hx x' Hx 1. Normalize image coordinates ~ x Tx ~ x T x 2. RANSAC with 4 points 0 Fundamental Matrix (Translation) 3. De-normalize: H T 1 ~ HT

29 Comparison of homography estimation and the 8-point algorithm Assume we have matched points x x with outliers Homography (No Translation) Correspondence Relation x' Hx x' Hx 1. Normalize image coordinates ~ x Tx ~ x T x 2. RANSAC with 4 points 3. De-normalize: 0 H T 1 ~ HT Fundamental Matrix (Translation) Correspondence Relation x T Fx 0 1. Normalize image coordinates ~ x Tx ~ x T x 2. RANSAC with 8 points 3. Enforce det F by SVD ~ 4. De-normalize: 0 F T 1 ~ FT

30 7-point algorithm Faster (need fewer points) and could be more robust (fewer points), but also need to check for degenerate cases

31 Gold standard algorithm Use 8-point algorithm to get initial value of F Use F to solve for P and P (discussed later) Jointly solve for 3d points X and F that minimize the squared re-projection error X x x' See Algorithm 11.2 and Algorithm 11.3 in HZ (pages ) for details

32 Comparison of estimation algorithms 8-point Normalized 8-point Nonlinear least squares Av. Dist pixels 0.92 pixel 0.86 pixel Av. Dist pixels 0.85 pixel 0.80 pixel

33 We can get projection matrices P and P up to a projective ambiguity K *rotation K *translation P I 0 P e F e e T F 0 See HZ p Code: function P = vgg_p_from_f(f) [U,S,V] = svd(f); e = U(:,3); P = [-vgg_contreps(e)*f e];

34 From epipolar geometry to camera calibration Estimating the fundamental matrix is known as weak calibration If we know the calibration matrices of the two cameras, we can estimate the essential matrix: E = K T FK The essential matrix gives us the relative rotation and translation between the cameras, or their extrinsic parameters

35 Moving on to stereo Fuse a calibrated binocular stereo pair to produce a depth image image 1 image 2 Dense depth map Many of these slides adapted from Steve Seitz and Lana Lazebnik

36 Basic stereo matching algorithm For each pixel in the first image Find corresponding epipolar line in the right image Examine all pixels on the epipolar line and pick the best match Triangulate the matches to get depth information Simplest case: epipolar lines are scanlines When does this happen?

37 Simplest Case: Parallel images Image planes of cameras are parallel to each other and to the baseline Camera centers are at same height Focal lengths are the same

38 Simplest Case: Parallel images Image planes of cameras are parallel to each other and to the baseline Camera centers are at same height Focal lengths are the same Then, epipolar lines fall along the horizontal scan lines of the images

39 Simplest Case: Parallel images R t E Ex x T, T T R t E Epipolar constraint: v T Tv Tv T v u v u T T v u R = I t = (T, 0, 0) The y-coordinates of corresponding points are the same t x x

40 Depth from disparity X x x O O f z x x z f f O Baseline O B disparity x x B z f Disparity is inversely proportional to depth.

41 Stereo image rectification

42 Stereo image rectification Reproject image planes onto a common plane parallel to the line between camera centers Pixel motion is horizontal after this transformation Two homographies (3x3 transform), one for each input image reprojection C. Loop and Z. Zhang. Computing Rectifying Homographies for Stereo Vision. IEEE Conf. Computer Vision and Pattern Recognition, 1999.

43 Rectification example

44 Basic stereo matching algorithm If necessary, rectify the two stereo images to transform epipolar lines into scanlines For each pixel x in the first image Find corresponding epipolar scanline in the right image Examine all pixels on the scanline and pick the best match x Compute disparity x-x and set depth(x) = fb/(x-x )

45 Correspondence search Left Right scanline Matching cost disparity Slide a window along the right scanline and compare contents of that window with the reference window in the left image Matching cost: SSD or normalized correlation

46 Correspondence search Left Right scanline SSD

47 Correspondence search Left Right scanline Norm. corr

48 Effect of window size Smaller window + More detail More noise W = 3 W = 20 Larger window + Smoother disparity maps Less detail

49 Failures of correspondence search Textureless surfaces Occlusions, repetition Non-Lambertian surfaces, specularities

50 Results with window search Data Window-based matching Ground truth

51 How can we improve window-based matching? So far, matches are independent for each point What constraints or priors can we add?

52 Stereo constraints/priors Uniqueness For any point in one image, there should be at most one matching point in the other image

53 Stereo constraints/priors Uniqueness For any point in one image, there should be at most one matching point in the other image Ordering Corresponding points should be in the same order in both views

54 Stereo constraints/priors Uniqueness For any point in one image, there should be at most one matching point in the other image Ordering Corresponding points should be in the same order in both views Ordering constraint doesn t hold

55 Priors and constraints Uniqueness For any point in one image, there should be at most one matching point in the other image Ordering Corresponding points should be in the same order in both views Smoothness We expect disparity values to change slowly (for the most part)

56 Stereo matching as energy minimization I 1 I 2 D W 1 (i ) W 2 (i+d(i )) D(i ) E E data ( D; I1, I2 ) Esmooth( D) E W ( i) W ( i D( i 2 data 1 2 )) i smooth D( i) D( j) neighborsi, j Energy functions of this form can be minimized using graph cuts E 2 Y. Boykov, O. Veksler, and R. Zabih, Fast Approximate Energy Minimization via Graph Cuts, PAMI 2001

57 Many of these constraints can be encoded in an energy function and solved using graph cuts Before Graph cuts Ground truth Y. Boykov, O. Veksler, and R. Zabih, Fast Approximate Energy Minimization via Graph Cuts, PAMI 2001 For the latest and greatest:

58 Summary Epipolar geometry Epipoles are intersection of baseline with image planes Matching point in second image is on a line passing through its epipole Fundamental matrix maps from a point in one image to a line (its epipolar line) in the other Can solve for F given corresponding points (e.g., interest points) Can recover canonical camera matrices from F (with projective ambiguity) Stereo depth estimation Estimate disparity by finding corresponding points along scanlines Depth is inverse to disparity

59 Next class: structure from motion

### Camera calibration and epipolar geometry. Odilon Redon, Cyclops, 1914

Camera calibration and epipolar geometry Odilon Redon, Cyclops, 94 Review: Alignment What is the geometric relationship between pictures taken by cameras that share the same center? How many points do

### C4 Computer Vision. 4 Lectures Michaelmas Term Tutorial Sheet Prof A. Zisserman. fundamental matrix, recovering ego-motion, applications.

C4 Computer Vision 4 Lectures Michaelmas Term 2004 1 Tutorial Sheet Prof A. Zisserman Overview Lecture 1: Stereo Reconstruction I: epipolar geometry, fundamental matrix. Lecture 2: Stereo Reconstruction

### Stereo Vision (Correspondences)

Stereo Vision (Correspondences) EECS 598-08 Fall 2014! Foundations of Computer Vision!! Instructor: Jason Corso (jjcorso)! web.eecs.umich.edu/~jjcorso/t/598f14!! Readings: FP 7; SZ 11; TV 7! Date: 10/27/14!!

### Epipolar Geometry Prof. D. Stricker

Outline 1. Short introduction: points and lines Epipolar Geometry Prof. D. Stricker 2. Two views geometry: Epipolar geometry Relation point/line in two views The geometry of two cameras Definition of the

### Epipolar Geometry. Readings: See Sections 10.1 and 15.6 of Forsyth and Ponce. Right Image. Left Image. e(p ) Epipolar Lines. e(q ) q R.

Epipolar Geometry We consider two perspective images of a scene as taken from a stereo pair of cameras (or equivalently, assume the scene is rigid and imaged with a single camera from two different locations).

### Depth from a single camera

Depth from a single camera Fundamental Matrix Essential Matrix Active Sensing Methods School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie 1 1 Geometry of two

### Introduction Epipolar Geometry Calibration Methods Further Readings. Stereo Camera Calibration

Stereo Camera Calibration Stereo Camera Calibration Stereo Camera Calibration Stereo Camera Calibration 12.10.2004 Overview Introduction Summary / Motivation Depth Perception Ambiguity of Correspondence

### 2-View Geometry. Mark Fiala Ryerson University Mark.fiala@ryerson.ca

CRV 2010 Tutorial Day 2-View Geometry Mark Fiala Ryerson University Mark.fiala@ryerson.ca 3-Vectors for image points and lines Mark Fiala 2010 2D Homogeneous Points Add 3 rd number to a 2D point on image

### Calibrating a Camera and Rebuilding a Scene by Detecting a Fixed Size Common Object in an Image

Calibrating a Camera and Rebuilding a Scene by Detecting a Fixed Size Common Object in an Image Levi Franklin Section 1: Introduction One of the difficulties of trying to determine information about a

### Homography. Dr. Gerhard Roth

Homography Dr. Gerhard Roth Epipolar Geometry P P l P r Epipolar Plane p l Epipolar Lines p r O l e l e r O r Epipoles P r = R(P l T) Homography Consider a point x = (u,v,1) in one image and x =(u,v,1)

### Lecture 19 Camera Matrices and Calibration

Lecture 19 Camera Matrices and Calibration Project Suggestions Texture Synthesis for In-Painting Section 10.5.1 in Szeliski Text Project Suggestions Image Stitching (Chapter 9) Face Recognition Chapter

### The calibration problem was discussed in details during lecture 3.

1 2 The calibration problem was discussed in details during lecture 3. 3 Once the camera is calibrated (intrinsics are known) and the transformation from the world reference system to the camera reference

### CSE 252B: Computer Vision II

CSE 252B: Computer Vision II Lecturer: Serge Belongie Scribes: Jia Mao, Andrew Rabinovich LECTURE 9 Affine and Euclidean Reconstruction 9.1. Stratified reconstruction Recall that in 3D reconstruction from

### Epipolar Geometry and Visual Servoing

Epipolar Geometry and Visual Servoing Domenico Prattichizzo joint with with Gian Luca Mariottini and Jacopo Piazzi www.dii.unisi.it/prattichizzo Robotics & Systems Lab University of Siena, Italy Scuoladi

### EXPERIMENTAL EVALUATION OF RELATIVE POSE ESTIMATION ALGORITHMS

EXPERIMENTAL EVALUATION OF RELATIVE POSE ESTIMATION ALGORITHMS Marcel Brückner, Ferid Bajramovic, Joachim Denzler Chair for Computer Vision, Friedrich-Schiller-University Jena, Ernst-Abbe-Platz, 7743 Jena,

### Structured light systems

Structured light systems Tutorial 1: 9:00 to 12:00 Monday May 16 2011 Hiroshi Kawasaki & Ryusuke Sagawa Today Structured light systems Part I (Kawasaki@Kagoshima Univ.) Calibration of Structured light

### Geometric Camera Parameters

Geometric Camera Parameters What assumptions have we made so far? -All equations we have derived for far are written in the camera reference frames. -These equations are valid only when: () all distances

### Minimal Projective Reconstruction for Combinations of Points and Lines in Three Views

Minimal Projective Reconstruction for Combinations of Points and Lines in Three Views Magnus Oskarsson 1, Andrew Zisserman and Kalle Åström 1 1 Centre for Mathematical Sciences Lund University,SE 1 00

### Computer Vision - part II

Computer Vision - part II Review of main parts of Section B of the course School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture Name Course Name 1 1 2

### Feature Tracking and Optical Flow

02/09/12 Feature Tracking and Optical Flow Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve

### Interactive 3D Scanning Without Tracking

Interactive 3D Scanning Without Tracking Matthew J. Leotta, Austin Vandergon, Gabriel Taubin Brown University Division of Engineering Providence, RI 02912, USA {matthew leotta, aev, taubin}@brown.edu Abstract

### Homographies and Panoramas. Slides from Steve Seitz, Rick Szeliski, Alexei Efros, Fredo Durand, and Kristin Grauman

Homographies and Panoramas Slides from Steve Seitz, Rick Szeliski, Alexei Efros, Fredo Durand, and Kristin Grauman cs129: Computational Photography James Hays, Brown, Fall 2012 Why Mosaic? Are you getting

### Camera geometry and image alignment

Computer Vision and Machine Learning Winter School ENS Lyon 2010 Camera geometry and image alignment Josef Sivic http://www.di.ens.fr/~josef INRIA, WILLOW, ENS/INRIA/CNRS UMR 8548 Laboratoire d Informatique,

### Advanced Techniques for Mobile Robotics Camera Calibration. Wolfram Burgard, Cyrill Stachniss, Kai Arras, Maren Bennewitz

Advanced Techniques for Mobile Robotics Camera Calibration Wolfram Burgard, Cyrill Stachniss, Kai Arras, Maren Bennewitz What is Camera Calibration? A camera projects 3D world points onto the 2D image

### Introduction to Computer Vision. Week 11, Fall 2010 Instructor: Prof. Ko Nishino

Introduction to Computer Vision Week 11, Fall 2010 Instructor: Prof. Ko Nishino The Projective Plane Why do we need homogeneous coordinates? represent points at infinity, homographies, perspective projection,

### Geometric Transformations and Image Warping: Mosaicing

Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker, Guido Gerig SCI Institute, School of Computing University of Utah (with slides from: Jinxiang Chai, TAMU) faculty.cs.tamu.edu/jchai/cpsc641_spring10/lectures/lecture8.ppt

### IMAGE FORMATION. Antonino Furnari

IPLab - Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli Studi di Catania http://iplab.dmi.unict.it IMAGE FORMATION Antonino Furnari furnari@dmi.unict.it http://dmi.unict.it/~furnari

### Wii Remote Calibration Using the Sensor Bar

Wii Remote Calibration Using the Sensor Bar Alparslan Yildiz Abdullah Akay Yusuf Sinan Akgul GIT Vision Lab - http://vision.gyte.edu.tr Gebze Institute of Technology Kocaeli, Turkey {yildiz, akay, akgul}@bilmuh.gyte.edu.tr

### Visual modeling with a hand-held camera

Visual modeling with a hand-held camera MARC POLLEFEYS Department of Computer Science, University of North Carolina, Chapel Hill, NC 27599-3175 marc@cs.unc.edu LUC VAN GOOL, MAARTEN VERGAUWEN, FRANK VERBIEST,

### Homography Estimation

Homography Estimation by Elan Dubrofsky B.Sc., Carleton University, 2007 A MASTER S ESSAY SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in The Faculty of Graduate

### Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

### Announcements. Active stereo with structured light. Project structured light patterns onto the object

Announcements Active stereo with structured light Project 3 extension: Wednesday at noon Final project proposal extension: Friday at noon > consult with Steve, Rick, and/or Ian now! Project 2 artifact

### Relating Vanishing Points to Catadioptric Camera Calibration

Relating Vanishing Points to Catadioptric Camera Calibration Wenting Duan* a, Hui Zhang b, Nigel M. Allinson a a Laboratory of Vision Engineering, University of Lincoln, Brayford Pool, Lincoln, U.K. LN6

### Projective Reconstruction from Line Correspondences

Projective Reconstruction from Line Correspondences Richard I. Hartley G.E. CRD, Schenectady, NY, 12301. Abstract The paper gives a practical rapid algorithm for doing projective reconstruction of a scene

### DETECTION OF PLANAR PATCHES IN HANDHELD IMAGE SEQUENCES

DETECTION OF PLANAR PATCHES IN HANDHELD IMAGE SEQUENCES Olaf Kähler, Joachim Denzler Friedrich-Schiller-University, Dept. Mathematics and Computer Science, 07743 Jena, Germany {kaehler,denzler}@informatik.uni-jena.de

### Notes on the Use of Multiple Image Sizes at OpenCV stereo

Notes on the Use of Multiple Image Sizes at OpenCV stereo Antonio Albiol April 5, 2012 Abstract This paper explains how to use different image sizes at different tasks done when using OpenCV for stereo

### Using Spatially Distributed Patterns for Multiple View Camera Calibration

Using Spatially Distributed Patterns for Multiple View Camera Calibration Martin Grochulla, Thorsten Thormählen, and Hans-Peter Seidel Max-Planc-Institut Informati, Saarbrücen, Germany {mgrochul, thormae}@mpi-inf.mpg.de

### 3D Scanner using Line Laser. 1. Introduction. 2. Theory

. Introduction 3D Scanner using Line Laser Di Lu Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute The goal of 3D reconstruction is to recover the 3D properties of a geometric

### Robust Estimation of Light Directions and Diffuse Reflectance of Known Shape Object

Robust Estimation of Light Directions and Diffuse Reflectance of Known Shape Object Takehiro TACHIKAWA, Shinsaku HIURA, Kosuke SATO Graduate School of Engineering Science, Osaka University Email: {tachikawa,

### Kinect & 3D. Tamara Berg Advanced Mul7media

Kinect & 3D Tamara Berg Advanced Mul7media Recovering 3D from Images How can we automa7cally compute 3D geometry from images? What cues in the image provide 3D informa7on? Slide credit: S. Narasimhan Visual

### Real-Time Automated Simulation Generation Based on CAD Modeling and Motion Capture

103 Real-Time Automated Simulation Generation Based on CAD Modeling and Motion Capture Wenjuan Zhu, Abhinav Chadda, Ming C. Leu and Xiaoqing F. Liu Missouri University of Science and Technology, zhuwe@mst.edu,

### Multi-View Geometry for General Camera Models

Multi-View Geometry for General Camera Models P. Sturm INRIA Rhône-Alpes, 3833 Montbonnot, France Abstract We consider the structure from motion problem for a previously introduced, highly general imaging

### An Efficient Solution to the Five-Point Relative Pose Problem

An Efficient Solution to the Five-Point Relative Pose Problem David Nistér Sarnoff Corporation CN5300, Princeton, NJ 08530 dnister@sarnoff.com Abstract An efficient algorithmic solution to the classical

### Project 1. Appreciate using the link without attachment Check the permission (use incognito to click the link)

Image Stitching 2 Project 1 Appreciate using the link without attachment Check the permission (use incognito to click the link) 2 Project 2 Creating your own panorama! 1. You can do by yourself or form

### Module 6: Pinhole camera model Lecture 30: Intrinsic camera parameters, Perspective projection using homogeneous coordinates

The Lecture Contains: Pinhole camera model 6.1 Intrinsic camera parameters A. Perspective projection using homogeneous coordinates B. Principal-point offset C. Image-sensor characteristics file:///d /...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2030/30_1.htm[12/31/2015

### An approach to stereo-point cloud registration using image homographies

An approach to stereo-point cloud registration using image homographies Stephen D. Fox a and Damian M. Lyons a a Fordham University, Robotics and Computer Vision Laboratory, Bronx, NY ABSTRACT A mobile

### Hand-held acquisition of 3D models with a video camera

Hand-held acquisition of 3D models with a video camera M. Pollefeys, R. Koch, M. Vergauwen and L. Van Gool ESAT-PSI, K.U.Leuven Kardinaal Mercierlaan 94 B-3001 Heverlee, Belgium Marc.Pollefeys@esat.kuleuven.ac.be

### Implementation of Machine Vision System for Finding Defects in Wheel Alignment

Implementation of Machine Vision System for Finding Defects in Wheel Alignment 1 Akshay Padegaonkar, 2 Madhura Brahme, 3 Mohit Bangale, 4 Alex Noel Joseph Raj 1, 2, 3 M. Tech Student, Embedded Systems,

### 2D-to-3D Photo Rendering for 3D Displays

2D-to-3D Photo Rendering for 3D Displays Dario Comanducci Dip. di Sistemi e Informatica, Univ. di Firenze Via S.Marta 3, 50139 Firenze, Italy comandu@dsi.unifi.it Carlo Colombo Dip. di Sistemi e Informatica,

### Current status of image matching for Earth observation

Current status of image matching for Earth observation Christian Heipke IPI - Institute for Photogrammetry and GeoInformation Leibniz Universität Hannover Secretary General, ISPRS Content Introduction

### Solving Simultaneous Equations and Matrices

Solving Simultaneous Equations and Matrices The following represents a systematic investigation for the steps used to solve two simultaneous linear equations in two unknowns. The motivation for considering

### SCREEN-CAMERA CALIBRATION USING A THREAD. Songnan Li, King Ngi Ngan, Lu Sheng

SCREEN-CAMERA CALIBRATION USING A THREAD Songnan Li, King Ngi Ngan, Lu Sheng Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR ABSTRACT In this paper, we propose

### Calibration of a trinocular system formed with wide angle lens cameras

Calibration of a trinocular system formed with wide angle lens cameras Carlos Ricolfe-Viala,* Antonio-Jose Sanchez-Salmeron, and Angel Valera Instituto de Automática e Informática Industrial, Universitat

### Algebra and Linear Algebra

Vectors Coordinate frames 2D implicit curves 2D parametric curves 3D surfaces Algebra and Linear Algebra Algebra: numbers and operations on numbers 2 + 3 = 5 3 7 = 21 Linear algebra: tuples, triples,...

### Parallel Tracking and Mapping for Small AR Workspaces

Parallel Tracking and Mapping for Small AR Workspaces Georg Klein and David Murray Active Vision Lab, Oxford This is a PDF of the slides of the talk given at ISMAR 2007 Aim AR with a hand-held camera Visual

### Lecture 5: Singular Value Decomposition SVD (1)

EEM3L1: Numerical and Analytical Techniques Lecture 5: Singular Value Decomposition SVD (1) EE3L1, slide 1, Version 4: 25-Sep-02 Motivation for SVD (1) SVD = Singular Value Decomposition Consider the system

### MetropoGIS: A City Modeling System DI Dr. Konrad KARNER, DI Andreas KLAUS, DI Joachim BAUER, DI Christopher ZACH

MetropoGIS: A City Modeling System DI Dr. Konrad KARNER, DI Andreas KLAUS, DI Joachim BAUER, DI Christopher ZACH VRVis Research Center for Virtual Reality and Visualization, Virtual Habitat, Inffeldgasse

### 3-D Point Cloud Generation from Rigid and Flexible Stereo Vision Systems

3-D Point Cloud Generation from Rigid and Flexible Stereo Vision Systems Nathaniel J. Short Thesis submitted to the faculty of the Virginia Polytechnic Institute and State University in partial fulfillment

### Lecture 2: Homogeneous Coordinates, Lines and Conics

Lecture 2: Homogeneous Coordinates, Lines and Conics 1 Homogeneous Coordinates In Lecture 1 we derived the camera equations λx = P X, (1) where x = (x 1, x 2, 1), X = (X 1, X 2, X 3, 1) and P is a 3 4

### CSCI 445 Amin Atrash. Ultrasound, Laser and Vision Sensors. Introduction to Robotics L. Itti & M. J. Mataric

Introduction to Robotics CSCI 445 Amin Atrash Ultrasound, Laser and Vision Sensors Today s Lecture Outline Ultrasound (sonar) Laser range-finders (ladar, not lidar) Vision Stereo vision Ultrasound/Sonar

### Conics with a Common Axis of Symmetry: Properties and Applications to Camera Calibration

Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Conics with a Common Axis of Symmetry: Properties and Applications to Camera Calibration Zijian Zhao UJF-Grenoble

### Auto-calibration of Pan-Tilt Cameras including Radial Distortion and Zoom

Auto-calibration of Pan-Tilt Cameras including Radial Distortion and Zoom Ricardo Galego, Alexandre Bernardino, and José Gaspar Institute for Systems and Robotics, Instituto Superior Técnico / UTL, Lisboa,

### Video Transmission Robot Vision Motion-Noise Separation

Video Transmission Robot Vision Motion-Noise Separation A. Ritchie, J. Conradi, A. Prevot, and E. A. fantis Image Processing Computer Graphics Intelligent Systems Lab. (ICIS) School of Computer Science

### Solution Guide III-C. 3D Vision. Building Vision for Business. MVTec Software GmbH

Solution Guide III-C 3D Vision MVTec Software GmbH Building Vision for Business Machine vision in 3D world coordinates, Version 10.0.4 All rights reserved. No part of this publication may be reproduced,

### Image Projection. Goal: Introduce the basic concepts and mathematics for image projection.

Image Projection Goal: Introduce the basic concepts and mathematics for image projection. Motivation: The mathematics of image projection allow us to answer two questions: Given a 3D scene, how does it

### Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision Information Meeting 15 Dec 2009 by Marianna Pronobis Content of the Meeting 1. Motivation & Objectives 2. What the course will be about (Stefan) 3. Content of

### Image formation. Image formation. Pinhole camera. Pinhole camera. Image formation. Matlab tutorial. Physical parameters of image formation

Image formation Image formation Matlab tutorial How are objects in the world captured in an image? Tuesday, Sept 2 Physical parameters of image formation Geometric Type of projection Camera pose Optical

### Motion Capture from Dynamic Orthographic Cameras

Motion Capture from Dynamic Orthographic Cameras Magnus Burenius, Josephine Sullivan, Stefan Carlsson KH CSC/CVAP S- 44 Stockholm, Sweden {burenius,sullivan,stefanc}@csc.kth.se Abstract We present an extension

### Feature Matching and RANSAC

Feature Matching and RANSAC Krister Parmstrand with a lot of slides stolen from Steve Seitz and Rick Szeliski 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Feature matching? SIFT keypoints

### Single Image 3D Reconstruction of Ball Motion and Spin From Motion Blur

Single Image 3D Reconstruction of Ball Motion and Spin From Motion Blur An Experiment in Motion from Blur Giacomo Boracchi, Vincenzo Caglioti, Alessandro Giusti Objective From a single image, reconstruct:

### Metrics on SO(3) and Inverse Kinematics

Mathematical Foundations of Computer Graphics and Vision Metrics on SO(3) and Inverse Kinematics Luca Ballan Institute of Visual Computing Optimization on Manifolds Descent approach d is a ascent direction

### Motion Capture from Dynamic Orthographic Cameras

Motion Capture from Dynamic Orthographic Cameras Magnus Burenius, Josephine Sullivan, Stefan Carlsson KH CSC/CVAP S- 44 Stockholm, Sweden {burenius,sullivan,stefanc}@csc.kth.se Abstract We present an extension

### An Iterative Image Registration Technique with an Application to Stereo Vision

An Iterative Image Registration Technique with an Application to Stereo Vision Bruce D. Lucas Takeo Kanade Computer Science Department Carnegie-Mellon University Pittsburgh, Pennsylvania 15213 Abstract

### Calibrating Pan-Tilt Cameras with Telephoto Lenses

Calibrating Pan-Tilt Cameras with Telephoto Lenses Xinyu Huang, Jizhou Gao, and Ruigang Yang Graphics and Vision Technology Lab (GRAVITY) Center for Visualization and Virtual Environments University of

### Projective Geometry: A Short Introduction. Lecture Notes Edmond Boyer

Projective Geometry: A Short Introduction Lecture Notes Edmond Boyer Contents 1 Introduction 2 11 Objective 2 12 Historical Background 3 13 Bibliography 4 2 Projective Spaces 5 21 Definitions 5 22 Properties

### Robust Pedestrian Detection and Tracking From A Moving Vehicle

Robust Pedestrian Detection and Tracking From A Moving Vehicle Nguyen Xuan Tuong a, Thomas Müller b and Alois Knoll b a Department of Computer Engineering, Nanyang Technological University, Singapore b

### P3.5P: Pose Estimation with Unknown Focal Length

: Pose Estimation with Unknown Focal Length Changchang Wu Google Inc. Abstract It is well known that the problem of camera pose estimation with unknown focal length has 7 degrees of freedom. Since each

### Symmetry-Based Photo Editing

Symmetry-Based Photo Editing Kun Huang Wei Hong Yi Ma Department of Electrical and Computer Engineering Coordinated Science Laboratory University of Illinois at Urbana-Champaign kunhuang, weihong, yima

### Improvements in Real-Time Correlation-Based Stereo Vision

Improvements in Real-Time Correlation-Based Stereo Vision Heiko Hirschmüller Centre for Computational Intelligence, De Montfort University, Leicester, LE1 9BH, UK, hhm@dmu.ac.uk. Abstract A stereo vision

### CS 534: Computer Vision 3D Model-based recognition

CS 534: Computer Vision 3D Model-based recognition Ahmed Elgammal Dept of Computer Science CS 534 3D Model-based Vision - 1 High Level Vision Object Recognition: What it means? Two main recognition tasks:!

### Whiteboard It! Convert Whiteboard Content into an Electronic Document

Whiteboard It! Convert Whiteboard Content into an Electronic Document Zhengyou Zhang Li-wei He Microsoft Research Email: zhang@microsoft.com, lhe@microsoft.com Aug. 12, 2002 Abstract This ongoing project

### An Image-Based System for Urban Navigation

An Image-Based System for Urban Navigation Duncan Robertson and Roberto Cipolla Cambridge University Engineering Department Trumpington Street, Cambridge, CB2 1PZ, UK Abstract We describe the prototype

### THE problem of visual servoing guiding a robot using

582 IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 13, NO. 4, AUGUST 1997 A Modular System for Robust Positioning Using Feedback from Stereo Vision Gregory D. Hager, Member, IEEE Abstract This paper

### An Efficient Solution to the Five-Point Relative Pose Problem

An Efficient Solution to the Five-Point Relative Pose Problem David Nistér Sarnoff Corporation CN5300, Princeton, NJ 08530 dnister@sarnoff.com Abstract An efficient algorithmic solution to the classical

### Fast field survey with a smartphone

Fast field survey with a smartphone A. Masiero F. Fissore, F. Pirotti, A. Guarnieri, A. Vettore CIRGEO Interdept. Research Center of Geomatics University of Padova Italy cirgeo@unipd.it 1 Mobile Mapping

### Lecture 12: Cameras and Geometry. CAP 5415 Fall 2010

Lecture 12: Cameras and Geometry CAP 5415 Fall 2010 The midterm What does the response of a derivative filter tell me about whether there is an edge or not? Things aren't working Did you look at the filters?

### 2D Geometric Transformations. COMP 770 Fall 2011

2D Geometric Transformations COMP 770 Fall 2011 1 A little quick math background Notation for sets, functions, mappings Linear transformations Matrices Matrix-vector multiplication Matrix-matrix multiplication

### Non-Metric Image-Based Rendering for Video Stabilization

Non-Metric Image-Based Rendering for Video Stabilization Chris Buehler Michael Bosse Leonard McMillan MIT Laboratory for Computer Science Cambridge, MA 02139 Abstract We consider the problem of video stabilization:

### Computational Optical Imaging - Optique Numerique. -- Active Light 3D--

Computational Optical Imaging - Optique Numerique -- Active Light 3D-- Autumn 2015 Ivo Ihrke Overview 3D scanning techniques Laser triangulation Structured light Photometric stereo Time-of-Flight Transient

### Image Formation and Camera Calibration

EECS 432-Advanced Computer Vision Notes Series 2 Image Formation and Camera Calibration Ying Wu Electrical Engineering & Computer Science Northwestern University Evanston, IL 60208 yingwu@ecenorthwesternedu

### Robust Panoramic Image Stitching

Robust Panoramic Image Stitching CS231A Final Report Harrison Chau Department of Aeronautics and Astronautics Stanford University Stanford, CA, USA hwchau@stanford.edu Robert Karol Department of Aeronautics

### V-PITS : VIDEO BASED PHONOMICROSURGERY INSTRUMENT TRACKING SYSTEM. Ketan Surender

V-PITS : VIDEO BASED PHONOMICROSURGERY INSTRUMENT TRACKING SYSTEM by Ketan Surender A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science (Electrical Engineering)

### A Flexible New Technique for Camera Calibration

A Flexible New Technique for Camera Calibration Zhengyou Zhang December 2, 1998 (updated on December 14, 1998) (updated on March 25, 1999) (updated on Aug. 1, 22; a typo in Appendix B) (updated on Aug.

### Computer Vision: Filtering

Computer Vision: Filtering Raquel Urtasun TTI Chicago Jan 10, 2013 Raquel Urtasun (TTI-C) Computer Vision Jan 10, 2013 1 / 82 Today s lecture... Image formation Image Filtering Raquel Urtasun (TTI-C) Computer

### A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation

A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation René Vidal,2 and Yi Ma 3 Center for Imaging Science, Johns Hopkins University, Baltimore MD 228, USA 2 National ICT Australia, Canberra ACT

### Automatic Labeling of Lane Markings for Autonomous Vehicles

Automatic Labeling of Lane Markings for Autonomous Vehicles Jeffrey Kiske Stanford University 450 Serra Mall, Stanford, CA 94305 jkiske@stanford.edu 1. Introduction As autonomous vehicles become more popular,

### 3D CAPTURE OF ARCHAEOLOGY AND ARCHITECTURE WITH A HAND-HELD CAMERA

3D CAPTURE OF ARCHAEOLOGY AND ARCHITECTURE WITH A HAND-HELD CAMERA Marc Pollefeys a, Luc Van Gool b, Maarten Vergauwen b, Kurt Cornelis b, Frank Verbiest b, Jan Tops b a Dept. of Computer Science, University

### 3D Vehicle Extraction and Tracking from Multiple Viewpoints for Traffic Monitoring by using Probability Fusion Map

Electronic Letters on Computer Vision and Image Analysis 7(2):110-119, 2008 3D Vehicle Extraction and Tracking from Multiple Viewpoints for Traffic Monitoring by using Probability Fusion Map Zhencheng