# Two-Frame Motion Estimation Based on Polynomial Expansion

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Two-Frame Motion Estimation Based on Polynomial Expansion Gunnar Farnebäck Computer Vision Laboratory, Linköping University, SE Linköping, Sweden Abstract. This paper presents a novel two-frame motion estimation algorithm. The first step is to approximate each neighborhood of both frames by quadratic polynomials, which can be done efficiently using the polynomial expansion transform. From observing how an exact polynomial transforms under translation a method to estimate displacement fields from the polynomial expansion coefficients is derived and after a series of refinements leads to a robust algorithm. Evaluation on the Yosemite sequence shows good results. 1 Introduction In previous work we have developed orientation tensor based algorithms to estimate motion, with excellent results both with respect to accuracy and speed [1, 2]. A limitation of those, however, is that the estimation of the spatiotemporal orientation tensors requires the motion field to be temporally consistent. This is often the case but turned out to be a problem in the WITAS project [3], where image sequences are obtained by a helicopter-mounted camera. Due to high frequency vibrations from the helicopter affecting the camera system, there are large, quickly varying, and difficult to predict displacements between successive frames. A natural solution is to estimate the motion, or displacement, field from only two frames and try to compensate for the background motion. This paper presents a novel method to estimate displacement. It is related to our orientation tensor methods in that the first processing step, a signal transform called polynomial expansion, is common. Naturally this is only done spatially now, instead of spatiotemporally. Another common theme is the inclusion of parametric motion models in the algorithms. 2 Polynomial Expansion The idea of polynomial expansion is to approximate some neighborhood of each pixel with a polynomial. Here we are only interested in quadratic polynomials, giving us the local signal model, expressed in a local coordinate system, f(x) x T Ax + b T x + c, (1)

2 where A is a symmetric matrix, b a vector and c a scalar. The coefficients are estimated from a weighted least squares fit to the signal values in the neighborhood. The weighting has two components called certainty and applicability. These terms are the same as in normalized convolution [4 6], which polynomial expansion is based on. The certainty is coupled to the signal values in the neighborhood. For example it is generally a good idea to set the certainty to zero outside the image. Then neighborhood points outside the image have no impact on the coefficient estimation. The applicability determines the relative weight of points in the neighborhood based on their position in the neighborhood. Typically one wants to give most weight to the center point and let the weights decrease radially. The width of the applicability determines the scale of the structures which will be captured by the expansion coefficients. While this may sound computationally very demanding it turns out that it can be implemented efficiently by a hierarchical scheme of separable convolutions. Further details on this can be found in [6]. 3 Displacement Estimation Since the result of polynomial expansion is that each neighborhood is approximated by a polynomial, we start by analyzing what happens if a polynomial undergoes an ideal translation. Consider the exact quadratic polynomial f 1 (x) =x T A 1 x + b T 1 x + c 1 (2) and construct a new signal f 2 by a global displacement by d, f 2 (x) =f 1 (x d) =(x d) T A 1 (x d)+b T 1 (x d)+c 1 = x T A 1 x +(b 1 2A 1 d) T x + d T A 1 d b T 1 d + c 1 = x T A 2 x + b T 2 x + c 2. (3) Equating the coefficients in the quadratic polynomials yields A 2 = A 1, (4) b 2 = b 1 2A 1 d, (5) c 2 = d T A 1 d b T 1 d + c 1. (6) The key observation is that by equation (5) we can solve for the translation d, at least if A 1 is non-singular, 2A 1 d = (b 2 b 1 ), (7) d = 1 2 A 1 1 (b 2 b 1 ). (8) We note that this observation holds for any signal dimensionality.

3 3.1 Practical Considerations Obviously the assumptions about an entire signal being a single polynomial and a global translation relating the two signals are quite unrealistic. Still the basic relation (7) can be used for real signals, although errors are introduced when the assumptions are relaxed. The question is whether these errors can be kept small enough to give a useful algorithm. To begin with we replace the global polynomial in equation (2) with local polynomial approximations. Thus we start by doing a polynomial expansion of both images, giving us expansion coefficients A 1 (x), b 1 (x), and c 1 (x) for the first image and A 2 (x), b 2 (x), and c 2 (x) for the second image. Ideally this should give A 1 = A 2 according to equation (4) but in practice we have to settle for the approximation A(x) = A 1(x)+A 2 (x). (9) 2 We also introduce b(x) = 1 2 (b 2(x) b 1 (x)) (10) to obtain the primary constraint A(x)d(x) = b(x), (11) where d(x) indicates that we have also replaced the global displacement in equation (3) with a spatially varying displacement field. 3.2 Estimation Over a Neighborhood In principle equation (11) can be solved pointwise, but the results turn out to be too noisy. Instead we make the assumption that the displacement field is only slowly varying, so that we can integrate information over a neighborhood of each pixel. Thus we try to find d(x) satisfying (11) as well as possible over a neighborhood I of x, or more formally minimizing w( x) A(x + x)d(x) b(x + x) 2, (12) x I where we let w( x) be a weight function for the points in the neighborhood. The minimum is obtained for d(x) =( wa T A) 1 wa T b, (13) where we have dropped some indexing to make the expression more readable. The minimum value is given by ) e(x) =( w b T b d(x) T wa T b. (14) In practical terms this means that we compute A T A, A T b, and b T b pointwise and average these with w before we solve for the displacement. The

4 minimum value e(x) can be used as a reversed confidence value, with small numbers indicating high confidence. The solution given by (13) exists and is unique unless the whole neighborhood is exposed to the aperture problem. Sometimes it is useful to add a certainty weight c(x + x) to (12). This is most easily handled by scaling A and b accordingly. 3.3 Parameterized Displacement Fields We can improve robustness if the displacement field can be parameterized according to some motion model. This is straightforward for motion models which are linear in their parameters, like the affine motion model or the eight parameter model. We derive this for the eight parameter model in 2D, d x (x, y) =a 1 + a 2 x + a 3 y + a 7 x 2 + a 8 xy, d y (x, y) =a 4 + a 5 x + a 6 y + a 7 xy + a 8 y 2. (15) We can rewrite this as d = Sp, (16) ( ) 1 xy000x S = 2 xy 0001xyxyy 2, (17) p = ( ) T a 1 a 2 a 3 a 4 a 5 a 6 a 7 a 8. (18) Inserting into (12) we obtain the weighted least squares problem w i A i S i p b i 2, (19) i where we use i to index the coordinates in a neighborhood. The solution is ( ) 1 p = w i S T i A T i A i S i i i w i S T i A T i b i. (20) We notice that like before we can compute S T A T AS and S T A T b pointwise and then average these with w. Naturally (20) reduces to (13) for the constant motion model. 3.4 Incorporating A Priori Knowledge A principal problem with the method so far is that we assume that the local polynomials at the same coordinates in the two signals are identical except for a displacement. Since the polynomial expansions are local models these will vary spatially, introducing errors in the constraints (11). For small displacements this is not too serious, but with larger displacements the problem increases. Fortunately we are not restricted to comparing two polynomials at the same coordinate. If we have a priori knowledge about the displacement field, we can compare

5 the polynomial at x in the first signal to the polynomial at x+ d(x) in the second signal, where d(x) is the a priori displacement field rounded to integer values. Then we effectively only need to estimate the relative displacement between the real value and the rounded a priori estimate, which hopefully is smaller. This observation is included in the algorithm by replacing equations (9) and (10) by where A(x) = A 1(x)+A 2 ( x), 2 (21) b(x) = 1 2 (b 2( x) b 1 (x)) + A(x) d(x), (22) x = x + d(x). (23) The first two terms in b are involved in computing the remaining displacement while the last term adds back the rounded a priori displacement. We can see that for d identically zero, these equations revert to (9) and (10), as would be expected. 3.5 Iterative and Multi-scale Displacement Estimation A consequence of the inclusion of an a priori displacement field in the algorithm is that we can close the loop and iterate. A better a priori estimate means a smaller relative displacement, which in turn improves the chances for a good displacement estimate. We consider two different approaches, iterative displacement estimation and multi-scale displacement estimation. In both the approaches we iterate with the estimated displacements from one step used as a priori displacement in the next step. The a priori displacement field in the first step would usually be set to zero, unless actual knowledge about it is available. In the first approach the same polynomial expansion coefficients are used in all iterations and need only be computed once. The weak spot of this approach is in the first iteration. If the displacements (relative the a priori displacements) are too large, the output displacements cannot be expected to be improvements and iterating will be meaningless. The problem of too large displacements can be reduced by doing the analysis at a coarser scale. This means that we use a larger applicability for the polynomial expansion and/or lowpass filter the signal first. The effect is that the estimation algorithm can handle larger displacements but at the same time the accuracy decreases. This observation points to the second approach with multiple scales. Start at a coarse scale to get a rough but reasonable displacement estimate and propagate this through finer scales to obtain increasingly more accurate estimates. A drawback is that we need to recompute the polynomial expansion coefficients for each scale, but this cost can be reduced by subsampling between scales.

6 4 Experimental Results The algorithm has been implemented in Matlab, with certain parts in the form of C mex files. Source code for the implementation is available from ~ gf. The algorithm has been evaluated on a commonly used test sequence with known velocity field, Lynn Quam s Yosemite sequence [7], figure 1. This synthetic sequence was generated with the help of a digital terrain map and therefore has a motion field with depth variation and discontinuities at occlusion boundaries. The accuracy of the velocity estimates has been measured using the average spatiotemporal angular error, arccos(ˆv T estˆv true ) [8]. The sky region is excluded from the error analysis because the variations in the cloud textures induce an image flow that is quite different from the ground truth values computed solely from the camera motion. We have estimated the displacement from the center frame and the frame before. The averaging over neighborhoods is done using a Gaussian weighting function (w in equation (19)) with standard deviation 6. The polynomial expansion is done with an Gaussian applicability with standard deviation 1.5. In order to reduce the errors near the borders, the polynomial expansions have been computed with certainty set to zero off the border. Additionally pixels close to the borders have been given a reduced weight (see section 3.2) because the expansion coefficients still can be assumed to be less reliable there. The constant and affine motion models have been used with a single iteration and with three iterations at the same scale. The results and a comparison with other methods can be found in table 1. Clearly this algorithm cannot compete with the most accurate ones, but that is to be expected since those take advantage of the spatio-temporal consistency over several frames. Still these results are good for a two-frame algorithm. A more thorough evaluation of the algorithm can be found in [6]. The main weakness of the algorithm is the assumption of a slowly varying displacement field, causing discontinuities to be smoothed out. This can be solved by combining the algorithm with a simultaneous segmentation procedure, e.g. the one used in [2]. Acknowledgements The work presented in this paper was supported by WITAS, the Wallenberg laboratory on Information Technology and Autonomous Systems, which is gratefully acknowledged. References 1. Farnebäck, G.: Fast and Accurate Motion Estimation using Orientation Tensors and Parametric Motion Models. In: Proceedings of 15th International Conference on Pattern Recognition. Volume 1., Barcelona, Spain, IAPR (2000)

7 Fig. 1. One frame of the Yosemite sequence and the corresponding true velocity field (subsampled). Table 1. Comparison with other methods, Yosemite sequence. The sky region is excluded for all results. Technique Average Standard Density error deviation Lucas & Kanade [9] % Uras et al. [10] % Fleet & Jepson [11] % Black & Anandan [12] % Szeliski & Coughlan [13] % Black & Jepson [14] % Ju et al. [15] % Karlholm [16] % Lai & Vemuri [17] % Bab-Hadiashar & Suter [18] % Mémin & Pérez [19] % Farnebäck, constant motion [1, 6] % Farnebäck, affine motion [1, 6] % Farnebäck, segmentation [2, 6] % Constant motion, 1 iteration % Constant motion, 3 iterations % Affine motion, 1 iteration % Affine motion, 3 iterations %

8 2. Farnebäck, G.: Very High Accuracy Velocity Estimation using Orientation Tensors, Parametric Motion, and Simultaneous Segmentation of the Motion Field. In: Proceedings of the Eighth IEEE International Conference on Computer Vision. Volume I., Vancouver, Canada (2001) URL: 4. Knutsson, H., Westin, C.F.: Normalized and Differential Convolution: Methods for Interpolation and Filtering of Incomplete and Uncertain Data. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York City, USA, IEEE (1993) Westin, C.F.: A Tensor Framework for Multidimensional Signal Processing. PhD thesis, Linköping University, Sweden, SE Linköping, Sweden (1994) Dissertation No 348, ISBN Farnebäck, G.: Polynomial Expansion for Orientation and Motion Estimation. PhD thesis, Linköping University, Sweden, SE Linköping, Sweden (2002) Dissertation No 790, ISBN Heeger, D.J.: Model for the extraction of image flow. J. Opt. Soc. Am. A 4 (1987) Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. of Computer Vision 12 (1994) Lucas, B., Kanade, T.: An Iterative Image Registration Technique with Applications to Stereo Vision. In: Proc. Darpa IU Workshop. (1981) Uras, S., Girosi, F., Verri, A., Torre, V.: A computational approach to motion perception. Biological Cybernetics (1988) Fleet, D.J., Jepson, A.D.: Computation of Component Image Velocity from Local Phase Information. Int. Journal of Computer Vision 5 (1990) Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding 63 (1996) Szeliski, R., Coughlan, J.: Hierarchical spline-based image registration. In: Proc. IEEE Conference on Computer Vision Pattern Recognition, Seattle, Washington (1994) Black, M.J., Jepson, A.: Estimating optical flow in segmented images using variable-order parametric models with local deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 18 (1996) Ju, S.X., Black, M.J., Jepson, A.D.: Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In: Proceedings CVPR 96, IEEE (1996) Karlholm, J.: Local Signal Models for Image Sequence Analysis. PhD thesis, Linköping University, Sweden, SE Linköping, Sweden (1998) Dissertation No 536, ISBN Lai, S.H., Vemuri, B.C.: Reliable and efficient computation of optical flow. International Journal of Computer Vision 29 (1998) Bab-Hadiashar, A., Suter, D.: Robust optic flow computation. International Journal of Computer Vision 29 (1998) Mémin, E., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision 46 (2002)

### Optical Flow as a property of moving objects used for their registration

Optical Flow as a property of moving objects used for their registration Wolfgang Schulz Computer Vision Course Project York University Email:wschulz@cs.yorku.ca 1. Introduction A soccer game is a real

### ROBUST VEHICLE TRACKING IN VIDEO IMAGES BEING TAKEN FROM A HELICOPTER

ROBUST VEHICLE TRACKING IN VIDEO IMAGES BEING TAKEN FROM A HELICOPTER Fatemeh Karimi Nejadasl, Ben G.H. Gorte, and Serge P. Hoogendoorn Institute of Earth Observation and Space System, Delft University

### Feature Tracking and Optical Flow

02/09/12 Feature Tracking and Optical Flow Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve

### Rotational Symmetries a Quick Tutorial

Rotational Symmetries a Quick Tutorial Björn Johansson May 4, 2001 This is a printable version of a tutorial in HTML format. The tutorial may be modified at any time as will this version. The latest version

### EECS 556 Image Processing W 09. Interpolation. Interpolation techniques B splines

EECS 556 Image Processing W 09 Interpolation Interpolation techniques B splines What is image processing? Image processing is the application of 2D signal processing methods to images Image representation

Linköping University Electronic Press Book Chapter Multi-modal Image Registration Using Polynomial Expansion and Mutual Information Daniel Forsberg, Gunnar Farnebäck, Hans Knutsson and Carl-Fredrik Westin

### An Iterative Image Registration Technique with an Application to Stereo Vision

An Iterative Image Registration Technique with an Application to Stereo Vision Bruce D. Lucas Takeo Kanade Computer Science Department Carnegie-Mellon University Pittsburgh, Pennsylvania 15213 Abstract

### Problem definition: optical flow

Motion Estimation http://www.sandlotscience.com/distortions/breathing_objects.htm http://www.sandlotscience.com/ambiguous/barberpole.htm Why estimate motion? Lots of uses Track object behavior Correct

### A Learning Based Method for Super-Resolution of Low Resolution Images

A Learning Based Method for Super-Resolution of Low Resolution Images Emre Ugur June 1, 2004 emre.ugur@ceng.metu.edu.tr Abstract The main objective of this project is the study of a learning based method

### Edge tracking for motion segmentation and depth ordering

Edge tracking for motion segmentation and depth ordering P. Smith, T. Drummond and R. Cipolla Department of Engineering University of Cambridge Cambridge CB2 1PZ,UK {pas1001 twd20 cipolla}@eng.cam.ac.uk

### Efficient Belief Propagation for Early Vision

Efficient Belief Propagation for Early Vision Pedro F. Felzenszwalb and Daniel P. Huttenlocher Department of Computer Science, Cornell University {pff,dph}@cs.cornell.edu Abstract Markov random field models

### Practical Tour of Visual tracking. David Fleet and Allan Jepson January, 2006

Practical Tour of Visual tracking David Fleet and Allan Jepson January, 2006 Designing a Visual Tracker: What is the state? pose and motion (position, velocity, acceleration, ) shape (size, deformation,

### Depth from a single camera

Depth from a single camera Fundamental Matrix Essential Matrix Active Sensing Methods School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie 1 1 Geometry of two

### Tracking of Small Unmanned Aerial Vehicles

Tracking of Small Unmanned Aerial Vehicles Steven Krukowski Adrien Perkins Aeronautics and Astronautics Stanford University Stanford, CA 94305 Email: spk170@stanford.edu Aeronautics and Astronautics Stanford

### High Quality Image Magnification using Cross-Scale Self-Similarity

High Quality Image Magnification using Cross-Scale Self-Similarity André Gooßen 1, Arne Ehlers 1, Thomas Pralow 2, Rolf-Rainer Grigat 1 1 Vision Systems, Hamburg University of Technology, D-21079 Hamburg

Tracking Moving Objects In Video Sequences Yiwei Wang, Robert E. Van Dyck, and John F. Doherty Department of Electrical Engineering The Pennsylvania State University University Park, PA16802 Abstract{Object

### Low-resolution Character Recognition by Video-based Super-resolution

2009 10th International Conference on Document Analysis and Recognition Low-resolution Character Recognition by Video-based Super-resolution Ataru Ohkura 1, Daisuke Deguchi 1, Tomokazu Takahashi 2, Ichiro

### Accurate and robust image superresolution by neural processing of local image representations

Accurate and robust image superresolution by neural processing of local image representations Carlos Miravet 1,2 and Francisco B. Rodríguez 1 1 Grupo de Neurocomputación Biológica (GNB), Escuela Politécnica

### A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow

, pp.233-237 http://dx.doi.org/10.14257/astl.2014.51.53 A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow Giwoo Kim 1, Hye-Youn Lim 1 and Dae-Seong Kang 1, 1 Department of electronices

### Determining optimal window size for texture feature extraction methods

IX Spanish Symposium on Pattern Recognition and Image Analysis, Castellon, Spain, May 2001, vol.2, 237-242, ISBN: 84-8021-351-5. Determining optimal window size for texture feature extraction methods Domènec

### Introduction to the Finite Element Method (FEM)

Introduction to the Finite Element Method (FEM) ecture First and Second Order One Dimensional Shape Functions Dr. J. Dean Discretisation Consider the temperature distribution along the one-dimensional

### 1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style

Factorisation 1.5 Introduction In Block 4 we showed the way in which brackets were removed from algebraic expressions. Factorisation, which can be considered as the reverse of this process, is dealt with

### Vision based Vehicle Tracking using a high angle camera

Vision based Vehicle Tracking using a high angle camera Raúl Ignacio Ramos García Dule Shu gramos@clemson.edu dshu@clemson.edu Abstract A vehicle tracking and grouping algorithm is presented in this work

### Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition

Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition 1. Image Pre-Processing - Pixel Brightness Transformation - Geometric Transformation - Image Denoising 1 1. Image Pre-Processing

### Face Model Fitting on Low Resolution Images

Face Model Fitting on Low Resolution Images Xiaoming Liu Peter H. Tu Frederick W. Wheeler Visualization and Computer Vision Lab General Electric Global Research Center Niskayuna, NY, 1239, USA {liux,tu,wheeler}@research.ge.com

### Cloud tracking with optical flow for short-term solar forecasting

Cloud tracking with optical flow for short-term solar forecasting Philip Wood-Bradley, José Zapata, John Pye Solar Thermal Group, Australian National University, Canberra, Australia Corresponding author:

### Illumination-Invariant Tracking via Graph Cuts

Illumination-Invariant Tracking via Graph Cuts Daniel Freedman and Matthew W. Turek Computer Science Department, Rensselaer Polytechnic Institute, Troy, NY 12180 Abstract Illumination changes are a ubiquitous

### Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

### the points are called control points approximating curve

Chapter 4 Spline Curves A spline curve is a mathematical representation for which it is easy to build an interface that will allow a user to design and control the shape of complex curves and surfaces.

### A Reliability Point and Kalman Filter-based Vehicle Tracking Technique

A Reliability Point and Kalman Filter-based Vehicle Tracing Technique Soo Siang Teoh and Thomas Bräunl Abstract This paper introduces a technique for tracing the movement of vehicles in consecutive video

### Lecture 19 Camera Matrices and Calibration

Lecture 19 Camera Matrices and Calibration Project Suggestions Texture Synthesis for In-Painting Section 10.5.1 in Szeliski Text Project Suggestions Image Stitching (Chapter 9) Face Recognition Chapter

### COMPUTING CLOUD MOTION USING A CORRELATION RELAXATION ALGORITHM Improving Estimation by Exploiting Problem Knowledge Q. X. WU

COMPUTING CLOUD MOTION USING A CORRELATION RELAXATION ALGORITHM Improving Estimation by Exploiting Problem Knowledge Q. X. WU Image Processing Group, Landcare Research New Zealand P.O. Box 38491, Wellington

### Mean-Shift Tracking with Random Sampling

1 Mean-Shift Tracking with Random Sampling Alex Po Leung, Shaogang Gong Department of Computer Science Queen Mary, University of London, London, E1 4NS Abstract In this work, boosting the efficiency of

### Unwrap Mosaics: A new representation for video editing

Unwrap Mosaics: A new representation for video editing Alex Rav-Acha, Pushmeet Kohli, Carsten Rother, Andrew Fitzgibbon Microsoft Research SIGGRAPH 2008 http://research.microsoft.com/en-us/um/cambridge/projects/visionimagevideoediting/unwrap/

### Fast field survey with a smartphone

Fast field survey with a smartphone A. Masiero F. Fissore, F. Pirotti, A. Guarnieri, A. Vettore CIRGEO Interdept. Research Center of Geomatics University of Padova Italy cirgeo@unipd.it 1 Mobile Mapping

### Color Segmentation Based Depth Image Filtering

Color Segmentation Based Depth Image Filtering Michael Schmeing and Xiaoyi Jiang Department of Computer Science, University of Münster Einsteinstraße 62, 48149 Münster, Germany, {m.schmeing xjiang}@uni-muenster.de

### C4 Computer Vision. 4 Lectures Michaelmas Term Tutorial Sheet Prof A. Zisserman. fundamental matrix, recovering ego-motion, applications.

C4 Computer Vision 4 Lectures Michaelmas Term 2004 1 Tutorial Sheet Prof A. Zisserman Overview Lecture 1: Stereo Reconstruction I: epipolar geometry, fundamental matrix. Lecture 2: Stereo Reconstruction

### Colorado School of Mines Computer Vision Professor William Hoff

Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Introduction to 2 What is? A process that produces from images of the external world a description

### Spatio-Temporal Segmentation of Video Data

M.I.T. Media Laboratory Vision and Modeling Group, Technical Report No. 262, February 1994. Appears in Proceedings of the SPIE: Image and Video Processing II, vol. 2182, San Jose, February 1994. Spatio-Temporal

### A PHOTOGRAMMETRIC APPRAOCH FOR AUTOMATIC TRAFFIC ASSESSMENT USING CONVENTIONAL CCTV CAMERA

A PHOTOGRAMMETRIC APPRAOCH FOR AUTOMATIC TRAFFIC ASSESSMENT USING CONVENTIONAL CCTV CAMERA N. Zarrinpanjeh a, F. Dadrassjavan b, H. Fattahi c * a Islamic Azad University of Qazvin - nzarrin@qiau.ac.ir

### Image Segmentation and Registration

Image Segmentation and Registration Dr. Christine Tanner (tanner@vision.ee.ethz.ch) Computer Vision Laboratory, ETH Zürich Dr. Verena Kaynig, Machine Learning Laboratory, ETH Zürich Outline Segmentation

### 2.2 Creaseness operator

2.2. Creaseness operator 31 2.2 Creaseness operator Antonio López, a member of our group, has studied for his PhD dissertation the differential operators described in this section [72]. He has compared

### Eyeglass Localization for Low Resolution Images

Eyeglass Localization for Low Resolution Images Earl Arvin Calapatia 1 1 De La Salle University 1 earl_calapatia@dlsu.ph Abstract: Facial data is a necessity in facial image processing technologies. In

### MRI Composing for Whole Body Imaging

MRI Composing for Whole Body Imaging B. Glocker 1,3, C. Wachinger 1, J. Zeltner 2, N. Paragios 3, N. Komodakis 4, M. Hansen 5, N. Navab 1 1 Computer Aided Medical Procedures (CAMP), TU München, Germany

### Epipolar Geometry. Readings: See Sections 10.1 and 15.6 of Forsyth and Ponce. Right Image. Left Image. e(p ) Epipolar Lines. e(q ) q R.

Epipolar Geometry We consider two perspective images of a scene as taken from a stereo pair of cameras (or equivalently, assume the scene is rigid and imaged with a single camera from two different locations).

### OBJECT TRACKING USING LOG-POLAR TRANSFORMATION

OBJECT TRACKING USING LOG-POLAR TRANSFORMATION A Thesis Submitted to the Gradual Faculty of the Louisiana State University and Agricultural and Mechanical College in partial fulfillment of the requirements

### Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/

Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters

### Automatic Labeling of Lane Markings for Autonomous Vehicles

Automatic Labeling of Lane Markings for Autonomous Vehicles Jeffrey Kiske Stanford University 450 Serra Mall, Stanford, CA 94305 jkiske@stanford.edu 1. Introduction As autonomous vehicles become more popular,

### EFFICIENT VEHICLE TRACKING AND CLASSIFICATION FOR AN AUTOMATED TRAFFIC SURVEILLANCE SYSTEM

EFFICIENT VEHICLE TRACKING AND CLASSIFICATION FOR AN AUTOMATED TRAFFIC SURVEILLANCE SYSTEM Amol Ambardekar, Mircea Nicolescu, and George Bebis Department of Computer Science and Engineering University

### Parameter Estimation for Bingham Models

Dr. Volker Schulz, Dmitriy Logashenko Parameter Estimation for Bingham Models supported by BMBF Parameter Estimation for Bingham Models Industrial application of ceramic pastes Material laws for Bingham

### Simultaneous Gamma Correction and Registration in the Frequency Domain

Simultaneous Gamma Correction and Registration in the Frequency Domain Alexander Wong a28wong@uwaterloo.ca William Bishop wdbishop@uwaterloo.ca Department of Electrical and Computer Engineering University

### Latest Results on High-Resolution Reconstruction from Video Sequences

Latest Results on High-Resolution Reconstruction from Video Sequences S. Lertrattanapanich and N. K. Bose The Spatial and Temporal Signal Processing Center Department of Electrical Engineering The Pennsylvania

### VEHICLE TRACKING USING ACOUSTIC AND VIDEO SENSORS

VEHICLE TRACKING USING ACOUSTIC AND VIDEO SENSORS Aswin C Sankaranayanan, Qinfen Zheng, Rama Chellappa University of Maryland College Park, MD - 277 {aswch, qinfen, rama}@cfar.umd.edu Volkan Cevher, James

### Non-rigid point set registration: Coherent Point Drift

Non-rigid point set registration: Coherent Point Drift Andriy Myronenko Xubo Song Miguel Á. Carreira-Perpiñán Department of Computer Science and Electrical Engineering OGI School of Science and Engineering

### Blind Deconvolution of Barcodes via Dictionary Analysis and Wiener Filter of Barcode Subsections

Blind Deconvolution of Barcodes via Dictionary Analysis and Wiener Filter of Barcode Subsections Maximilian Hung, Bohyun B. Kim, Xiling Zhang August 17, 2013 Abstract While current systems already provide

### Digital Imaging and Multimedia. Filters. Ahmed Elgammal Dept. of Computer Science Rutgers University

Digital Imaging and Multimedia Filters Ahmed Elgammal Dept. of Computer Science Rutgers University Outlines What are Filters Linear Filters Convolution operation Properties of Linear Filters Application

### since by using a computer we are limited to the use of elementary arithmetic operations

> 4. Interpolation and Approximation Most functions cannot be evaluated exactly: x, e x, ln x, trigonometric functions since by using a computer we are limited to the use of elementary arithmetic operations

### Path Tracking for a Miniature Robot

Path Tracking for a Miniature Robot By Martin Lundgren Excerpt from Master s thesis 003 Supervisor: Thomas Hellström Department of Computing Science Umeå University Sweden 1 Path Tracking Path tracking

### Lagrange Interpolation is a method of fitting an equation to a set of points that functions well when there are few points given.

Polynomials (Ch.1) Study Guide by BS, JL, AZ, CC, SH, HL Lagrange Interpolation is a method of fitting an equation to a set of points that functions well when there are few points given. Sasha s method

One-Dimensional Processing for Edge Detection using Hilbert Transform P. Kiran Kumar, Sukhendu Das and B. Yegnanarayana Department of Computer Science and Engineering Indian Institute of Technology, Madras

### Subspace Analysis and Optimization for AAM Based Face Alignment

Subspace Analysis and Optimization for AAM Based Face Alignment Ming Zhao Chun Chen College of Computer Science Zhejiang University Hangzhou, 310027, P.R.China zhaoming1999@zju.edu.cn Stan Z. Li Microsoft

### Object tracking & Motion detection in video sequences

Introduction Object tracking & Motion detection in video sequences Recomended link: http://cmp.felk.cvut.cz/~hlavac/teachpresen/17compvision3d/41imagemotion.pdf 1 2 DYNAMIC SCENE ANALYSIS The input to

### Shape from Single Stripe Pattern Illumination

Published in Luc Van Gool (Editor): Pattern Recognition, Lecture Notes in Computer Science 2449, Springer 2002, page 240-247. Shape from Single Stripe Pattern Illumination S. Winkelbach and F. M. Wahl

### An Energy-Based Vehicle Tracking System using Principal Component Analysis and Unsupervised ART Network

Proceedings of the 8th WSEAS Int. Conf. on ARTIFICIAL INTELLIGENCE, KNOWLEDGE ENGINEERING & DATA BASES (AIKED '9) ISSN: 179-519 435 ISBN: 978-96-474-51-2 An Energy-Based Vehicle Tracking System using Principal

### Building an Advanced Invariant Real-Time Human Tracking System

UDC 004.41 Building an Advanced Invariant Real-Time Human Tracking System Fayez Idris 1, Mazen Abu_Zaher 2, Rashad J. Rasras 3, and Ibrahiem M. M. El Emary 4 1 School of Informatics and Computing, German-Jordanian

### General Framework for an Iterative Solution of Ax b. Jacobi s Method

2.6 Iterative Solutions of Linear Systems 143 2.6 Iterative Solutions of Linear Systems Consistent linear systems in real life are solved in one of two ways: by direct calculation (using a matrix factorization,

### Optical Flow. Thomas Pock

Optical Flow Thomas Pock 1 Optical Flow (II) Content Global approaches (Horn-Schunck, TV-L1) Coarse-to-fine warping 2 The Horn and Schunck (HS) Method 3 The Horn and Schunck (HS) Method Global energy to

### Automotive Applications of 3D Laser Scanning Introduction

Automotive Applications of 3D Laser Scanning Kyle Johnston, Ph.D., Metron Systems, Inc. 34935 SE Douglas Street, Suite 110, Snoqualmie, WA 98065 425-396-5577, www.metronsys.com 2002 Metron Systems, Inc

### Bayesian Image Super-Resolution

Bayesian Image Super-Resolution Michael E. Tipping and Christopher M. Bishop Microsoft Research, Cambridge, U.K..................................................................... Published as: Bayesian

### Automatic Traffic Estimation Using Image Processing

Automatic Traffic Estimation Using Image Processing Pejman Niksaz Science &Research Branch, Azad University of Yazd, Iran Pezhman_1366@yahoo.com Abstract As we know the population of city and number of

### CHAPTER 6 TEXTURE ANIMATION

CHAPTER 6 TEXTURE ANIMATION 6.1. INTRODUCTION Animation is the creating of a timed sequence or series of graphic images or frames together to give the appearance of continuous movement. A collection of

### Introduction to acoustic imaging

Introduction to acoustic imaging Contents 1 Propagation of acoustic waves 3 1.1 Wave types.......................................... 3 1.2 Mathematical formulation.................................. 4 1.3

### Real-time Traffic Congestion Detection Based on Video Analysis

Journal of Information & Computational Science 9: 10 (2012) 2907 2914 Available at http://www.joics.com Real-time Traffic Congestion Detection Based on Video Analysis Shan Hu a,, Jiansheng Wu a, Ling Xu

### Deinterlacing/interpolation of TV signals. Elham Shahinfard Advisors: Prof. M. Ahmadi Prof. M. Sid-Ahmed

Deinterlacing/interpolation of TV signals Elham Shahinfard Advisors: Prof. M. Ahmadi Prof. M. Sid-Ahmed Outline A Review of some terminologies Converting from NTSC to HDTV; What changes need to be considered?

### Face detection is a process of localizing and extracting the face region from the

Chapter 4 FACE NORMALIZATION 4.1 INTRODUCTION Face detection is a process of localizing and extracting the face region from the background. The detected face varies in rotation, brightness, size, etc.

### Polynomials and the Fast Fourier Transform (FFT) Battle Plan

Polynomials and the Fast Fourier Transform (FFT) Algorithm Design and Analysis (Wee 7) 1 Polynomials Battle Plan Algorithms to add, multiply and evaluate polynomials Coefficient and point-value representation

### Introduction to Engineering System Dynamics

CHAPTER 0 Introduction to Engineering System Dynamics 0.1 INTRODUCTION The objective of an engineering analysis of a dynamic system is prediction of its behaviour or performance. Real dynamic systems are

### Detecting and positioning overtaking vehicles using 1D optical flow

Detecting and positioning overtaking vehicles using 1D optical flow Daniel Hultqvist 1, Jacob Roll 1, Fredrik Svensson 1, Johan Dahlin 2, and Thomas B. Schön 3 Abstract We are concerned with the problem

### Computer Vision: Filtering

Computer Vision: Filtering Raquel Urtasun TTI Chicago Jan 10, 2013 Raquel Urtasun (TTI-C) Computer Vision Jan 10, 2013 1 / 82 Today s lecture... Image formation Image Filtering Raquel Urtasun (TTI-C) Computer

### OpenFOAM Optimization Tools

OpenFOAM Optimization Tools Henrik Rusche and Aleks Jemcov h.rusche@wikki-gmbh.de and a.jemcov@wikki.co.uk Wikki, Germany and United Kingdom OpenFOAM Optimization Tools p. 1 Agenda Objective Review optimisation

### Optical Flow. Shenlong Wang CSC2541 Course Presentation Feb 2, 2016

Optical Flow Shenlong Wang CSC2541 Course Presentation Feb 2, 2016 Outline Introduction Variation Models Feature Matching Methods End-to-end Learning based Methods Discussion Optical Flow Goal: Pixel motion

### 10.3 POWER METHOD FOR APPROXIMATING EIGENVALUES

55 CHAPTER NUMERICAL METHODS. POWER METHOD FOR APPROXIMATING EIGENVALUES In Chapter 7 we saw that the eigenvalues of an n n matrix A are obtained by solving its characteristic equation n c n n c n n...

### Window-based, discontinuity preserving stereo

indow-based, discontinuity preserving stereo Motilal Agrawal SRI International 333 Ravenswood Ave. Menlo Park, CA 94025 agrawal@ai.sri.com Larry S. Davis Univ. of Maryland Dept. of Computer Science College

### Template-based Eye and Mouth Detection for 3D Video Conferencing

Template-based Eye and Mouth Detection for 3D Video Conferencing Jürgen Rurainsky and Peter Eisert Fraunhofer Institute for Telecommunications - Heinrich-Hertz-Institute, Image Processing Department, Einsteinufer

### Numerical methods for finding the roots of a function

Numerical methods for finding the roots of a function The roots of a function f (x) are defined as the values for which the value of the function becomes equal to zero. So, finding the roots of f (x) means

### Numerical Methods For Image Restoration

Numerical Methods For Image Restoration CIRAM Alessandro Lanza University of Bologna, Italy Faculty of Engineering CIRAM Outline 1. Image Restoration as an inverse problem 2. Image degradation models:

### Adaptive PIV with variable interrogation window size and shape

Adaptive PIV with variable interrogation window size and shape Bernhard Wieneke¹, Karsten Pfeiffer¹ 1: LaVision GmbH, Göttingen, bwieneke@lavision.de Abstract PIV processing requires a careful selection

### Static Environment Recognition Using Omni-camera from a Moving Vehicle

Static Environment Recognition Using Omni-camera from a Moving Vehicle Teruko Yata, Chuck Thorpe Frank Dellaert The Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213 USA College of Computing

### Jiří Matas. Hough Transform

Hough Transform Jiří Matas Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University, Prague Many slides thanks to Kristen Grauman and Bastian

### Implementation of Canny Edge Detector of color images on CELL/B.E. Architecture.

Implementation of Canny Edge Detector of color images on CELL/B.E. Architecture. Chirag Gupta,Sumod Mohan K cgupta@clemson.edu, sumodm@clemson.edu Abstract In this project we propose a method to improve

### Automatic parameter regulation for a tracking system with an auto-critical function

Automatic parameter regulation for a tracking system with an auto-critical function Daniela Hall INRIA Rhône-Alpes, St. Ismier, France Email: Daniela.Hall@inrialpes.fr Abstract In this article we propose

### SPINDLE ERROR MOVEMENTS MEASUREMENT ALGORITHM AND A NEW METHOD OF RESULTS ANALYSIS 1. INTRODUCTION

Journal of Machine Engineering, Vol. 15, No.1, 2015 machine tool accuracy, metrology, spindle error motions Krzysztof JEMIELNIAK 1* Jaroslaw CHRZANOWSKI 1 SPINDLE ERROR MOVEMENTS MEASUREMENT ALGORITHM

### Final Year Project Progress Report. Frequency-Domain Adaptive Filtering. Myles Friel. Supervisor: Dr.Edward Jones

Final Year Project Progress Report Frequency-Domain Adaptive Filtering Myles Friel 01510401 Supervisor: Dr.Edward Jones Abstract The Final Year Project is an important part of the final year of the Electronic

### CS229 Project Final Report. Sign Language Gesture Recognition with Unsupervised Feature Learning

CS229 Project Final Report Sign Language Gesture Recognition with Unsupervised Feature Learning Justin K. Chen, Debabrata Sengupta, Rukmani Ravi Sundaram 1. Introduction The problem we are investigating

### Analyzing Facial Expressions for Virtual Conferencing

IEEE Computer Graphics & Applications, pp. 70-78, September 1998. Analyzing Facial Expressions for Virtual Conferencing Peter Eisert and Bernd Girod Telecommunications Laboratory, University of Erlangen,

### 222 The International Arab Journal of Information Technology, Vol. 1, No. 2, July 2004 particular pixels. High pass filter and low pass filter are gen

The International Arab Journal of Information Technology, Vol. 1, No. 2, July 2004 221 A New Approach for Contrast Enhancement Using Sigmoid Function Naglaa Hassan 1&2 and Norio Akamatsu 1 1 Department

### A multi-scale approach to InSAR time series analysis

A multi-scale approach to InSAR time series analysis M. Simons, E. Hetland, P. Muse, Y. N. Lin & C. DiCaprio U Interferogram stack time A geophysical perspective on deformation tomography Examples: Long