Introduction to Inverse Problems (2 lectures)

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Introduction to Inverse Problems (2 lectures)"

Transcription

1 Introduction to Inverse Problems (2 lectures) Summary Direct and inverse problems Examples of direct (forward) problems Deterministic and statistical points of view Ill-posed and ill-conditioned problems An illustrative example: The deconvolution problem Truncated Fourier decomposition (TFD); Tikhonov regularization Generalized Tikhonov regularization; Bayesian perspective. Iterative optimization. IP, José Bioucas Dias, 2007, IST 1

2 Direct/Inverse problems Direct (forward) problem Causes Inverse problem Effects Example: Direct problem: the computation of the trajectories of bodies from the knowledge of the forces. Inverse problem: determination of the forces from the knowledge of the trajectories Newton solved the first direct/inverse problem: the determintion of the gravitation force from the Kepler laws describing the trajectories of planets 2

3 An example: a linear time invariant (LTI) system Direct problem: Fourier domain Inverse problem: Source of difficulties: is unbounded A perturbation on leads to a perturbation on given by high frequencies of the perturbation are amplified, degrading the estimate of f 3

4 Image deblurring Observation model in (linear) image restoration/reconstruction observed image noise Linear operator original image (e.g., blur, tomography, MRI,...) Goal: estimate f from g 4

5 Image deblurring via regularization original Blurred, 9x9 uniform restored 5

6 MRI example Hydrogen density 2D frequency samples (9.4%) 6

7 Compressed Sensing (sparse representation) Sparse vector f Random matrix Observed data y Compressed Sensing N=1000 M=

8 Classes of direct problems Deterministic observation mechanism Original data (image) Operator + Observed data (image) perturbation 8

9 Classes of direct problems (deterministic) Linear space-invariant imaging systems Blur (motion, out-of-focus, Diffraction-limited imaging atmospheric) Near field acoustic holography Channel equalization Parameter identification Linear space-variant imaging systems (first kind Fredholm equation) X-ray tomography MR imaging Radar imaging Sonar imaging Inverse diffraction Inverse source Linear regression 9

10 Classes of direct problems Statistical observation mechanism Original data (image) Observed data (image) Ex: Linear/nonlinear observations in additive Gaussian noise + 10

11 Classes of direct problems (statistic) Linear/nonlinear observation driven by non-additive noise Parameters of a distribution Random signal/ image Rayleigh noise in coherent imaging Poisson noise in photo-electric conversion SPET (single photon emission tomography) PET (positron emission tomography) Ex: Amplitude in a coherent imaging system (radar, ultrasound) Terrain reflectance Inphase/quadrature backscattered signal 11

12 Well-posed/ill-posed inverse problems [Hadamard, 1923] Definition: Let be a (possible nonlinear) operator The inverse problem of solving Hadamard sense if: is well-posed in the 1) A solution exists for any in the observed data space 2) The solution is unique 3) The inverse mapping is continuous An inverse problem that is not well-posed is termed ill-posed The operator A of an inverse well/ill-posed problem is termed well/ill-posed 12

13 Finite/Infinite dimensional linear operators Linear Operators: The linear inverse problem equivalently, is well-posed if 1) and 2) holds or, and If is finite-dimensional, the corresponding inverse problem is well-posed iif either one of the properties 1) and 2) holds Example: In infinite-dimensional spaces Consider A defined on, If a solution of exists, it is unique since However, there are elements not in Thus, A is ill-posed (point 1 of the Hadamard conditions does not hold) Stability is also lacking: Take Then, does not converge when 13

14 Ill-conditioned inverse problems lll-posed lll-conditioned Many well-posed inverse problems are ill-conditioned, in the sense that For linear operators (tight bound) 14

15 Example: Discrete deconvolution Cyclic convolution Matrix notation N-periodic funtions A is cyclic Toeplitz 15

16 Example: Discrete deconvolution 16

17 Eigen-decomposition of cyclic matrices (unitary) Eigenvector (Fourier) matrix Eigenvalue matrix (diagonal) is the DFT of at frequency 17

18 Example: Discrete deconvolution (geometric viewpoint) Action of A on f Note: 18

19 Example: Discrete deconvolution (inferring f) Assume that Then is invertible and Thus, assuming the direct model We have error 19

20 Example: cyclic convolution with a Gaussian kernel What went wrong? 20

21 Example: Discrete deconvolution (estimation error) Size of the error Assume that Thus Which is a set enclosed by an ellipsoid with radii 21

22 Example: Discrete deconvolution (estimation error) The estimation error is the vector The components satisfy 22

23 Cyclic convolution with a Gaussian kernel (cont.) (unit impulse function) Noise dominates at high frequencies and is amplified by 23

24 Example: Discrete deconvolution (A is ill-posed) Assume now that is not invertible and it may happen that i.e, some are zero Least-squares solution Projection error 24

25 Example: Discrete deconvolution (A is ill-posed) Least-squares approach Orthogonal components 25

26 Example: Discrete deconvolution (A is ill-posed) Invisible objects is the minimum norm solution (related to the Moore-Penrose inverse) 26

27 Example: Discrete deconvolution (Regularization) A is ill-conditioned A is ill-posed In both cases small eigenvalues are sources of instabilities Often, the smaller the eigenvalue the more oscilating the corresponding eigenvector (high frequences) Regularization by filtereing: shrink/threshold large values of i.e, multiply the eigenvalues by a regularizer filter such that as 27

28 Example: Discrete deconvolution Regularization by filtering (frequency multiplication time convolution) Such that 1) as 2 ) The larger eigenvalues are retained Truncated Fourier Decomposition (TFD) Tikhonov (Wiener) filter 28

29 Example: Discrete deconvolution (Regularization by filtering) TFD Tikhonov Tikhonov regularization Thus Solution of the variational problem 29

30 Example: Discrete deconvolution (1D example) Gaussian shaped of standard deviation = f g frequency

31 Example: Discrete deconvolution (1D example -TFD) f f f f 1 f f f f 1 f f 1 f f

32 Example: Discrete deconvolution (2D example-fd) uniform 32

33 Example: Discrete deconvolution (2D example-tfd) 33

34 Curing Ill-posed/Ill-conditioned inverse problems Golden rule for solving ill-posed/ill-conditioned inverse problems Search for solutions which: are compatible with the observed data satisfy additional constraints (a priori or prior information) coming from the (physics) problem 34

35 Generalized Tikhonov regularization Tikhonov and TFD regularization are not well suited to deal with data Nonhomogeneities, such as edges Generalized Tikhonov regularization Bayesian viewpoint Data Discrepancy term Penalty/ Regularization term Negative loglikelihood Negative logprior 35

36 Dominating approaches to regularization 1) 2) 3) 4) In given circumstances 2), 3), and 4) are equivalent 36

37 Example: Discrete deconvolution (Nonquadratic regularization) penalize oscillatory solutions discontinuity preserving (robust) regularization is nonconvex hard optimization problem non-discontinuity preserving regularization is convex treatable optimization problem 37

38 Optimization - Quadratic Linear system of equations Large systems require iterative methods - Non-quadratic and smooth Methods: Steepest descent, nonlinear conjugate gradient, Newton, trust regions, - Non-quadratic and nonsmooth Constrained optimization (Linear, quadratic, second-order cone programs) Methods: Iterative Shrinkage/Thesholding; Coordinate Subspace Optimization; forward-backward splitting; Primal-dual Newton Majorization Minimizaton (MM) class 38

39 Majorization Minorization (MM) Framework Let Majorization Minorization algorithm:...with equality if and only if Easy to prove monotonicity: Notes: should be easy to maximize EM is an algorithm of this type. 39

40 Example: Discrete deconvolution (1D example NQ Regula.) Tikhonov f f f f f f f f f f f f

41 Example: Discrete deconvolution (2D example-total Variation) Total variation regularization (TV) where TV regularizer penalizes highly oscilatory solutions, while it preserves the edges 41

42 Bibliography [Ch1. RB2, Ch1. L1] Important topics Euclidian and Hilbert spaces of functions [App. A, RB2] Linear operators in function spaces [App. B, RB2] Euclidian vector spaces and matrices [App. C, RB2] Properties of the DFT and the FFT algorithm [App. B, RB2] Matlab scripts TFD_regularization_1D.m TFD_regularization_2D.m TFD_Error_1D.m TV_regulatization_1D.m 42

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB. Sohail A. Dianat. Rochester Institute of Technology, New York, U.S.A. Eli S.

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB. Sohail A. Dianat. Rochester Institute of Technology, New York, U.S.A. Eli S. ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB Sohail A. Dianat Rochester Institute of Technology, New York, U.S.A. Eli S. Saber Rochester Institute of Technology, New York, U.S.A. (g) CRC Press Taylor

More information

Numerical Methods For Image Restoration

Numerical Methods For Image Restoration Numerical Methods For Image Restoration CIRAM Alessandro Lanza University of Bologna, Italy Faculty of Engineering CIRAM Outline 1. Image Restoration as an inverse problem 2. Image degradation models:

More information

Computational Optical Imaging - Optique Numerique. -- Deconvolution --

Computational Optical Imaging - Optique Numerique. -- Deconvolution -- Computational Optical Imaging - Optique Numerique -- Deconvolution -- Winter 2014 Ivo Ihrke Deconvolution Ivo Ihrke Outline Deconvolution Theory example 1D deconvolution Fourier method Algebraic method

More information

Another Example: the Hubble Space Telescope

Another Example: the Hubble Space Telescope 296 DIP Chapter and 2: Introduction and Integral Equations Motivation: Why Inverse Problems? A large-scale example, coming from a collaboration with Università degli Studi di Napoli Federico II in Naples.

More information

Sparse recovery and compressed sensing in inverse problems

Sparse recovery and compressed sensing in inverse problems Gerd Teschke (7. Juni 2010) 1/68 Sparse recovery and compressed sensing in inverse problems Gerd Teschke (joint work with Evelyn Herrholz) Institute for Computational Mathematics in Science and Technology

More information

Regularized Matrix Computations

Regularized Matrix Computations Regularized Matrix Computations Andrew E. Yagle Department of EECS, The University of Michigan, Ann Arbor, MI 489- Abstract We review the basic results on: () the singular value decomposition (SVD); ()

More information

Advanced Signal Processing and Digital Noise Reduction

Advanced Signal Processing and Digital Noise Reduction Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester New

More information

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti

More information

Eigenvalues of C SMS (16) 1 A. Eigenvalues of A (32) 1 A (16,32) 1 A

Eigenvalues of C SMS (16) 1 A. Eigenvalues of A (32) 1 A (16,32) 1 A Negative Results for Multilevel Preconditioners in Image Deblurring C. R. Vogel Department of Mathematical Sciences Montana State University Bozeman, MT 59717-0240 USA Abstract. A one-dimensional deconvolution

More information

COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16. Lecture 2: Linear Regression Gradient Descent Non-linear basis functions

COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16. Lecture 2: Linear Regression Gradient Descent Non-linear basis functions COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16 Lecture 2: Linear Regression Gradient Descent Non-linear basis functions LINEAR REGRESSION MOTIVATION Why Linear Regression? Regression

More information

Inverse problems. Nikolai Piskunov 2014. Regularization: Example Lecture 4

Inverse problems. Nikolai Piskunov 2014. Regularization: Example Lecture 4 Inverse problems Nikolai Piskunov 2014 Regularization: Example Lecture 4 Now that we know the theory, let s try an application: Problem: Optimal filtering of 1D and 2D data Solution: formulate an inverse

More information

AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS

AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS Revised Edition James Epperson Mathematical Reviews BICENTENNIAL 0, 1 8 0 7 z ewiley wu 2007 r71 BICENTENNIAL WILEY-INTERSCIENCE A John Wiley & Sons, Inc.,

More information

1. Introduction. Consider the computation of an approximate solution of the minimization problem

1. Introduction. Consider the computation of an approximate solution of the minimization problem A NEW TIKHONOV REGULARIZATION METHOD MARTIN FUHRY AND LOTHAR REICHEL Abstract. The numerical solution of linear discrete ill-posed problems typically requires regularization, i.e., replacement of the available

More information

Numerical Recipes in C

Numerical Recipes in C 2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Numerical Recipes in C The Art of Scientific Computing Second Edition

More information

Lecture 14. Point Spread Function (PSF)

Lecture 14. Point Spread Function (PSF) Lecture 14 Point Spread Function (PSF), Modulation Transfer Function (MTF), Signal-to-noise Ratio (SNR), Contrast-to-noise Ratio (CNR), and Receiver Operating Curves (ROC) Point Spread Function (PSF) Recollect

More information

P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition

P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition K. Osypov* (WesternGeco), D. Nichols (WesternGeco), M. Woodward (WesternGeco) & C.E. Yarman (WesternGeco) SUMMARY Tomographic

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

More information

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/ Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters

More information

Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

More information

Bindel, Fall 2012 Matrix Computations (CS 6210) Week 8: Friday, Oct 12

Bindel, Fall 2012 Matrix Computations (CS 6210) Week 8: Friday, Oct 12 Why eigenvalues? Week 8: Friday, Oct 12 I spend a lot of time thinking about eigenvalue problems. In part, this is because I look for problems that can be solved via eigenvalues. But I might have fewer

More information

Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

More information

Section for Cognitive Systems DTU Informatics, Technical University of Denmark

Section for Cognitive Systems DTU Informatics, Technical University of Denmark Transformation Invariant Sparse Coding Morten Mørup & Mikkel N Schmidt Morten Mørup & Mikkel N. Schmidt Section for Cognitive Systems DTU Informatics, Technical University of Denmark Redundancy Reduction

More information

ALGEBRAIC EIGENVALUE PROBLEM

ALGEBRAIC EIGENVALUE PROBLEM ALGEBRAIC EIGENVALUE PROBLEM BY J. H. WILKINSON, M.A. (Cantab.), Sc.D. Technische Universes! Dsrmstedt FACHBEREICH (NFORMATiK BIBL1OTHEK Sachgebieto:. Standort: CLARENDON PRESS OXFORD 1965 Contents 1.

More information

Modern methods for solving inverse problems

Modern methods for solving inverse problems Modern methods for solving inverse problems Jürgen Hesser Experimental Radiation Oncology University Medical Center Mannheim University of Heidelberg 1 Motivation Image Guided Radiation Therapy (IGRT)

More information

Contents. Gbur, Gregory J. Mathematical methods for optical physics and engineering digitalisiert durch: IDS Basel Bern

Contents. Gbur, Gregory J. Mathematical methods for optical physics and engineering digitalisiert durch: IDS Basel Bern Preface page xv 1 Vector algebra 1 1.1 Preliminaries 1 1.2 Coordinate System invariance 4 1.3 Vector multiplication 9 1.4 Useful products of vectors 12 1.5 Linear vector Spaces 13 1.6 Focus: periodic media

More information

STUDY GUIDE LINEAR ALGEBRA. David C. Lay University of Maryland College Park AND ITS APPLICATIONS THIRD EDITION UPDATE

STUDY GUIDE LINEAR ALGEBRA. David C. Lay University of Maryland College Park AND ITS APPLICATIONS THIRD EDITION UPDATE STUDY GUIDE LINEAR ALGEBRA AND ITS APPLICATIONS THIRD EDITION UPDATE David C. Lay University of Maryland College Park Copyright 2006 Pearson Addison-Wesley. All rights reserved. Reproduced by Pearson Addison-Wesley

More information

Chapter 3. Inverse Problems

Chapter 3. Inverse Problems 20 Chapter 3 Inverse Problems Mathematicians often cannot help but cringe when physicists are doing math, and even more so whenever physicists claim to be rigorous with their math. This chapter is not

More information

Blind Deconvolution of Corrupted Barcode Signals

Blind Deconvolution of Corrupted Barcode Signals Blind Deconvolution of Corrupted Barcode Signals Everardo Uribe and Yifan Zhang Advisors: Ernie Esser and Yifei Lou Interdisciplinary Computational and Applied Mathematics Program University of California,

More information

System Identification for Acoustic Comms.:

System Identification for Acoustic Comms.: System Identification for Acoustic Comms.: New Insights and Approaches for Tracking Sparse and Rapidly Fluctuating Channels Weichang Li and James Preisig Woods Hole Oceanographic Institution The demodulation

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

Numerical Methods I Eigenvalue Problems

Numerical Methods I Eigenvalue Problems Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001, Fall 2010 September 30th, 2010 A. Donev (Courant Institute)

More information

Linear Algebra. Introduction

Linear Algebra. Introduction Linear Algebra Caren Diefenderfer, Hollins University, chair David Hill, Temple University, lead writer Sheldon Axler, San Francisco State University Nancy Neudauer, Pacific University David Strong, Pepperdine

More information

Blind Deconvolution of Barcodes via Dictionary Analysis and Wiener Filter of Barcode Subsections

Blind Deconvolution of Barcodes via Dictionary Analysis and Wiener Filter of Barcode Subsections Blind Deconvolution of Barcodes via Dictionary Analysis and Wiener Filter of Barcode Subsections Maximilian Hung, Bohyun B. Kim, Xiling Zhang August 17, 2013 Abstract While current systems already provide

More information

2.5 Complex Eigenvalues

2.5 Complex Eigenvalues 1 25 Complex Eigenvalues Real Canonical Form A semisimple matrix with complex conjugate eigenvalues can be diagonalized using the procedure previously described However, the eigenvectors corresponding

More information

By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

By choosing to view this document, you agree to all provisions of the copyright laws protecting it. This material is posted here with permission of the IEEE Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services Internal

More information

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

More information

1646 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 6, NO. 12, DECEMBER 1997

1646 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 6, NO. 12, DECEMBER 1997 1646 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL 6, NO 12, DECEMBER 1997 Restoration of a Single Superresolution Image from Several Blurred, Noisy, and Undersampled Measured Images Michael Elad and Arie

More information

Latest Results on High-Resolution Reconstruction from Video Sequences

Latest Results on High-Resolution Reconstruction from Video Sequences Latest Results on High-Resolution Reconstruction from Video Sequences S. Lertrattanapanich and N. K. Bose The Spatial and Temporal Signal Processing Center Department of Electrical Engineering The Pennsylvania

More information

Need for Speed in MRI & Parallel Imaging I

Need for Speed in MRI & Parallel Imaging I G16.4428 Practical Magnetic Resonance Imaging II Sackler Institute of Biomedical Sciences New York University School of Medicine Need for Speed in MRI & Parallel Imaging I Ricardo Otazo, PhD ricardo.otazo@nyumc.org

More information

Change Point Estimation of Bar Code Signals

Change Point Estimation of Bar Code Signals 1 Change Point Estimation of Bar Code Signals Leming Qu, and Yi-Cheng Tu Abstract Existing methods for bar code signal reconstruction is based on either the local approach or the regularization approach

More information

Numerical Analysis Introduction. Student Audience. Prerequisites. Technology.

Numerical Analysis Introduction. Student Audience. Prerequisites. Technology. Numerical Analysis Douglas Faires, Youngstown State University, (Chair, 2012-2013) Elizabeth Yanik, Emporia State University, (Chair, 2013-2015) Graeme Fairweather, Executive Editor, Mathematical Reviews,

More information

Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

More information

Computational Methods CMSC/AMSC/MAPL 460. Eigenvalues and Eigenvectors. Ramani Duraiswami, Dept. of Computer Science

Computational Methods CMSC/AMSC/MAPL 460. Eigenvalues and Eigenvectors. Ramani Duraiswami, Dept. of Computer Science Computational Methods CMSC/AMSC/MAPL 460 Eigenvalues and Eigenvectors Ramani Duraiswami, Dept. of Computer Science Eigen Values of a Matrix Definition: A N N matrix A has an eigenvector x (non-zero) with

More information

Principal Component Analysis Application to images

Principal Component Analysis Application to images Principal Component Analysis Application to images Václav Hlaváč Czech Technical University in Prague Faculty of Electrical Engineering, Department of Cybernetics Center for Machine Perception http://cmp.felk.cvut.cz/

More information

Signal Processing for Intelligent Sensor Systems with MATLAB 9

Signal Processing for Intelligent Sensor Systems with MATLAB 9 Signal Processing for Intelligent Sensor Systems with MATLAB 9 Second Edition David С Swanson @ CRC Press Taylor & Francis Gro jroup Boca Raton London New York CRC Press is an imprint of the Taylor & Francis

More information

Compressive Acquisition of Sparse Deflectometric Maps

Compressive Acquisition of Sparse Deflectometric Maps Compressive Acquisition of Sparse Deflectometric Maps Prasad Sudhakar*, Laurent Jacques*, Adriana Gonzalez Gonzalez* Xavier Dubois +, Philippe Antoine +, Luc Joannes + *: Louvain University (UCL), Louvain-la-Neuve,

More information

ORTHOGONAL PROJECTION REGULARIZATION OPERATORS

ORTHOGONAL PROJECTION REGULARIZATION OPERATORS ORTHOGONAL PROJECTION REGULARIZATION OPERATORS S. MORIGI, L. REICHEL, AND F. SGALLARI Abstract. Tikhonov regularization often is applied with a finite difference regularization operator that approximates

More information

Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition

Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition 710.080 2VO 710.081 1KU 1 Optical Flow (I) Content Introduction Local approach (Lucas Kanade) Global approaches (Horn-Schunck,

More information

Digital Image Processing

Digital Image Processing Digital Image Processing Using MATLAB Second Edition Rafael C. Gonzalez University of Tennessee Richard E. Woods MedData Interactive Steven L. Eddins The MathWorks, Inc. Gatesmark Publishing A Division

More information

International Journal of Advanced Research in Computer Science and Software Engineering

International Journal of Advanced Research in Computer Science and Software Engineering Volume 4, Issue 8, August 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Comparative Study

More information

Lecture 9: Continuous

Lecture 9: Continuous CSC2515 Fall 2007 Introduction to Machine Learning Lecture 9: Continuous Latent Variable Models 1 Example: continuous underlying variables What are the intrinsic latent dimensions in these two datasets?

More information

PTE505: Inverse Modeling for Subsurface Flow Data Integration (3 Units)

PTE505: Inverse Modeling for Subsurface Flow Data Integration (3 Units) PTE505: Inverse Modeling for Subsurface Flow Data Integration (3 Units) Instructor: Behnam Jafarpour, Mork Family Department of Chemical Engineering and Material Science Petroleum Engineering, HED 313,

More information

Curve Fitting. Next: Numerical Differentiation and Integration Up: Numerical Analysis for Chemical Previous: Optimization.

Curve Fitting. Next: Numerical Differentiation and Integration Up: Numerical Analysis for Chemical Previous: Optimization. Next: Numerical Differentiation and Integration Up: Numerical Analysis for Chemical Previous: Optimization Subsections Least-Squares Regression Linear Regression General Linear Least-Squares Nonlinear

More information

Linear Algebra Review. Vectors

Linear Algebra Review. Vectors Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

More information

New Method for Deconvolving Adaptive Optics Images

New Method for Deconvolving Adaptive Optics Images New Method for Deconvolving Adaptive Optics Images or: (All you ever wanted to know about...) Myopic maximum-likelihood deconvolution with a space-varying kernel Ralf Flicker, François Rigaut Gemini Observatory,

More information

Dual Methods for Total Variation-Based Image Restoration

Dual Methods for Total Variation-Based Image Restoration Dual Methods for Total Variation-Based Image Restoration Jamylle Carter Institute for Mathematics and its Applications University of Minnesota, Twin Cities Ph.D. (Mathematics), UCLA, 2001 Advisor: Tony

More information

The Power Method for Eigenvalues and Eigenvectors

The Power Method for Eigenvalues and Eigenvectors Numerical Analysis Massoud Malek The Power Method for Eigenvalues and Eigenvectors The spectrum of a square matrix A, denoted by σ(a) is the set of all eigenvalues of A. The spectral radius of A, denoted

More information

Linear Regression CS434. Supervised learning

Linear Regression CS434. Supervised learning Linear Regression CS434 A regression problem We want to learn to predict a person s height based on his/her knee height and/or arm span This is useful for patients who are bed bound and cannot stand to

More information

Principles of Digital Communication

Principles of Digital Communication Principles of Digital Communication Robert G. Gallager January 5, 2008 ii Preface: introduction and objectives The digital communication industry is an enormous and rapidly growing industry, roughly comparable

More information

Journal of Computational and Applied Mathematics

Journal of Computational and Applied Mathematics Journal of Computational and Applied Mathematics 226 (2009) 92 102 Contents lists available at ScienceDirect Journal of Computational and Applied Mathematics journal homepage: www.elsevier.com/locate/cam

More information

Advanced Computational Techniques in Engineering

Advanced Computational Techniques in Engineering Course Details Course Code: 5MWCI432 Level: M Tech CSE Semester: Semester 4 Instructor: Prof Dr S P Ghrera Advanced Computational Techniques in Engineering Outline Course Description: Data Distributions,

More information

Computational Methods. Least Squares Approximation/Optimization

Computational Methods. Least Squares Approximation/Optimization Computational Methods Least Squares Approximation/Optimization Manfred Huber 2010 1 Least Squares Least squares methods are aimed at finding approximate solutions when no precise solution exists Find the

More information

Optical Flow. Thomas Pock

Optical Flow. Thomas Pock Optical Flow Thomas Pock 1 Optical Flow (II) Content Global approaches (Horn-Schunck, TV-L1) Coarse-to-fine warping 2 The Horn and Schunck (HS) Method 3 The Horn and Schunck (HS) Method Global energy to

More information

Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

More information

Numerical Recipes in C++

Numerical Recipes in C++ Numerical Recipes in C++ The Art of Scientific Computing Second Edition William H. Press Los Alamos National Laboratory Saul A. Teukolsky Department of Physics, Cornell University William T. Vetterling

More information

Nonlinear Iterative Partial Least Squares Method

Nonlinear Iterative Partial Least Squares Method Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., Richard-Plouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments for

More information

Vector Derivatives, Gradients, and Generalized Gradient Descent Algorithms

Vector Derivatives, Gradients, and Generalized Gradient Descent Algorithms Vector Derivatives, Gradients, and Generalized Gradient Descent Algorithms ECE 275A Statistical Parameter Estimation Ken Kreutz-Delgado ECE Department, UC San Diego November 1, 2013 Ken Kreutz-Delgado

More information

Linear Threshold Units

Linear Threshold Units Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

More information

Signal Detection. Outline. Detection Theory. Example Applications of Detection Theory

Signal Detection. Outline. Detection Theory. Example Applications of Detection Theory Outline Signal Detection M. Sami Fadali Professor of lectrical ngineering University of Nevada, Reno Hypothesis testing. Neyman-Pearson (NP) detector for a known signal in white Gaussian noise (WGN). Matched

More information

Numerical Methods I Solving Linear Systems: Sparse Matrices, Iterative Methods and Non-Square Systems

Numerical Methods I Solving Linear Systems: Sparse Matrices, Iterative Methods and Non-Square Systems Numerical Methods I Solving Linear Systems: Sparse Matrices, Iterative Methods and Non-Square Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001,

More information

CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen

CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 3: DATA TRANSFORMATION AND DIMENSIONALITY REDUCTION Chapter 3: Data Preprocessing Data Preprocessing: An Overview Data Quality Major

More information

LECTURE NOTES: FINITE ELEMENT METHOD

LECTURE NOTES: FINITE ELEMENT METHOD LECTURE NOTES: FINITE ELEMENT METHOD AXEL MÅLQVIST. Motivation The finite element method has two main strengths... Geometry. Very complex geometries can be used. This is probably the main reason why finite

More information

Review Jeopardy. Blue vs. Orange. Review Jeopardy

Review Jeopardy. Blue vs. Orange. Review Jeopardy Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round $200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?

More information

Inverse Problems and Regularization An Introduction

Inverse Problems and Regularization An Introduction Inverse Problems and Regularization An Introduction Stefan Kindermann Industrial Mathematics Institute University of Linz, Austria What are Inverse Problems? One possible definition [Engl, Hanke, Neubauer

More information

NMR Measurement of T1-T2 Spectra with Partial Measurements using Compressive Sensing

NMR Measurement of T1-T2 Spectra with Partial Measurements using Compressive Sensing NMR Measurement of T1-T2 Spectra with Partial Measurements using Compressive Sensing Alex Cloninger Norbert Wiener Center Department of Mathematics University of Maryland, College Park http://www.norbertwiener.umd.edu

More information

Mean value theorem, Taylors Theorem, Maxima and Minima.

Mean value theorem, Taylors Theorem, Maxima and Minima. MA 001 Preparatory Mathematics I. Complex numbers as ordered pairs. Argand s diagram. Triangle inequality. De Moivre s Theorem. Algebra: Quadratic equations and express-ions. Permutations and Combinations.

More information

Applied Linear Algebra I Review page 1

Applied Linear Algebra I Review page 1 Applied Linear Algebra Review 1 I. Determinants A. Definition of a determinant 1. Using sum a. Permutations i. Sign of a permutation ii. Cycle 2. Uniqueness of the determinant function in terms of properties

More information

Understanding and Applying Kalman Filtering

Understanding and Applying Kalman Filtering Understanding and Applying Kalman Filtering Lindsay Kleeman Department of Electrical and Computer Systems Engineering Monash University, Clayton 1 Introduction Objectives: 1. Provide a basic understanding

More information

4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac.

4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac. 4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac.uk 1 1 Outline The LMS algorithm Overview of LMS issues

More information

Total Variation Regularization in PET Reconstruction

Total Variation Regularization in PET Reconstruction Total ariation Regularization in PET Reconstruction Milán Magdics, Balázs Tóth, Balázs Kovács, and László Szirmay-Kalos BME IIT, Hungary magdics@iit.bme.hu Abstract. Positron Emission Tomography reconstruction

More information

2094 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER Information-Theoretic Image Formation. (Invited Paper)

2094 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER Information-Theoretic Image Formation. (Invited Paper) 2094 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 Information-Theoretic Image Formation Joseph A. O Sullivan, Senior Member, IEEE, Richard E. Blahut, Fellow, IEEE, and Donald L.

More information

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB Sohail A. Dianat Rochester Institute of Technology, New York, U.S.A. Eli S. Saber Rochester Institute

More information

WAVES AND FIELDS IN INHOMOGENEOUS MEDIA

WAVES AND FIELDS IN INHOMOGENEOUS MEDIA WAVES AND FIELDS IN INHOMOGENEOUS MEDIA WENG CHO CHEW UNIVERSITY OF ILLINOIS URBANA-CHAMPAIGN IEEE PRESS Series on Electromagnetic Waves Donald G. Dudley, Series Editor IEEE Antennas and Propagation Society,

More information

THE PROBLEM OF finding localized energy solutions

THE PROBLEM OF finding localized energy solutions 600 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 3, MARCH 1997 Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Re-weighted Minimum Norm Algorithm Irina F. Gorodnitsky, Member, IEEE,

More information

Mean-Shift Tracking with Random Sampling

Mean-Shift Tracking with Random Sampling 1 Mean-Shift Tracking with Random Sampling Alex Po Leung, Shaogang Gong Department of Computer Science Queen Mary, University of London, London, E1 4NS Abstract In this work, boosting the efficiency of

More information

Lecture 5: Variants of the LMS algorithm

Lecture 5: Variants of the LMS algorithm 1 Standard LMS Algorithm FIR filters: Lecture 5: Variants of the LMS algorithm y(n) = w 0 (n)u(n)+w 1 (n)u(n 1) +...+ w M 1 (n)u(n M +1) = M 1 k=0 w k (n)u(n k) =w(n) T u(n), Error between filter output

More information

Cortical Source Localization of Human Scalp EEG. Kaushik Majumdar Indian Statistical Institute Bangalore Center

Cortical Source Localization of Human Scalp EEG. Kaushik Majumdar Indian Statistical Institute Bangalore Center Cortical Source Localization of Human Scalp EEG Kaushik Majumdar Indian Statistical Institute Bangalore Center Cortical Basis of Scalp EEG Baillet et al, IEEE Sig Proc Mag, Nov 2001, p 16 Mountcastle,

More information

Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

Lecture Topic: Low-Rank Approximations

Lecture Topic: Low-Rank Approximations Lecture Topic: Low-Rank Approximations Low-Rank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original

More information

Finite Dimensional Hilbert Spaces and Linear Inverse Problems

Finite Dimensional Hilbert Spaces and Linear Inverse Problems Finite Dimensional Hilbert Spaces and Linear Inverse Problems ECE 174 Lecture Supplement Spring 2009 Ken Kreutz-Delgado Electrical and Computer Engineering Jacobs School of Engineering University of California,

More information

arxiv:math/ v2 [math.fa] 2 Nov 2003

arxiv:math/ v2 [math.fa] 2 Nov 2003 An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Ingrid Daubechies Michel Defrise Christine De Mol February 1, 2008 arxiv:math/0307152v2 [math.fa] 2 Nov 2003 Abstract

More information

Discrete Fourier Series & Discrete Fourier Transform Chapter Intended Learning Outcomes

Discrete Fourier Series & Discrete Fourier Transform Chapter Intended Learning Outcomes Discrete Fourier Series & Discrete Fourier Transform Chapter Intended Learning Outcomes (i) Understanding the relationships between the transform, discrete-time Fourier transform (DTFT), discrete Fourier

More information

Introduction and message of the book

Introduction and message of the book 1 Introduction and message of the book 1.1 Why polynomial optimization? Consider the global optimization problem: P : for some feasible set f := inf x { f(x) : x K } (1.1) K := { x R n : g j (x) 0, j =

More information

Bayesian Image Super-Resolution

Bayesian Image Super-Resolution Bayesian Image Super-Resolution Michael E. Tipping and Christopher M. Bishop Microsoft Research, Cambridge, U.K..................................................................... Published as: Bayesian

More information

Chapter 6. Orthogonality

Chapter 6. Orthogonality 6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

More information

168 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 1, JANUARY Yonina C. Eldar, Member, IEEE, Aharon Ben-Tal, and Arkadi Nemirovski

168 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 1, JANUARY Yonina C. Eldar, Member, IEEE, Aharon Ben-Tal, and Arkadi Nemirovski 168 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 1, JANUARY 2005 Robust Mean-Squared Error Estimation in the Presence of Model Uncertainties Yonina C. Eldar, Member, IEEE, Aharon Ben-Tal, and Arkadi

More information

Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication

Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication Time Domain and Frequency Domain Techniques For Multi Shaker Time Waveform Replication Thomas Reilly Data Physics Corporation 1741 Technology Drive, Suite 260 San Jose, CA 95110 (408) 216-8440 This paper

More information

Outline. Random Variables. Examples. Random Variable

Outline. Random Variables. Examples. Random Variable Outline Random Variables M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Random variables. CDF and pdf. Joint random variables. Correlated, independent, orthogonal. Correlation,

More information

ADVANCED ALGORITHMS FOR EQUALIZATION ON ADSL CHANNEL

ADVANCED ALGORITHMS FOR EQUALIZATION ON ADSL CHANNEL ADVANCED ALGORITHMS FOR EQUALIZATION ON ADSL CHANNEL T. Mazanec Institute of Information Theory and Automation, Dept. of Signal Processing Abstract Digital subscriber line modems equalize response of line

More information

The Image Deblurring Problem

The Image Deblurring Problem page 1 Chapter 1 The Image Deblurring Problem You cannot depend on your eyes when your imagination is out of focus. Mark Twain When we use a camera, we want the recorded image to be a faithful representation

More information