Another Example: the Hubble Space Telescope

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Another Example: the Hubble Space Telescope"

Transcription

1 296 DIP Chapter and 2: Introduction and Integral Equations Motivation: Why Inverse Problems? A large-scale example, coming from a collaboration with Università degli Studi di Napoli Federico II in Naples. From measurements of the magnetic field above Vesuvius, we determine the activity inside the volcano. Measurements on the surface Reconstruction inside the volcano 296 DIP Chapter and 2: Introduction and Integral Equations 2 Another Example: the Hubble Space Telescope For several years, the HST produced blurred images.

2 296 DIP Chapter and 2: Introduction and Integral Equations 3 Inverse Problems... typically arise when one wants to compute information about some interior properties using exterior measurements. Inverse Problem One of these is known Input System Known but with errors Output 296 DIP Chapter and 2: Introduction and Integral Equations 4 Inverse Problems: Examples A quite generic formulation: input system dω = output Ω Image restoration scenery lens image Tomography X-ray source object damping Seismology seismic wave layers reflections

3 296 DIP Chapter and 2: Introduction and Integral Equations 5 Discrete Ill-Posed Problems Our generic ill-posed problem: A Fredholm integral equation of the first kind K(s, t) f dt = g(s), s. Definition of a discrete ill-posed problem (DIP):. a square or over-determined system of linear algebraic equations A x = b or min x A x b 2 2. whose coefficient matrix A has a huge condition number, and 3. comes from the discretization of an inverse/ill-posed problem. 296 DIP Chapter and 2: Introduction and Integral Equations 6 Computational Issues The plots below show solutions x to the DIP A x = b..5 x 6 Gaussian elimination 2.5 Truncated SVD TSVD solution Exact solution Standard numerical methods (x = A\b) produce useless results. Specialized methods (this course) produce reasonable results.

4 296 DIP Chapter and 2: Introduction and Integral Equations 7 The Mechanisms of Ill-Conditioned Problems Consider a linear system with coefficient matrix and right-hand side ( ). A =.7., b =.25 = A There is no vector x such that A x = b. The least squares solution, which solves the problem is given by x LS = ( ) min x A x b 2, A x LS b 2 =.22. Far from exact solution (, ) T yet the residual is small. 296 DIP Chapter and 2: Introduction and Integral Equations 8 Other Solutions with Small Residual Two other solutions with a small residual are ( ).65 x = A x b 2 =.3 ( ) x = A x b 2 = All the solutions x LS, x and x have small residuals, yet they are far from the exact solution! The matrix A is ill conditioned. Small perturbations of the data (here: b) can lead to large perturbations of the solution. A small residual does not imply a good solution. (All this is well known stuff from matrix computations.)

5 296 DIP Chapter and 2: Introduction and Integral Equations 9 Stabilization! It turns out that we can modify the problem such that the solution is more stable, i.e., less sensitive to perturbations. Example: enforce an upper bound on the solution norm x 2 : min x A x b 2 subject to x 2 δ. The solution x δ depends in a nonlinear way on δ: ( ) ( ).8.84 x. =, x =.5.54 ( ) ( ) x.385 =, x = By supplying the correct additional information we can compute a good approximate solution DIP Chapter and 2: Introduction and Integral Equations Inverse Problems Ill-Conditioned Problems Whenever we solve an inverse problem on a computer, we face difficulties because the computational problems are ill conditioned. The purpose of my lectures are:. To explain why ill-conditioned computations always arise when solving inverse problems. 2. To explain the fundamental mechanisms underlying the ill conditioning. 3. To explain how we can modify the problem in order to stabilize the solution. 4. To show how this can be done efficiently on a computer. Regularization methods is at the heart of all this.

6 296 DIP Chapter and 2: Introduction and Integral Equations Inverse Problems are Ill-Posed Problems Hadamard s definition of a well-posed problem (early 2th century):. the problem must have a solution, 2. the solution must be unique, and 3. it must depend continuously on data and parameters. If the problem violates any of these requirements, it is ill posed. Condition can be fixed by reformulating/redefining the solution. Condition 2 can be fixed by additional requirements to the solution, e.g., that of minimum norm. Condition 3 is harder to fix because it implies that arbitrarily small perturbations of data and parameters can produce arbitrarily large perturbations of the solution. 296 DIP Chapter and 2: Introduction and Integral Equations 2 Model Problem: Gravity Surveying Unknown mass density distribution f at depth d below surface, from to on t axis. Measurements of vertical component of gravitational field g(s) at surface, from to on the s axis. g(s) s d f t θ

7 296 DIP Chapter and 2: Introduction and Integral Equations 3 Setting Up the Integral Equation The value of g(s) due to the part dt on the t axis dg = sin θ r 2 f dt, where r = d 2 + (s t) 2. Using that sin θ = d/r, we get sin θ r 2 f dt = d f dt. (d 2 + (s t) 2 ) 3/2 The total value of g(s) for s is therefore g(s) = This is the forward problem. d f dt. (d 2 + (s t) 2 ) 3/2 296 DIP Chapter and 2: Introduction and Integral Equations 4 Our Integral Equation Fredholm integral equation of the first kind: d f dt = g(s), s. (d 2 + (s t) 2 ) 3/2 The kernel K, which represents the model, is K(s, t) = h(s t) = d (d 2 + (s t) 2 ) 3/2, and the right-hand side g is what we are able to measure. From K and g we want to compute f, i.e., an inverse problem.

8 296 DIP Chapter and 2: Introduction and Integral Equations 5 Numerical Examples f g(s) 2 d =.25 d =.5 d = Observations: The signal/ data g(s) is a smoothed version of the source f. The deeper the source, the weaker the signal. The discontinuity in f is not visible in g(s). 296 DIP Chapter and 2: Introduction and Integral Equations 6 Fredholm Integral Equations of the First Kind Our generic inverse problem: K(s, t) f dt = g(s), s. Here, the kernel K(s, t) and the right-hand side g(s) are known functions, while f is the unknown function. In multiple dimensions, this equation takes the form Ω t K(s, t) f dt = g(s), s Ω s. An important special case: K(s, t) = h(s t) deconvolution: h(s t) f dt = g(s), s (and similarly in more dimensions).

9 296 DIP Chapter and 2: Introduction and Integral Equations 7 Another Example: -D Image Restoration Kernel K: point spread function for an infinitely long slit of width one wavelength. Independent variables t and s are the angles of the incoming and scattered light. 4 2 Regularization Tools: shaw Surface plot of A; D image reconstruction ( ) 2 K(s, t) = (cos(s) + cos) 2 sin(u) u π/2 π/2 u = π (sin(s) + sin) K(s, t) f dt = g(s), π/2 s π/2 296 DIP Chapter and 2: Introduction and Integral Equations 8 Yet Another Example: Second Derivative Plot of K(s,t) second derivate problem Kernel K: Green s function for the second derivative s(t ), s < t K(s, t) = t(s ), s t s t Regularization Tools: deriv2. Not differentiable across the line t = s. K(s, t) f dt = g(s), s Solution: f = g, t.

10 296 DIP Chapter and 2: Introduction and Integral Equations 9 The Riemann-Lebesgue Lemma Consider the function f = sin(2πp t), p =, 2,... then for p and arbitrary K we have g(s) = K(s, t) f dt. Smoothing: high frequencies are damped in the mapping f g. Hence, the mapping from g to f must amplify the high frequencies. Therefore we can expect difficulties when trying to reconstruct f from noisy data g. 296 DIP Chapter and 2: Introduction and Integral Equations 2 Illustration of the Riemann-Lebesgue Lemma Gravity problem with f = sin(2πp t), p =, 2, 4, and 8. f p g p (s) p = p = 2 p = 4 p = 8.5 Higher frequencies are dampen more than low frequencies.

11 296 DIP Chapter and 2: Introduction and Integral Equations 2 Difficulties with High Frequencies In this example δg(s) = K(s, t) δf dt and δg 2 =.. δf p δg p (s), δg p 2 = p = p = 2 p = 4 p = 8.5 Higher frequencies are amplified more in the reconstruction process. 296 DIP Chapter and 2: Introduction and Integral Equations 22 Why do We Care? Why bother about these (strange) issues? Ill-posed problems model a variety of real applications: Medical imaging (brain scanning, etc.) Geophysical prospecting (search for oil, land-mines, etc.) Image deblurring (astronomy, CSI a, etc.) Deconvolution of instrument s response. We can only hope to compute useful solutions to these problems if we fully understand their inherent difficulties... and how these difficulties carry over to the discretized problems involved in a computer solution, and how to deal with them in a satisfactory way. a Crime Scene Investigation.

12 296 DIP Chapter and 2: Introduction and Integral Equations 23 Some Important Questions How to discretize the inverse problem; here, the integral equation? Why is the matrix in the discretized problem always so ill conditioned? Why can we still compute an approximate solution? How can we compute it stably and efficiently? Is additional information available? How can we incorporate it in the solution scheme? How should we implement the numerical scheme? How do we solve large-scale problems? 296 DIP Chapter and 2: Introduction and Integral Equations 24 The Singular Value Expansion (SVE) For any square integrable kernel K holds K(s, t) = µ i u i (s) v i, i= where u i, u j = v i, v j = δ ij, and µ µ 2 µ The fundamental relation and the expansions f = K(s, t) v i dt = µ i u i (s), i =, 2,... v i, f v i and g(s) = i= lead to the expression for the solution: u i, g f = v i. µ i i= u i, g u i (s) i=

13 296 DIP Chapter and 2: Introduction and Integral Equations 25 The Singular Values Ordering Norm of kernel Hence, K 2 µ µ 2 µ 3. K(s, t) 2 ds dt = µ i = O(i q ), q > /2 i.e., the µ i decay faster than i /2. We have: µ 2 i. If the derivatives of order,..., q exist and are continuous, then µ i is approximately O(i q /2 ). i= Second derivative: µ i i 2 ( moderately ill posed ). -D image reconstruction: µ i e 2i ( severely ill posed ). 296 DIP Chapter and 2: Introduction and Integral Equations 26 Example of SVE (Degenerate) We can occasionally calculate the SVE analytically. Example (s + 2t) f dt = g(s), s. For this kernel K(s, t) = s + 2t we have µ = µ 2 = 2/ 3, µ 3 = µ 4 =... =. u (s) = / 2, u 2 (s) = 3/2 s v = 3/2 t, v 2 = / 2. A solution exists only if g range(k) = span{u, u 2 }, i.e., if g is of the form g(s) = c + c 2 s.

14 296 DIP Chapter and 2: Introduction and Integral Equations 27 The Smoothing Effect The smoother the kernel K, the faster the µ i decay to zero: If the derivatives of order,..., q exist and are continuous, then µ i is approximately O(i q /2 ). The smaller the µ i, the more oscillations (or zero-crossings) in the singular functions u i and v i. v 5 v 6 v 7 v 8 v v 2 v 3 v Since v i µ i u i (s), higher frequencies are damped more than lower frequencies (smoothing) in the forward problem. 296 DIP Chapter and 2: Introduction and Integral Equations 28 The Picard Condition In order that there exists a square integrable solution f to the integral equation, the right-hand side g must satisfy ( ) 2 ui, g <. i= µ i Equivalent condition: g range(k). In plain words: the absolute value of the coefficients (u i, g) must decay faster than the singular values µ i! Main difficulty: a noisy g does not satisfy the PC!

15 296 DIP Chapter and 2: Introduction and Integral Equations 29 Illustration of the Picard Condition 5 g f(s) No noise in g(s) Noise in g(s) µ i u i, g u i, g / µ i The violation of the Picard condition is the simple explanation of the instability of linear inverse problems in the form of first-kind Fredholm integral equations. SVE analysis + Picard plot insight remedy algorithms. 296 DIP Chapter and 2: Introduction and Integral Equations 3 A Problem with no Solution Ursell (974) presented the following innocently-looking problem: f dt =, s. s + t + This problem has no square integrable solution! Expand right-hand side g(s) = in terms of the singular functions: g k (s) = k u i, g u i (s); g g k 2 for k. i= Now consider f k s+t+ dt = g k(s), whose solution f k is f k = k i= u i, g σ i v i. Clearly f k 2 is finite for all k ; but f k 2 for k.

16 296 DIP Chapter and 2: Introduction and Integral Equations 3 Ursell Problem Numerical Results g g k f k f f 2 5 f 3 5 f f 5 2 x 4 f 6 2 x 5 f 7 2 x 6 f DIP Chapter and 2: Introduction and Integral Equations 32 Analytic SVEs are Rare A few cases where analytic SVEs are available, e.g., the Radon transform. But in most applications we must use numerical methods for analysis and solution of the integral equation. The rest of these lectures are devoted to numerical methods! Our analysis has given us an understanding of the difficulties we are facing and they will manifest themselves again in any numerical approach we re using to solve the integral equation.

Computational Optical Imaging - Optique Numerique. -- Deconvolution --

Computational Optical Imaging - Optique Numerique. -- Deconvolution -- Computational Optical Imaging - Optique Numerique -- Deconvolution -- Winter 2014 Ivo Ihrke Deconvolution Ivo Ihrke Outline Deconvolution Theory example 1D deconvolution Fourier method Algebraic method

More information

Numerical Methods For Image Restoration

Numerical Methods For Image Restoration Numerical Methods For Image Restoration CIRAM Alessandro Lanza University of Bologna, Italy Faculty of Engineering CIRAM Outline 1. Image Restoration as an inverse problem 2. Image degradation models:

More information

Introduction to Inverse Problems (2 lectures)

Introduction to Inverse Problems (2 lectures) Introduction to Inverse Problems (2 lectures) Summary Direct and inverse problems Examples of direct (forward) problems Deterministic and statistical points of view Ill-posed and ill-conditioned problems

More information

Computer Vision: Filtering

Computer Vision: Filtering Computer Vision: Filtering Raquel Urtasun TTI Chicago Jan 10, 2013 Raquel Urtasun (TTI-C) Computer Vision Jan 10, 2013 1 / 82 Today s lecture... Image formation Image Filtering Raquel Urtasun (TTI-C) Computer

More information

1. Introduction. Consider the computation of an approximate solution of the minimization problem

1. Introduction. Consider the computation of an approximate solution of the minimization problem A NEW TIKHONOV REGULARIZATION METHOD MARTIN FUHRY AND LOTHAR REICHEL Abstract. The numerical solution of linear discrete ill-posed problems typically requires regularization, i.e., replacement of the available

More information

The Image Deblurring Problem

The Image Deblurring Problem page 1 Chapter 1 The Image Deblurring Problem You cannot depend on your eyes when your imagination is out of focus. Mark Twain When we use a camera, we want the recorded image to be a faithful representation

More information

Jennifer Sloan May 23, Linear Inverse Problems

Jennifer Sloan May 23, Linear Inverse Problems Jennifer Sloan May 23, 2006 Linear Inverse Problems OUR PLAN Review Pertinet Linear Algebra Topics Forward Problem for Linear Systems Inverse Problem for Linear Systems Discuss Well-posedness and Overdetermined

More information

Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

More information

Chapter 3. Inverse Problems

Chapter 3. Inverse Problems 20 Chapter 3 Inverse Problems Mathematicians often cannot help but cringe when physicists are doing math, and even more so whenever physicists claim to be rigorous with their math. This chapter is not

More information

This is the 39th lecture and our topic for today is FIR Digital Filter Design by Windowing.

This is the 39th lecture and our topic for today is FIR Digital Filter Design by Windowing. Digital Signal Processing Prof: S. C. Dutta Roy Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture - 39 FIR Digital Filter Design by Windowing This is the 39th lecture and

More information

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

More information

Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold:

Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold: Linear Systems Example: Find x, x, x such that the following three equations hold: x + x + x = 4x + x + x = x + x + x = 6 We can write this using matrix-vector notation as 4 {{ A x x x {{ x = 6 {{ b General

More information

Some stability results of parameter identification in a jump diffusion model

Some stability results of parameter identification in a jump diffusion model Some stability results of parameter identification in a jump diffusion model D. Düvelmeyer Technische Universität Chemnitz, Fakultät für Mathematik, 09107 Chemnitz, Germany Abstract In this paper we discuss

More information

Estimating Dynamics for (DC-motor)+(1st Link) of the Furuta Pendulum

Estimating Dynamics for (DC-motor)+(1st Link) of the Furuta Pendulum Estimating Dynamics for (DC-motor)+(1st Link) of the Furuta Pendulum 1 Anton and Pedro Abstract Here the steps done for identification of dynamics for (DC-motor)+(1st Link) of the Furuta Pendulum are described.

More information

P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition

P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition P164 Tomographic Velocity Model Building Using Iterative Eigendecomposition K. Osypov* (WesternGeco), D. Nichols (WesternGeco), M. Woodward (WesternGeco) & C.E. Yarman (WesternGeco) SUMMARY Tomographic

More information

Numerical Linear Algebra Chap. 4: Perturbation and Regularisation

Numerical Linear Algebra Chap. 4: Perturbation and Regularisation Numerical Linear Algebra Chap. 4: Perturbation and Regularisation Heinrich Voss voss@tu-harburg.de Hamburg University of Technology Institute of Numerical Simulation TUHH Heinrich Voss Numerical Linear

More information

FOURIER TRANSFORM METHODS IN GEOPHYSICS David Sandwell, January, 2013

FOURIER TRANSFORM METHODS IN GEOPHYSICS David Sandwell, January, 2013 1 FOURIER TRANSFORM METHODS IN GEOPHYSICS David Sandwell, January, 2013 1. Fourier Transforms Fourier transform are use in many areas of geophysics such as image processing, time series analysis, and antenna

More information

Automatic parameter setting for Arnoldi-Tikhonov methods

Automatic parameter setting for Arnoldi-Tikhonov methods Automatic parameter setting for Arnoldi-Tikhonov methods S. Gazzola, P. Novati Department of Mathematics University of Padova, Italy Abstract In the framework of iterative regularization techniques for

More information

ORTHOGONAL PROJECTION REGULARIZATION OPERATORS

ORTHOGONAL PROJECTION REGULARIZATION OPERATORS ORTHOGONAL PROJECTION REGULARIZATION OPERATORS S. MORIGI, L. REICHEL, AND F. SGALLARI Abstract. Tikhonov regularization often is applied with a finite difference regularization operator that approximates

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 3 Linear Least Squares Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

More information

APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (Copyright 2001, David T. Sandwell)

APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (Copyright 2001, David T. Sandwell) 1 APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (Copyright 2001, David T. Sandwell) (Reference The Fourier Transform and its Application, second edition, R.N. Bracewell, McGraw-Hill Book Co., New

More information

3 Numerical Methods for Convection

3 Numerical Methods for Convection 3 Numerical Methods for Convection The topic of computational fluid dynamics (CFD) could easily occupy an entire semester indeed, we have such courses in our catalog. The objective here is to eamine some

More information

Understanding Poles and Zeros

Understanding Poles and Zeros MASSACHUSETTS INSTITUTE OF TECHNOLOGY DEPARTMENT OF MECHANICAL ENGINEERING 2.14 Analysis and Design of Feedback Control Systems Understanding Poles and Zeros 1 System Poles and Zeros The transfer function

More information

1 Review of Least Squares Solutions to Overdetermined Systems

1 Review of Least Squares Solutions to Overdetermined Systems cs4: introduction to numerical analysis /9/0 Lecture 7: Rectangular Systems and Numerical Integration Instructor: Professor Amos Ron Scribes: Mark Cowlishaw, Nathanael Fillmore Review of Least Squares

More information

Solving Sets of Equations. 150 B.C.E., 九章算術 Carl Friedrich Gauss,

Solving Sets of Equations. 150 B.C.E., 九章算術 Carl Friedrich Gauss, Solving Sets of Equations 5 B.C.E., 九章算術 Carl Friedrich Gauss, 777-855 Gaussian-Jordan Elimination In Gauss-Jordan elimination, matrix is reduced to diagonal rather than triangular form Row combinations

More information

AST1100 Lecture Notes

AST1100 Lecture Notes AST1100 Lecture Notes 3 Extrasolar planets 1 Detecting extrasolar planets Most models of star formation tell us that the formation of planets is a common process. We expect most stars to have planets orbiting

More information

ROOT LOCUS TECHNIQUES

ROOT LOCUS TECHNIQUES ROOT LOCUS TECHNIQUES In this lecture you will learn the following : The definition of a root locus How to sketch root locus How to use the root locus to find the poles of a closed loop system How to use

More information

Chapter 4 Rotating Coordinate Systems and the Equations of Motion

Chapter 4 Rotating Coordinate Systems and the Equations of Motion Chapter 4 Rotating Coordinate Systems and the Equations of Motion 1. Rates of change of vectors We have derived the Navier Stokes equations in an inertial (non accelerating frame of reference) for which

More information

Geophysical data integration and its application to seismic tomography

Geophysical data integration and its application to seismic tomography Geophysical data integration and its application to seismic tomography Mohammad Maysami ABSTRACT For oil exploration and reservoir monitoring purposes, we probe the earth s subsurface with a variety of

More information

People s Physics book 3e Ch 25-1

People s Physics book 3e Ch 25-1 The Big Idea: In most realistic situations forces and accelerations are not fixed quantities but vary with time or displacement. In these situations algebraic formulas cannot do better than approximate

More information

Automatic parameter setting for Arnoldi-Tikhonov methods

Automatic parameter setting for Arnoldi-Tikhonov methods Automatic parameter setting for Arnoldi-Tikhonov methods S. Gazzola, P. Novati Department of Mathematics University of Padova, Italy March 16, 2012 Abstract In the framework of iterative regularization

More information

Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

More information

Lecture 20. PLANAR KINEMATIC-PROBLEM EXAMPLES

Lecture 20. PLANAR KINEMATIC-PROBLEM EXAMPLES Lecture 20. PLANAR KINEMATIC-PROBLEM EXAMPLES Figure 4.15 Slider-crank mechanism. TASK: For a given constant rotation rate, find the velocity and acceleration terms of the piston for one cycle of θ. Geometric

More information

Compressive Sensing. Examples in Image Compression. Lecture 4, July 30, Luiz Velho Eduardo A. B. da Silva Adriana Schulz

Compressive Sensing. Examples in Image Compression. Lecture 4, July 30, Luiz Velho Eduardo A. B. da Silva Adriana Schulz Compressive Sensing Examples in Image Compression Lecture 4, July, 09 Luiz Velho Eduardo A. B. da Silva Adriana Schulz Today s Lecture Discuss applications of CS in image compression Evaluate CS efficiency

More information

8. Linear least-squares

8. Linear least-squares 8. Linear least-squares EE13 (Fall 211-12) definition examples and applications solution of a least-squares problem, normal equations 8-1 Definition overdetermined linear equations if b range(a), cannot

More information

Blind Deconvolution of Corrupted Barcode Signals

Blind Deconvolution of Corrupted Barcode Signals Blind Deconvolution of Corrupted Barcode Signals Everardo Uribe and Yifan Zhang Advisors: Ernie Esser and Yifei Lou Interdisciplinary Computational and Applied Mathematics Program University of California,

More information

NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS

NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS DO Q LEE Abstract. Computing the solution to Least Squares Problems is of great importance in a wide range of fields ranging from numerical

More information

Inverse Problems and Regularization An Introduction

Inverse Problems and Regularization An Introduction Inverse Problems and Regularization An Introduction Stefan Kindermann Industrial Mathematics Institute University of Linz, Austria What are Inverse Problems? One possible definition [Engl, Hanke, Neubauer

More information

Lecture 5 Least-squares

Lecture 5 Least-squares EE263 Autumn 2007-08 Stephen Boyd Lecture 5 Least-squares least-squares (approximate) solution of overdetermined equations projection and orthogonality principle least-squares estimation BLUE property

More information

Lecture L19 - Vibration, Normal Modes, Natural Frequencies, Instability

Lecture L19 - Vibration, Normal Modes, Natural Frequencies, Instability S. Widnall 16.07 Dynamics Fall 2009 Version 1.0 Lecture L19 - Vibration, Normal Modes, Natural Frequencies, Instability Vibration, Instability An important class of problems in dynamics concerns the free

More information

Harmonic Oscillator Physics

Harmonic Oscillator Physics Physics 34 Lecture 9 Harmonic Oscillator Physics Lecture 9 Physics 34 Quantum Mechanics I Friday, February th, 00 For the harmonic oscillator potential in the time-independent Schrödinger equation: d ψx

More information

Interval Pseudo-Inverse Matrices and Interval Greville Algorithm

Interval Pseudo-Inverse Matrices and Interval Greville Algorithm Interval Pseudo-Inverse Matrices and Interval Greville Algorithm P V Saraev Lipetsk State Technical University, Lipetsk, Russia psaraev@yandexru Abstract This paper investigates interval pseudo-inverse

More information

Linear Least Squares

Linear Least Squares Linear Least Squares Suppose we are given a set of data points {(x i,f i )}, i = 1,...,n. These could be measurements from an experiment or obtained simply by evaluating a function at some points. One

More information

Using the Singular Value Decomposition

Using the Singular Value Decomposition Using the Singular Value Decomposition Emmett J. Ientilucci Chester F. Carlson Center for Imaging Science Rochester Institute of Technology emmett@cis.rit.edu May 9, 003 Abstract This report introduces

More information

2.5 Gaussian Elimination

2.5 Gaussian Elimination page 150 150 CHAPTER 2 Matrices and Systems of Linear Equations 37 10 the linear algebra package of Maple, the three elementary 20 23 1 row operations are 12 1 swaprow(a,i,j): permute rows i and j 3 3

More information

Stability Of Structures: Basic Concepts

Stability Of Structures: Basic Concepts 23 Stability Of Structures: Basic Concepts ASEN 3112 Lecture 23 Slide 1 Objective This Lecture (1) presents basic concepts & terminology on structural stability (2) describes conceptual procedures for

More information

The Method of Least Squares

The Method of Least Squares 33 The Method of Least Squares KEY WORDS confidence interval, critical sum of squares, dependent variable, empirical model, experimental error, independent variable, joint confidence region, least squares,

More information

CAN PLASMA SCATTERING MIMIC A COSMOLOGICAL RED SHIFT?

CAN PLASMA SCATTERING MIMIC A COSMOLOGICAL RED SHIFT? CAN PLASMA SCATTERING MIMIC A COSMOLOGICAL RED SHIFT? S. SCHRAMM* AND S. E. KOONIN W. K. Kellogg Radiation Laboratory California Institute of Technology, Pasadena, CA 91125 ABSTRACT We investigate the

More information

Notes on weak convergence (MAT Spring 2006)

Notes on weak convergence (MAT Spring 2006) Notes on weak convergence (MAT4380 - Spring 2006) Kenneth H. Karlsen (CMA) February 2, 2006 1 Weak convergence In what follows, let denote an open, bounded, smooth subset of R N with N 2. We assume 1 p

More information

Change Point Estimation of Bar Code Signals

Change Point Estimation of Bar Code Signals 1 Change Point Estimation of Bar Code Signals Leming Qu, and Yi-Cheng Tu Abstract Existing methods for bar code signal reconstruction is based on either the local approach or the regularization approach

More information

Lecture 5: Singular Value Decomposition SVD (1)

Lecture 5: Singular Value Decomposition SVD (1) EEM3L1: Numerical and Analytical Techniques Lecture 5: Singular Value Decomposition SVD (1) EE3L1, slide 1, Version 4: 25-Sep-02 Motivation for SVD (1) SVD = Singular Value Decomposition Consider the system

More information

ME231 Measurements Laboratory Spring Fourier Series. Edmundo Corona c

ME231 Measurements Laboratory Spring Fourier Series. Edmundo Corona c ME23 Measurements Laboratory Spring 23 Fourier Series Edmundo Corona c If you listen to music you may have noticed that you can tell what instruments are used in a given song or symphony. In some cases,

More information

Least squares: the big idea

Least squares: the big idea Week 5: Wednesday, Feb 22 Least squares: the big idea Least squares problems are a special sort of minimization. Suppose A R m n and m > n. In general, we will not be able to exactly solve overdetermined

More information

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10 Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 10 Boundary Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

More information

Practical Numerical Training UKNum

Practical Numerical Training UKNum Practical Numerical Training UKNum 7: Systems of linear equations C. Mordasini Max Planck Institute for Astronomy, Heidelberg Program: 1) Introduction 2) Gauss Elimination 3) Gauss with Pivoting 4) Determinants

More information

Eigenvalues of C SMS (16) 1 A. Eigenvalues of A (32) 1 A (16,32) 1 A

Eigenvalues of C SMS (16) 1 A. Eigenvalues of A (32) 1 A (16,32) 1 A Negative Results for Multilevel Preconditioners in Image Deblurring C. R. Vogel Department of Mathematical Sciences Montana State University Bozeman, MT 59717-0240 USA Abstract. A one-dimensional deconvolution

More information

Micro-g LaCoste Gravity Company

Micro-g LaCoste Gravity Company Micro-g LaCoste Gravity Company Location: Lafayette, Colorado (Boulder/Denver area) Business: Absolute gravity meters and gradiometers and AirSea, Borehole, and Earth- Tide spring gravity meters Expertise:

More information

Topic 4: Continuous-Time Fourier Transform (CTFT)

Topic 4: Continuous-Time Fourier Transform (CTFT) ELEC264: Signals And Systems Topic 4: Continuous-Time Fourier Transform (CTFT) Aishy Amer Concordia University Electrical and Computer Engineering o Introduction to Fourier Transform o Fourier transform

More information

Assignment 7; Due Friday, November 11

Assignment 7; Due Friday, November 11 Assignment 7; Due Friday, November 9.8 a The set Q is not connected because we can write it as a union of two nonempty disjoint open sets, for instance U = (, 2) and V = ( 2, ). The connected subsets are

More information

Foundation Engineering Prof. Mahendra Singh Department of Civil Engineering Indian Institute of Technology, Roorkee

Foundation Engineering Prof. Mahendra Singh Department of Civil Engineering Indian Institute of Technology, Roorkee Foundation Engineering Prof. Mahendra Singh Department of Civil Engineering Indian Institute of Technology, Roorkee Module - 03 Lecture - 09 Stability of Slopes Welcome back to the classes of on this Stability

More information

1 1. The ROC of the z-transform H(z) of the impulse response sequence h[n] is defined

1 1. The ROC of the z-transform H(z) of the impulse response sequence h[n] is defined A causal LTI digital filter is BIBO stable if and only if its impulse response h[ is absolutely summable, i.e., S h [ < n We now develop a stability condition in terms of the pole locations of the transfer

More information

Improved Residual Analysis in ADC Testing. István Kollár

Improved Residual Analysis in ADC Testing. István Kollár Improved Residual Analysis in ADC Testing István Kollár Dept. of Measurement and Information Systems Budapest University of Technology and Economics H-1521 Budapest, Magyar tudósok krt. 2., HUNGARY Tel.:

More information

Abstract Linear Algebra, Fall Solutions to Problems III

Abstract Linear Algebra, Fall Solutions to Problems III Abstract Linear Algebra, Fall 211 - Solutions to Problems III 1. Let P 3 denote the real vector space of all real polynomials p(t) of degree at most 3. Consider the linear map T : P 3 P 3 given by T p(t)

More information

Moving Least Squares Approximation

Moving Least Squares Approximation Chapter 7 Moving Least Squares Approimation An alternative to radial basis function interpolation and approimation is the so-called moving least squares method. As we will see below, in this method the

More information

Chapter 4: Time-Domain Representation for Linear Time-Invariant Systems

Chapter 4: Time-Domain Representation for Linear Time-Invariant Systems Chapter 4: Time-Domain Representation for Linear Time-Invariant Systems In this chapter we will consider several methods for describing relationship between the input and output of linear time-invariant

More information

Towards a Theory for Arbitrarily Shaped Sound Field Reproduction Systems

Towards a Theory for Arbitrarily Shaped Sound Field Reproduction Systems Towards a Theory for Arbitrarily Shaped Sound Field Reproduction Systems Sascha Spors, Matthias Rath and Jens Ahrens Sascha.Spors@telekom.de Acoustics 08, Paris ===!" Laboratories AG Laboratories Aim of

More information

p Values and Alternative Boundaries

p Values and Alternative Boundaries p Values and Alternative Boundaries for CUSUM Tests Achim Zeileis Working Paper No. 78 December 2000 December 2000 SFB Adaptive Information Systems and Modelling in Economics and Management Science Vienna

More information

Module 4. Contents. Digital Filters - Implementation and Design. Signal Flow Graphs. Digital Filter Structures. FIR and IIR Filter Design Techniques

Module 4. Contents. Digital Filters - Implementation and Design. Signal Flow Graphs. Digital Filter Structures. FIR and IIR Filter Design Techniques Module 4 Digital Filters - Implementation and Design Digital Signal Processing. Slide 4.1 Contents Signal Flow Graphs Basic filtering operations Digital Filter Structures Direct form FIR and IIR filters

More information

LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12,

LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12, LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12, 2000 45 4 Iterative methods 4.1 What a two year old child can do Suppose we want to find a number x such that cos x = x (in radians). This is a nonlinear

More information

Chapter 24 Physical Pendulum

Chapter 24 Physical Pendulum Chapter 4 Physical Pendulum 4.1 Introduction... 1 4.1.1 Simple Pendulum: Torque Approach... 1 4. Physical Pendulum... 4.3 Worked Examples... 4 Example 4.1 Oscillating Rod... 4 Example 4.3 Torsional Oscillator...

More information

Parabolic Equations. Chapter 5. Contents. 5.1.2 Well-Posed Initial-Boundary Value Problem. 5.1.3 Time Irreversibility of the Heat Equation

Parabolic Equations. Chapter 5. Contents. 5.1.2 Well-Posed Initial-Boundary Value Problem. 5.1.3 Time Irreversibility of the Heat Equation 7 5.1 Definitions Properties Chapter 5 Parabolic Equations Note that we require the solution u(, t bounded in R n for all t. In particular we assume that the boundedness of the smooth function u at infinity

More information

An introduction to the electric conduction in crystals

An introduction to the electric conduction in crystals An introduction to the electric conduction in crystals Andreas Wacker, Matematisk Fysik, Lunds Universitet Andreas.Wacker@fysik.lu.se November 23, 2010 1 Velocity of band electrons The electronic states

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

More information

Fourier Series and Integrals

Fourier Series and Integrals Fourier Series and Integrals Fourier Series et f(x) be a piecewise linear function on [,] (This means that f(x) may possess a finite number of finite discontinuities on the interval). Then f(x) can be

More information

Nonlinear Algebraic Equations Example

Nonlinear Algebraic Equations Example Nonlinear Algebraic Equations Example Continuous Stirred Tank Reactor (CSTR). Look for steady state concentrations & temperature. s r (in) p,i (in) i In: N spieces with concentrations c, heat capacities

More information

Lecture L18 - Exploring the Neighborhood: the Restricted Three-Body Problem

Lecture L18 - Exploring the Neighborhood: the Restricted Three-Body Problem S. Widnall 16.07 Dynamics Fall 008 Version 1.0 Lecture L18 - Exploring the Neighborhood: the Restricted Three-Body Problem The Three-Body Problem In Lecture 15-17, we presented the solution to the two-body

More information

What is a Filter? Output Signal. Input Signal Amplitude. Frequency. Low Pass Filter

What is a Filter? Output Signal. Input Signal Amplitude. Frequency. Low Pass Filter What is a Filter? Input Signal Amplitude Output Signal Frequency Time Sequence Low Pass Filter Time Sequence What is a Filter Input Signal Amplitude Output Signal Frequency Signal Noise Signal Noise Frequency

More information

3.2 Sources, Sinks, Saddles, and Spirals

3.2 Sources, Sinks, Saddles, and Spirals 3.2. Sources, Sinks, Saddles, and Spirals 6 3.2 Sources, Sinks, Saddles, and Spirals The pictures in this section show solutions to Ay 00 C By 0 C Cy D 0. These are linear equations with constant coefficients

More information

The application of Fourier analysis in solving the Computed Tomography (CT) inverse problem.

The application of Fourier analysis in solving the Computed Tomography (CT) inverse problem. The application of Fourier analysis in solving the Computed Tomography (CT) inverse problem. by Nasser M. Abbasi July 3, 2010 Abstract Digital signal processing plays an increasingly important rule in

More information

APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (David Sandwell, Copyright, 2004)

APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (David Sandwell, Copyright, 2004) 1 APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (David Sandwell, Copyright, 2004) (Reference The Fourier Transform and its Application, second edition, R.N. Bracewell, McGraw-Hill Book Co., New York,

More information

International Journal of Advanced Research in Computer Science and Software Engineering

International Journal of Advanced Research in Computer Science and Software Engineering Volume 4, Issue 8, August 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Comparative Study

More information

Chapter 5: Application: Fourier Series

Chapter 5: Application: Fourier Series 321 28 9 Chapter 5: Application: Fourier Series For lack of time, this chapter is only an outline of some applications of Functional Analysis and some proofs are not complete. 5.1 Definition. If f L 1

More information

5.2 Accuracy and Stability for u t = c u x

5.2 Accuracy and Stability for u t = c u x c 006 Gilbert Strang 5. Accuracy and Stability for u t = c u x This section begins a major topic in scientific computing: Initial-value problems for partial differential equations. Naturally we start with

More information

3.4. Solving Simultaneous Linear Equations. Introduction. Prerequisites. Learning Outcomes

3.4. Solving Simultaneous Linear Equations. Introduction. Prerequisites. Learning Outcomes Solving Simultaneous Linear Equations 3.4 Introduction Equations often arise in which there is more than one unknown quantity. When this is the case there will usually be more than one equation involved.

More information

Interpolating and Approximating Implicit Surfaces from Polygon Soup

Interpolating and Approximating Implicit Surfaces from Polygon Soup Interpolating and Approximating Implicit Surfaces from Polygon Soup 1. Briefly summarize the paper s contributions. Does it address a new problem? Does it present a new approach? Does it show new types

More information

Robust and LPV control of MIMO systems Part 3: Robustness analysis

Robust and LPV control of MIMO systems Part 3: Robustness analysis Robust and LPV control of MIMO systems Part 3: Robustness analysis Olivier Sename GIPSA-Lab Tecnologico de Monterrey, July 2016 Olivier Sename (GIPSA-Lab) Robust and LPV control - part 3 Tecnologico de

More information

Regularized Matrix Computations

Regularized Matrix Computations Regularized Matrix Computations Andrew E. Yagle Department of EECS, The University of Michigan, Ann Arbor, MI 489- Abstract We review the basic results on: () the singular value decomposition (SVD); ()

More information

SOME RECENT DEVELOPMENTS AND OPEN PROBLEMS IN SOLUTION METHODS FOR MATHEMATICAL INVERSE PROBLEMS

SOME RECENT DEVELOPMENTS AND OPEN PROBLEMS IN SOLUTION METHODS FOR MATHEMATICAL INVERSE PROBLEMS SOME RECENT DEVELOPMENTS AND OPEN PROBLEMS IN SOLUTION METHODS FOR MATHEMATICAL INVERSE PROBLEMS Patricia K. Lamm lamm@math.msu.edu Department of Mathematics, Michigan State University, East Lansing, MI

More information

We consider a hydrogen atom in the ground state in the uniform electric field

We consider a hydrogen atom in the ground state in the uniform electric field Lecture 13 Page 1 Lectures 13-14 Hydrogen atom in electric field. Quadratic Stark effect. Atomic polarizability. Emission and Absorption of Electromagnetic Radiation by Atoms Transition probabilities and

More information

FIELDS-MITACS Conference. on the Mathematics of Medical Imaging. Photoacoustic and Thermoacoustic Tomography with a variable sound speed

FIELDS-MITACS Conference. on the Mathematics of Medical Imaging. Photoacoustic and Thermoacoustic Tomography with a variable sound speed FIELDS-MITACS Conference on the Mathematics of Medical Imaging Photoacoustic and Thermoacoustic Tomography with a variable sound speed Gunther Uhlmann UC Irvine & University of Washington Toronto, Canada,

More information

Lecture Notes. Inverse Problems. Winter 2007/08

Lecture Notes. Inverse Problems. Winter 2007/08 Lecture Notes Inverse Problems Winter 2007/08 Martin Burger Institut für Numerische und Angewandte Mathematik Westfälische Wilhelms-Universität Münster martin.burger@uni-muenster.de 1 Contents 1 Introduction

More information

Lecture notes Numerical Mathematics, First Course Autumn Sven-Åke Gustafson Høgskolen i Stavanger, Norway

Lecture notes Numerical Mathematics, First Course Autumn Sven-Åke Gustafson Høgskolen i Stavanger, Norway Lecture notes Numerical Mathematics, First Course Autumn 2003 Sven-Åke Gustafson Høgskolen i Stavanger, Norway August 22, 2003 Contents 1 On the representation of numbers in a computer 1 1.1 Introduction................................

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

More information

3.4. Solving simultaneous linear equations. Introduction. Prerequisites. Learning Outcomes

3.4. Solving simultaneous linear equations. Introduction. Prerequisites. Learning Outcomes Solving simultaneous linear equations 3.4 Introduction Equations often arise in which there is more than one unknown quantity. When this is the case there will usually be more than one equation involved.

More information

4. The Infinite Square Well

4. The Infinite Square Well 4. The Infinite Square Well Copyright c 215 216, Daniel V. Schroeder In the previous lesson I emphasized the free particle, for which V (x) =, because its energy eigenfunctions are so simple: they re the

More information

ME128 Computer-Aided Mechanical Design Course Notes Introduction to Design Optimization

ME128 Computer-Aided Mechanical Design Course Notes Introduction to Design Optimization ME128 Computer-ided Mechanical Design Course Notes Introduction to Design Optimization 2. OPTIMIZTION Design optimization is rooted as a basic problem for design engineers. It is, of course, a rare situation

More information

Inverse problems. Nikolai Piskunov 2014. Regularization: Example Lecture 4

Inverse problems. Nikolai Piskunov 2014. Regularization: Example Lecture 4 Inverse problems Nikolai Piskunov 2014 Regularization: Example Lecture 4 Now that we know the theory, let s try an application: Problem: Optimal filtering of 1D and 2D data Solution: formulate an inverse

More information

Lecture 11 Feedback: static analysis

Lecture 11 Feedback: static analysis S. Boyd EE102 Lecture 11 Feedback: static analysis feedback: overview, standard configuration, terms static linear case sensitivity static nonlinear case linearizing effect of feedback 11 1 Feedback: general

More information

Part 21. Isometries. Overview. Contents. 1 Isometry. Printed version of the lecture Differential Geometry on 20. November 2009

Part 21. Isometries. Overview. Contents. 1 Isometry. Printed version of the lecture Differential Geometry on 20. November 2009 Part 21 Isometries Printed version of the lecture Differential Geometry on 20. November 2009 Tommy R. Jensen, Department of Mathematics, KNU 21.1 Overview Contents 1 Isometry 1 2 Orthogonal Transformations

More information

On real axis: the number of poles and zeros on RHS of a testing point s o = even number (0 is counted even).

On real axis: the number of poles and zeros on RHS of a testing point s o = even number (0 is counted even). Zero-degree Loci for Negative K We consider KG(s) = K sm + b 1 s m 1 + + b m. s n + a 1 s n 1 + + a n By convention, we have 1+KG(s) =0, K =0. The angle of G(s) is0 o + 360 o l Change: 180 o + 360 o is

More information