Another Example: the Hubble Space Telescope

Transcription

1 296 DIP Chapter and 2: Introduction and Integral Equations Motivation: Why Inverse Problems? A large-scale example, coming from a collaboration with Università degli Studi di Napoli Federico II in Naples. From measurements of the magnetic field above Vesuvius, we determine the activity inside the volcano. Measurements on the surface Reconstruction inside the volcano 296 DIP Chapter and 2: Introduction and Integral Equations 2 Another Example: the Hubble Space Telescope For several years, the HST produced blurred images.

2 296 DIP Chapter and 2: Introduction and Integral Equations 3 Inverse Problems... typically arise when one wants to compute information about some interior properties using exterior measurements. Inverse Problem One of these is known Input System Known but with errors Output 296 DIP Chapter and 2: Introduction and Integral Equations 4 Inverse Problems: Examples A quite generic formulation: input system dω = output Ω Image restoration scenery lens image Tomography X-ray source object damping Seismology seismic wave layers reflections

3 296 DIP Chapter and 2: Introduction and Integral Equations 5 Discrete Ill-Posed Problems Our generic ill-posed problem: A Fredholm integral equation of the first kind K(s, t) f dt = g(s), s. Definition of a discrete ill-posed problem (DIP):. a square or over-determined system of linear algebraic equations A x = b or min x A x b 2 2. whose coefficient matrix A has a huge condition number, and 3. comes from the discretization of an inverse/ill-posed problem. 296 DIP Chapter and 2: Introduction and Integral Equations 6 Computational Issues The plots below show solutions x to the DIP A x = b..5 x 6 Gaussian elimination 2.5 Truncated SVD TSVD solution Exact solution Standard numerical methods (x = A\b) produce useless results. Specialized methods (this course) produce reasonable results.

4 296 DIP Chapter and 2: Introduction and Integral Equations 7 The Mechanisms of Ill-Conditioned Problems Consider a linear system with coefficient matrix and right-hand side ( ). A =.7., b =.25 = A There is no vector x such that A x = b. The least squares solution, which solves the problem is given by x LS = ( ) min x A x b 2, A x LS b 2 =.22. Far from exact solution (, ) T yet the residual is small. 296 DIP Chapter and 2: Introduction and Integral Equations 8 Other Solutions with Small Residual Two other solutions with a small residual are ( ).65 x = A x b 2 =.3 ( ) x = A x b 2 = All the solutions x LS, x and x have small residuals, yet they are far from the exact solution! The matrix A is ill conditioned. Small perturbations of the data (here: b) can lead to large perturbations of the solution. A small residual does not imply a good solution. (All this is well known stuff from matrix computations.)

5 296 DIP Chapter and 2: Introduction and Integral Equations 9 Stabilization! It turns out that we can modify the problem such that the solution is more stable, i.e., less sensitive to perturbations. Example: enforce an upper bound on the solution norm x 2 : min x A x b 2 subject to x 2 δ. The solution x δ depends in a nonlinear way on δ: ( ) ( ).8.84 x. =, x =.5.54 ( ) ( ) x.385 =, x = By supplying the correct additional information we can compute a good approximate solution DIP Chapter and 2: Introduction and Integral Equations Inverse Problems Ill-Conditioned Problems Whenever we solve an inverse problem on a computer, we face difficulties because the computational problems are ill conditioned. The purpose of my lectures are:. To explain why ill-conditioned computations always arise when solving inverse problems. 2. To explain the fundamental mechanisms underlying the ill conditioning. 3. To explain how we can modify the problem in order to stabilize the solution. 4. To show how this can be done efficiently on a computer. Regularization methods is at the heart of all this.

6 296 DIP Chapter and 2: Introduction and Integral Equations Inverse Problems are Ill-Posed Problems Hadamard s definition of a well-posed problem (early 2th century):. the problem must have a solution, 2. the solution must be unique, and 3. it must depend continuously on data and parameters. If the problem violates any of these requirements, it is ill posed. Condition can be fixed by reformulating/redefining the solution. Condition 2 can be fixed by additional requirements to the solution, e.g., that of minimum norm. Condition 3 is harder to fix because it implies that arbitrarily small perturbations of data and parameters can produce arbitrarily large perturbations of the solution. 296 DIP Chapter and 2: Introduction and Integral Equations 2 Model Problem: Gravity Surveying Unknown mass density distribution f at depth d below surface, from to on t axis. Measurements of vertical component of gravitational field g(s) at surface, from to on the s axis. g(s) s d f t θ

7 296 DIP Chapter and 2: Introduction and Integral Equations 3 Setting Up the Integral Equation The value of g(s) due to the part dt on the t axis dg = sin θ r 2 f dt, where r = d 2 + (s t) 2. Using that sin θ = d/r, we get sin θ r 2 f dt = d f dt. (d 2 + (s t) 2 ) 3/2 The total value of g(s) for s is therefore g(s) = This is the forward problem. d f dt. (d 2 + (s t) 2 ) 3/2 296 DIP Chapter and 2: Introduction and Integral Equations 4 Our Integral Equation Fredholm integral equation of the first kind: d f dt = g(s), s. (d 2 + (s t) 2 ) 3/2 The kernel K, which represents the model, is K(s, t) = h(s t) = d (d 2 + (s t) 2 ) 3/2, and the right-hand side g is what we are able to measure. From K and g we want to compute f, i.e., an inverse problem.

8 296 DIP Chapter and 2: Introduction and Integral Equations 5 Numerical Examples f g(s) 2 d =.25 d =.5 d = Observations: The signal/ data g(s) is a smoothed version of the source f. The deeper the source, the weaker the signal. The discontinuity in f is not visible in g(s). 296 DIP Chapter and 2: Introduction and Integral Equations 6 Fredholm Integral Equations of the First Kind Our generic inverse problem: K(s, t) f dt = g(s), s. Here, the kernel K(s, t) and the right-hand side g(s) are known functions, while f is the unknown function. In multiple dimensions, this equation takes the form Ω t K(s, t) f dt = g(s), s Ω s. An important special case: K(s, t) = h(s t) deconvolution: h(s t) f dt = g(s), s (and similarly in more dimensions).

9 296 DIP Chapter and 2: Introduction and Integral Equations 7 Another Example: -D Image Restoration Kernel K: point spread function for an infinitely long slit of width one wavelength. Independent variables t and s are the angles of the incoming and scattered light. 4 2 Regularization Tools: shaw Surface plot of A; D image reconstruction ( ) 2 K(s, t) = (cos(s) + cos) 2 sin(u) u π/2 π/2 u = π (sin(s) + sin) K(s, t) f dt = g(s), π/2 s π/2 296 DIP Chapter and 2: Introduction and Integral Equations 8 Yet Another Example: Second Derivative Plot of K(s,t) second derivate problem Kernel K: Green s function for the second derivative s(t ), s < t K(s, t) = t(s ), s t s t Regularization Tools: deriv2. Not differentiable across the line t = s. K(s, t) f dt = g(s), s Solution: f = g, t.

10 296 DIP Chapter and 2: Introduction and Integral Equations 9 The Riemann-Lebesgue Lemma Consider the function f = sin(2πp t), p =, 2,... then for p and arbitrary K we have g(s) = K(s, t) f dt. Smoothing: high frequencies are damped in the mapping f g. Hence, the mapping from g to f must amplify the high frequencies. Therefore we can expect difficulties when trying to reconstruct f from noisy data g. 296 DIP Chapter and 2: Introduction and Integral Equations 2 Illustration of the Riemann-Lebesgue Lemma Gravity problem with f = sin(2πp t), p =, 2, 4, and 8. f p g p (s) p = p = 2 p = 4 p = 8.5 Higher frequencies are dampen more than low frequencies.

11 296 DIP Chapter and 2: Introduction and Integral Equations 2 Difficulties with High Frequencies In this example δg(s) = K(s, t) δf dt and δg 2 =.. δf p δg p (s), δg p 2 = p = p = 2 p = 4 p = 8.5 Higher frequencies are amplified more in the reconstruction process. 296 DIP Chapter and 2: Introduction and Integral Equations 22 Why do We Care? Why bother about these (strange) issues? Ill-posed problems model a variety of real applications: Medical imaging (brain scanning, etc.) Geophysical prospecting (search for oil, land-mines, etc.) Image deblurring (astronomy, CSI a, etc.) Deconvolution of instrument s response. We can only hope to compute useful solutions to these problems if we fully understand their inherent difficulties... and how these difficulties carry over to the discretized problems involved in a computer solution, and how to deal with them in a satisfactory way. a Crime Scene Investigation.

12 296 DIP Chapter and 2: Introduction and Integral Equations 23 Some Important Questions How to discretize the inverse problem; here, the integral equation? Why is the matrix in the discretized problem always so ill conditioned? Why can we still compute an approximate solution? How can we compute it stably and efficiently? Is additional information available? How can we incorporate it in the solution scheme? How should we implement the numerical scheme? How do we solve large-scale problems? 296 DIP Chapter and 2: Introduction and Integral Equations 24 The Singular Value Expansion (SVE) For any square integrable kernel K holds K(s, t) = µ i u i (s) v i, i= where u i, u j = v i, v j = δ ij, and µ µ 2 µ The fundamental relation and the expansions f = K(s, t) v i dt = µ i u i (s), i =, 2,... v i, f v i and g(s) = i= lead to the expression for the solution: u i, g f = v i. µ i i= u i, g u i (s) i=

13 296 DIP Chapter and 2: Introduction and Integral Equations 25 The Singular Values Ordering Norm of kernel Hence, K 2 µ µ 2 µ 3. K(s, t) 2 ds dt = µ i = O(i q ), q > /2 i.e., the µ i decay faster than i /2. We have: µ 2 i. If the derivatives of order,..., q exist and are continuous, then µ i is approximately O(i q /2 ). i= Second derivative: µ i i 2 ( moderately ill posed ). -D image reconstruction: µ i e 2i ( severely ill posed ). 296 DIP Chapter and 2: Introduction and Integral Equations 26 Example of SVE (Degenerate) We can occasionally calculate the SVE analytically. Example (s + 2t) f dt = g(s), s. For this kernel K(s, t) = s + 2t we have µ = µ 2 = 2/ 3, µ 3 = µ 4 =... =. u (s) = / 2, u 2 (s) = 3/2 s v = 3/2 t, v 2 = / 2. A solution exists only if g range(k) = span{u, u 2 }, i.e., if g is of the form g(s) = c + c 2 s.

14 296 DIP Chapter and 2: Introduction and Integral Equations 27 The Smoothing Effect The smoother the kernel K, the faster the µ i decay to zero: If the derivatives of order,..., q exist and are continuous, then µ i is approximately O(i q /2 ). The smaller the µ i, the more oscillations (or zero-crossings) in the singular functions u i and v i. v 5 v 6 v 7 v 8 v v 2 v 3 v Since v i µ i u i (s), higher frequencies are damped more than lower frequencies (smoothing) in the forward problem. 296 DIP Chapter and 2: Introduction and Integral Equations 28 The Picard Condition In order that there exists a square integrable solution f to the integral equation, the right-hand side g must satisfy ( ) 2 ui, g <. i= µ i Equivalent condition: g range(k). In plain words: the absolute value of the coefficients (u i, g) must decay faster than the singular values µ i! Main difficulty: a noisy g does not satisfy the PC!

15 296 DIP Chapter and 2: Introduction and Integral Equations 29 Illustration of the Picard Condition 5 g f(s) No noise in g(s) Noise in g(s) µ i u i, g u i, g / µ i The violation of the Picard condition is the simple explanation of the instability of linear inverse problems in the form of first-kind Fredholm integral equations. SVE analysis + Picard plot insight remedy algorithms. 296 DIP Chapter and 2: Introduction and Integral Equations 3 A Problem with no Solution Ursell (974) presented the following innocently-looking problem: f dt =, s. s + t + This problem has no square integrable solution! Expand right-hand side g(s) = in terms of the singular functions: g k (s) = k u i, g u i (s); g g k 2 for k. i= Now consider f k s+t+ dt = g k(s), whose solution f k is f k = k i= u i, g σ i v i. Clearly f k 2 is finite for all k ; but f k 2 for k.

16 296 DIP Chapter and 2: Introduction and Integral Equations 3 Ursell Problem Numerical Results g g k f k f f 2 5 f 3 5 f f 5 2 x 4 f 6 2 x 5 f 7 2 x 6 f DIP Chapter and 2: Introduction and Integral Equations 32 Analytic SVEs are Rare A few cases where analytic SVEs are available, e.g., the Radon transform. But in most applications we must use numerical methods for analysis and solution of the integral equation. The rest of these lectures are devoted to numerical methods! Our analysis has given us an understanding of the difficulties we are facing and they will manifest themselves again in any numerical approach we re using to solve the integral equation.