Eigenvalues of C SMS (16) 1 A. Eigenvalues of A (32) 1 A (16,32) 1 A

Transcription

1 Negative Results for Multilevel Preconditioners in Image Deblurring C. R. Vogel Department of Mathematical Sciences Montana State University Bozeman, MT USA Abstract. A one-dimensional deconvolution problem is discretized and certain multilevel preconditioned iterative methods are applied to solve the resulting linear system. The numerical results suggest that multilevel multiplicative preconditioners may have no advantage over two-level multiplicative preconditioners. In fact, in the numerical experiments they perform worse than comparable two-level preconditioners. 1 Introduction Z In image deblurring, the goal is to estimate the true image u true from noisy, blurred data z(x) = k(x? y) u true (y) dy + (x): (1) Here represents noise in the recorded data, and the convolution kernel function k, which is called the point spread function in this application, is known. Since deconvolution is unstable with respect to perturbations in the data, regularization (i.e., stabilization which retains certain desired features of the true solution) must be applied. To a discretized version of the model equation (1), z = Ku true +, we apply standard (zero order) Tikhonov regularization, i.e., we minimize T (u) = jjku? zjj 2 + jjujj 2 ; > 0: (2) The is the regularization parameter. The resulting minimizer u solves the symmetric, positive denite (SPD) linear system Au = b; A = K K + I; (3) with b = K z. The superscript \" denotes matrix conjugate transpose. The system (3) is often quite large. For example, n = = 65; 536 unknowns arise from two-dimensional image data recorded on a pixel array. For real-time imaging applications, it is necessary to solve these systems very quickly. Due to these size and time constraints, iterative methods are required. Since the coecient matrix A in (3) is SPD, the conjugate gradient (CG) method is appropriate. A tends to be highly ill-conditioned, so preconditioning is needed to increase the convergence rate. (It should be noted that CG can

2 be applied to the unregularized system, and the iteration count becomes the regularization parameter. See for example [4] for details. We will not take this approach here.) Convolution integral operators typically lead to matrices K with Toeplitz structure (see [2]). Circulant preconditioners [7, 2, 3] have proven to be highly eective for large Toeplitz systems. Standard multigrid methods have also been implemented (see for example [1]), but their utility seems limited by the inability to nd a good \smoother", i.e., an iterative scheme which quickly damps high frequency components of solutions on ne grids. It is well-known that wavelet multilevel decompositions tend to \sparsify" the matrices which arise in the discretization of integral operators. The task which remains is to eciently solve the transformed system. Rieder [6] showed that fairly standard block iterative schemes (e.g., Jacobi and Gauss-Seidel iterations) are eective. (The correspondence between block index and grid level makes these methods \multilevel".) Hanke and Vogel [5, 8] extended Rieder's results to two-level preconditioners. Their analysis showed that multiplicative (i.e., Gauss- Seidel-like) preconditioners are generally far superior to additive (Jacobi-like) preconditioners in terms of convergence properties. In particular, they obtained the bounds on the condition numbers, cond(c add?1 1?2 as! 0; (4) cond(c mult?1 2?1 as! 0; (5) where C add and C mult denote the additive and multiplicative preconditioning matrices, respectively. They also presented numerical results indicating that these bounds were sharp. In addition to rapid convergence, these two-level schemes oer other advantages. Toeplitz structure is not required for ecient implementation. Essentially all that is needed is a means of computing matrix-vector products Av and a means of computing coarse-grid projections of vectors. See [8] for details. A disadvantage in certain situations is the need to invert the \coarse-grid representation" of the matrix A, which is the A 11 in equation (8) below. The constants b i in (4)-(5) depend on how well the integral operator is represented on the coarse grid. If is relatively small, then the b i 's also must be relatively small to maintain rapid convergence. This typically means that A 11 must be relatively large, and hence, expensive to invert. The cost of inverting relatively large coarse-grid representation matrices A 11 in the two-level case motivated our interest in multilevel schemes. We conducted a preliminary numerical study which suggested that, at least with obvious implementations, multilevel schemes oer no advantage over two-level schemes. In the nal section, we present the test problem used in this study. This is preceded by a brief sketch of multilevel iterative methods. 2 Multilevel Decomposition and Iterative Schemes Let the columns of V comprise the discrete Haar wavelet basis for R n, n = 2 p, normalized so that V V = I. Note that the discrete Haar wavelet vectors are

3 orthogonal with respect to the usual Euclidean inner product, so orthonormality can be achieved simply by rescaling these vectors. The system (3) can be transformed to ~Ax = ~ b; (6) where ~A = V AV; u = V x; ~ b = V b: (7) Partition the wavelet transformed matrix ~ A into blocks Aij, 1 i; j m, ~A = 2 64 A 11 A 12 : : : A 1m A 21 A 22 : : : A 2m..... A m1 A m2 : : : A mm 3 75 : (8) Multilevel iterative methods can be derived from a natural splitting of these blocks, i.e., ~A = L + D + U; (9) where U consists of the upper triangular blocks A ij ; j > i, D consists of the diagonal blocks A ii, and L = U. For instance, to derive a multilevel additive Jacobi iteration to solve (3), take an initial guess u 0, set x 0 = V u 0, iterate x +1 = D?1 ( ~ b? (L + U)x ); = 0; 1; : : : ; and then back-transform via (7) to obtain an approximate solution to the original system (3). To derive the additive Schwarz iteration presented by Rieder in [6], replace the A mm block in the block diagonal matrix D by I, where I is the identity matrix of the appropriate size. Similarly, one can derive multilevel Jacobi and additive Schwarz preconditioners. To apply such a Jacobi preconditioner to a vector r 2 R n, one rst applies the Haar wavelet transform to this vector, obtaining ~r = V r. One then computes x = D?1 ~r, and then back-transforms via (7) to get u = V D?1 V r = C?1 J r: The matrix C J = V DV is the multilevel Jacobi preconditioning matrix. To derive a multilevel additive Schwarz preconditioner, one again replaces the A mm block in D by I. These Jacobi/additive Schwarz iterative methods neglect o-diagonal terms in the block decomposition (8). Incorporating these o-diagonal terms leads to multiplicative iterative methods. Perhaps the simplest example is multilevel Gauss-Seidel iteration, which can be expressed as u = V x, where x is obtained from x +1 = (L + D)?1 ( ~ b? Ux ): (10) A multilevel multiplicative Schwarz iteration is obtained by again replacing the A mm block of D by I.

4 To obtain symmetric Gauss-Seidel/multiplicative Schwarz iterations, follow (10) by x +2 = (D + U)?1 ( ~ b? Lx +1 ): (11) To obtain the action of a multilevel symmetric Gauss-Seidel preconditioner on a vector r, replace ~ b in (10)-(11) by ~r = V r, set x = 0, and then back-transform via (7) to obtain C?1 SGS r = V (D + U)?1 (V r? L(L + D)?1 V r) = V (D + U)?1 D(L + D)?1 V r: (12) Consequently, the symmetric Gauss-Seidel preconditioning matrix is C SGS = V (L + D)D?1 (D + U)V : (13) Once again replacing the A mm block in D by I yields a corresponding multilevel symmetric multiplicative Schwarz preconditioner (SMS), which we denote by C SMS. 3 Numerical Results A symmetric Toeplitz matrix K = h toeplitz(k) was generated from a discretization k = (k(x 1 ); : : : ; k(x n )) of the Gaussian kernel function k(x) = exp(?x2 = 2 ) p ; 0 x 1; (14) 2 2 Here h = 1=n and x i = (i? 1)h; i = 1; : : : ; n. We selected the kernel width parameter = 0:05 and the number of grid points n = 2 7 = 128. The n n matrix K is extremely ill-conditioned, having eigenvalues which decay to zero like exp(? 2 j 2 ) for large j. The matrix A = K K + I was computed with regularization parameter = 10?3. The distribution of the eigenvalues of A is shown in the upper left subplot of Fig. 1. From this distribution, it can be seen that the eigencomponents corresponding to roughly the smallest 110 eigenvalues of K have been ltered out by the regularization. The value of the regularization parameter is nearly optimal for error-contaminated data whose signal-to-noise ratio is 100. We computed several two- and three-level SMS preconditioners for the system (3). The notation C SMS (r) denotes the two-level (m = 2 in equation (8)) SMS preconditioner with the \coarse grid" block A 11 of size r r. To obtain C SMS (r) the matrix A is transformed and partitioned into 2 2 blocks, cf. (8). The (n? r) (n? r) submatrix A 22 is replaced by I n?r, the splitting (9) is applied, and the right-hand-side of (13) is computed. The eigenvalues of the matrix products C SMS (r)?1 A were computed for coarse grid block sizes r = 16 and r = 32. The distributions of these eigenvalues are displayed in the upper right and lower left subplots of Fig. 1. The reduced relative spread and clustering of these eigenvalues ensures rapid CG convergence. Recall the eigenvalue

5 relative spread can be quantied by the condition number, which is the ratio of the largest to the smallest eigenvalue. With course grid block size r = 16, the condition number of C SMS (r)?1 A is 48:419, while for block size r = 32, the corresponding condition number is 1:4244. As one might expect, doubling the size of the coarse grid block A 11 substantially decreased the condition number. In contrast, the condition number of the matrix A (without preconditioning) is nearly Eigenvalues of A Eigenvalues of C SMS (16) 1 A Eigenvalues of C SMS (32) 1 A 10 0 Eigenvalues of C SMS (16,32) 1 A Fig. 1. Eigenvalue distributions for various multilevel symmetric multiplicative Schwarz preconditioned systems. Let C SMS (r; s) denote the three-level multiplicative Schwarz preconditioner whose coarse grid block A 11 in (8) has size rr and whose second diagonal block A 22 has size (s? r) (s? r). The third diagonal block A 33 is replaced by I n?s. With r = 16 and s = 32, the distribution of the eigenvalues of C SMS (r; s)?1 A is shown in the lower right subplot of Fig. 1. The corresponding condition number is 200:85. This is substantially worse than the result for the two-level preconditioner with coarse grid block size What is surprising is that this is worse than the result for the two-level preconditioner with coarse grid block size This comes in spite of the fact that much more work is required to apply C SMS (16; 32)?1 than to apply C SMS (16)?1. The condition numbers for the various matrix products C?1 A in the example presented above are summarized in column 2 (Test Case 1) of the table below. The I in column 1 indicates that no preconditioning is applied, i.e.,

6 the condition number of A appears across the corresponding row. In column 3 (Test Case 2), results are presented for the same kernel function, cf., equation (14), but the kernel width = :1 is increased, and the regularization parameter = 10?4 is decreased. As in Case 1, the two-level preconditioners outperform comparable three-level preconditioners. The dierence in performance is even more pronounced with the broader kernel than with the narrower kernel. In column 4 (Test Case 3), we present results for the sinc squared kernel, k(x) = (sin(x=)=(x=)) 2, with kernel width = :2, and regularization parameter = 10?5. The results are comparable to those obtained in Case 2. Preconditioner Test Case 1 Test Case 2 Test Case 3 C Condition No. of C?1 A Cond. No. C?1 A Cond. No. C?1 A I C SMS (16) C SMS (32) C SMS (16; 32) Conclusions. For all three test problems, the 3-level SMS preconditioners yielded larger condition numbers and less eigenvalue clustering than comparable 2-level SMS preconditioners. While these test cases may be unrealistically simple, they suggest that no advantage is to be gained by implementing multilevel preconditioners for more realistic (and complicated) problems. References 1. R. Chan, T. Chan, and W. L. Wan, Multigrid for Dierential-Convolution Problems Arising from Image Processing, in Proceedings of the Workshop on Sci. Comput., Springer Verlag, R. H. Chan and M. K. Ng, Conjugate Gradient Method for Toeplitz Systems, SIAM Review, 38 (1996), pp R. H. Chan, M. K. Ng and R. J. Plemmons, Generalization of Strang's preconditioner with applications to Toeplitz least squares problems, Numerical Lin. Alg. and Applications, 3 (1996), pp. 45{ H. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers, Dordrecht, M. Hanke and C. R. Vogel, Two-Level Preconditioners for Regularized Inverse Problems I: Theory, Numer. Math., to appear. 6. A. Rieder, A wavelet multilevel method for ill-posed problems stablized by Tikhonov regularization, Numer. Math., 75 (1997), G. Strang, A Proposal for Toeplitz Matrix Calculations, Stud. Appl. Math, 74 (1986), C. R. Vogel and M. Hanke, Two-level preconditioners for regularized inverse problems II: Implementation and numerical results, SIAM J. Sci. Comput., to appear.