AMS526: Numerical Analysis I (Numerical Linear Algebra)

Transcription

1 AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 19: SVD revisited; Software for Linear Algebra Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 9

2 Outline 1 Computing SVD 2 Software for Linear Algebra Xiangmin Jiao Numerical Analysis I 2 / 9

3 Computing the SVD Intuitive idea for computing SVD of A R m n : Form A A and compute its eigenvalue decomposition A A = V ΛV Let Σ = Λ, i.e., diag( λ 1, λ 2,..., λ n ) Solve system UΣ = AV to obtain U This method can be very efficient if m n. However, it is not very stable, especially for smaller singular values because of the squaring of the condition number For SVD of A, σk σ k = O(ɛ machine A ), where σ k and σ k denote the computed and exact kth singular value If computed from eigenvalue decomposition of A A, σ k σ k = O(ɛ machine A 2 /σ k ), which is problematic if σ k A If one is interested in only relatively large singular values, then using eigenvalue decomposition is not a problem. For general situations, a more stable algorithm is desired. Xiangmin Jiao Numerical Analysis I 3 / 9

4 Computing the SVD Typical algorithm for computing SVD are similar to computation of eigenvalues [ ] 0 A Consider A R m m, then hermitian matrix H = has A 0 eigenvalue decomposition [ ] [ ] [ ] V V V V Σ 0 H =, U U U U 0 Σ where A = UΣV gives the SVD. This approach is stable. In practice, such a reduction is done implicitly without forming the large matrix Typically done in two or more stages: First, reduce to bidiagonal form by applying different orthogonal transformations on left and right, Second, reduce to diagonal form using a variant of QR algorithm or divide-and-conquer algorithm Xiangmin Jiao Numerical Analysis I 4 / 9

5 Generalized Eigenvalue Problem Generalized eigenvalue problem has the form Ax = λbx, where A and B are m m matrices For example, in structural vibration problems, A represents the stiffness matrix, B the mass matrix, and eigenvalues and eigenvectors determine natural frequencies and modes of vibration of structures If A or B is nonsingular, then it can be converted into standard eigenvalue problem (B 1 A)x = λx or (A 1 B)x = (1/λ)x If A and B are both symmetric, preceding transformation loses symmetry and in turn may lose orthogonality of generalized eigenvectors. If B is positive definite, alternative transformation is (L 1 AL T )y = λy, where B = LL T and y = L T x If A and B are both singular or indefinite, then use QZ algorithm to reduce A and B into triangular matrices simultaneously by orthogonal transformation (see Golub and van Loan for detail) Xiangmin Jiao Numerical Analysis I 5 / 9

6 Outline 1 Computing SVD 2 Software for Linear Algebra Xiangmin Jiao Numerical Analysis I 6 / 9

7 Software for Linear Algebra LAPACK: Linear Algebra PACKage ( Standard library for solving linear systems and eigenvalue problems Successor of LINPACK ( and EISPACK ( Depends on BLAS (Basic Linear Algebra Subprograms) Parallel extensions include ScaLAPACK and PLAPACK Note: Uses Fortran conventions for matrix arrangements MATLAB Factorization A: lu(a) and chol(a) Solve Ax = b: x = A\b Uses back/forward substitution for triangular matrices Uses Cholesky factorization for positive-definite matrices Uses LU factorization with column pivoting for nonsymmetric matrices Uses Householder QR for least squares problems Uses some special routines for matrices with special sparsity patterns Uses LAPACK and other packages internally Serial and parallel solvers for sparse matrices (e.g., SuperLU, TAUCS) Xiangmin Jiao Numerical Analysis I 7 / 9

8 Some Commonly Used Functions Example BLAS routines: Matrix-vector multip.: dgemv; Matrix-matrix multip: dgemm LU Factorization Solve linear system Est. cond General Symmetric General Symmetric LAPACK dgetrf dpotrf/dsytrf dgesv dposv/dposvx dgecon LINPACK dgefa dpofa/dsifa dgesl dposl/dsisl dgeco MATLAB lu chol \ \ rcond Linear least squares Eigenvalue/vector SVD QR Solve Rank-deficient General Sym. LAPACK dgeqrf dgels dgelsy dgeev dsyev dgesvd LINPACK dqrdc dqrsl dqrst - - dsvdc MATLAB qr \ \ eig eig svd For BLAS, LINPACK, and LAPACK, first letter s stands for single-precision real, d for double-precision real, c for single-precision complex, and z for double-precision complex. Boldface LAPACK routines are driver routines; others are computational routines. Xiangmin Jiao Numerical Analysis I 8 / 9

9 Using LAPACK Routines in C Programs LAPACK was written in Fortran 77. Special attention is required when calling from C. Key differences between C and Fortran 1 Storage of matrices: column major (Fortran) versus row major (C/C++) 2 Argument passing for subroutines in C and Fortran: pass by reference (Fortran) and pass by value (C/C++) Simple example C code, example.c, for solving linear system using sgesv. See class website for sample code. To compile, issue command cc -o example example.c -llapack -lblas Hint: To find a function name, refer to LAPACK Users Guide. To find out arguments for a given function, search on netlib.org Xiangmin Jiao Numerical Analysis I 9 / 9