Matrix Multiplication Chapter III General Linear Systems

Transcription

1 Matrix Multiplication Chapter III General Linear Systems By Gokturk Poyrazoglu The State University of New York at Buffalo BEST Group Winter Lecture Series

2 General Outline 1. Triangular Systems 2. The LU Factorization 3. Pivoting

3 Triangular Systems Outline 1. Row oriented Forward Substitution 2. Row oriented Back Substitution 3. Column oriented Forward Substitution 4. Column oriented Back Substitution 5. Multiple Right-Hand Sides 6. Nonsquare Triangular Systems Solving 7. Algebra of Triangular Matrices

4 Forward Substitution Consider a lower triangular system The unknowns are: General Procedure:

5 Row Oriented Forward Substitution L is a square lower triangular matrix b is a vector Overwrite b with the solution of Lx=b

6 Row Oriented Back Substitution U is a square upper triangular matrix b is a vector Solution x: Overwrite b with the solution of Ux=b

7 Column Oriented Versions Example: Solve for x1 (x1=3); remove from the equations by

8 Column Oriented Forward Substitution L is a square lower triangular matrix b is a vector Overwrite b with the solution of Lx=b

9 Column Oriented Back Substitution U is a square upper triangular matrix b is a vector Overwrite b with the solution of Ux=b

10 Multiple Right-Hand Sides Consider block matrices L, X, and B Solve L 11 X 1 =B 1 for X 1. Remove X 1 from block equations as follows :

11 Nonsquare Triangular System Solving Consider a block matrix L where m > n Assume L 11 is nonsingular and lower triangular. Solve L 11 x=b 1 for x Then x should solve the system L 21 L 1 11 b 1 = b 2 Otherwise, there is no solution to the overall system.

12 Triangular Matrix Properties Unit Triangular Matrix: Triangular matrix with 1 s on the diagonal. Other Properties Inverse of an upper triangular is an upper triangular matrix. The product of two upper triangular is an upper triangular. Inverse of a unit upper triangular is a unit upper triangular. The product of two unit upper triangular is an upper triangular.

13 The LU Factorization Outline Background Gauss transformations Application Upper Triangularizing Existence of LU Other versions of LU Rectangular Matrix Block LU

14 Background Example : Multiply the 1 st equation by 2, Subtract it from the 2 nd equation. Matrix Notation:

15 Gauss Transformations Consider a vector V (a stack of 2 block vectors v1 and v2) Suppose, and define Gauss Vector Define Gauss Transformation matrix

16 Application Consider a matrix C, and apply an outer product update Repeat the process; Algorithm

17 Upper Triangularizing Consider a square matrix-a, then Example: Note :Diagonal components of A (pivots) should be zero

18 Existence of LU If no zero pivots are encountered; then and so that LU factorization does NOT exist if That means k th pivot is zero. is singular.

19 Construction of matrix-l Consider the example k th column of L is defined by the multipliers from k th step

20 Outer Product Point of View Consider a matrix A as; Gauss Elimination results: where Hence;

21 LU Factorization of a Rectangular Matrix Such matrices L and U exist if Examples: is nonsingular. Algorithm for the 1 st example: Operation: nr 2 -r 3 /3 flops

22 Roundoff Error in Gaussian Elimination

23 Triangular Solving with Inexact Triangles If a small pivot is encountered, then we can expect large numbers to be present in L and U. Example : Solution is in contrast to exact solution

24 Pivoting Outline Interchange Permutations Partial Pivoting Complete pivoting Rook Pivoting

25 Interchange Permutations Consider a permutation matrix: If we multiply matrix A from the left, rows 1 and 4 interchanged If we multiply matrix A from the right; columns 1 and 4 interchanged

26 Partial Pivoting Motivation: To guarantee that no multiplier is greater than 1 in absolute value. Example : Consider matrix-a, get the largest entry in the first column to a11

27 Partial Pivoting Particular row interchange strategy is called partial pivoting. In general : where U is an upper triangular, and no multiplier is greater than 1 in absolute value as a consequence of partial pivoting.

28 Complete Pivoting Partial Pivoting was scanning the current subcolumn for maximal element; Complete Pivoting scans the current submatrix to find the largest entry to pivot. Hence;

29 Complete Pivoting Properties Gaussian Elimination with Complete Pivoting is STABLE. No significant reason to choose Complete pivoting over Partial pivoting. Only if matrix A is rank deficient. In principal, when the pivot of current submatrix is ZERO at the beginning of step r+1; that indicates that the rank(a) =r; In practice, encountering an exactly zero pivot is lesslikely.

30 Rook Pivoting Computes the factorization : Choosing Pivot: Search for an element of current submatrix that is maximal in BOTH its ROW and COLUMN. Complete Pivoting Rook Pivoting Candidate

31 Comparison Flops: Partial pivoting : O(n 2 ) Complete Pivoting O(n 3 ) Rook Pivoting O(n 2 ) Rook has same level of reliability as complete pivoting and represents same O(n 2 ) overhead as partial pivoting. Complete Pivoting may be used for rank identification in principal. All pivoting methods are stable.

32 Extra Proof Slides

33 Proof of LU Factorization Theorem (slide 18)

34 Proof of Slide 23