GAUSSIAN ELIMINATION - REVISITED

Transcription

1 GAUSSIAN ELIMINATION - REVISITED Consider solving the linear system 2x 1 + x 2 x 3 +2x 4 = 5 4x 1 +5x 2 3x 3 +6x 4 = 9 2x 1 +5x 2 2x 3 +6x 4 = 4 4x 1 +11x 2 4x 3 +8x 4 = 2 by Gaussian elimination without pivoting. We denote this linear system by Ax = b. Theaugmentedmatrix for this system is [A b] = To eliminate x 1 from equations 2, 3, and 4, use multipliers m 2,1 =2, m 3,1 = 1, m 4,1 =

2 To eliminate x 1 from equations 2, 3, and 4, use multipliers m 2,1 =2, m 3,1 = 1, m 4,1 =2 This will introduce zeros into the positions below the diagonal in column 1, yielding To eliminate x 2 from equations 3 and 4, use multipliers m 3,2 =2, m 4,2 =3 This reduces the augmented matrix to

3 To eliminate x 3 from equation 4, use the multiplier m 4,3 = 1 This reduces the augmented matrix to Return this to the familiar linear system 2x 1 + x 2 x 3 +2x 4 = 5 3x 2 x 3 +2x 4 = 1 x 3 +4x 4 = 11 2x 4 = 6 Solving by back substitution, we obtain x 4 =3, x 3 =1, x 2 = 2, x 1 =1

4 There is a surprising result involving matrices associated with this elimination process. Introduce the uppertriangularmatrix U = which resulted from the elimination process. introduce the lower triangular matrix L = m 2, m 3,1 m 3,2 1 0 m 4,1 m 4,2 m 4,3 1 = Then This uses the multipliers introduced in the elimination process. Then A = LU =

5 In general, when the process of Gaussian elimination without pivoting is applied to solving a linear system Ax = b, weobtaina = LU with L and U constructed as above. For the case in which partial pivoting is used, we obtain the slightly modified result LU = PA where L and U are constructed as before and P is a permutation matrix. For example, consider Then PA = P = a 1,1 a 1,2 a 1,3 a 1,4 a 2,1 a 2,2 a 2,3 a 2,4 a 3,1 a 3,2 a 3,3 a 3,4 a 4,1 a 4,2 a 4,3 a 4,4 = A 3, A 1, A 4, A 2,

6 PA = = A 3, A 1, A 4, A 2, a 1,1 a 1,2 a 1,3 a 1,4 a 2,1 a 2,2 a 2,3 a 2,4 a 3,1 a 3,2 a 3,3 a 3,4 a 4,1 a 4,2 a 4,3 a 4,4 The matrix PA is obtained from A by switching around rows of A. TheresultLU = PA means that the LUfactorization is valid for the matrix A with its rows suitably permuted.

7 Consequences: If we have a factorization A = LU with L lower triangular and U upper triangular, then we can solve the linear system Ax = b in a relatively straightforward way. The linear system can be written as LUx = b Write this as a two stage process: Lg = b, Ux = g The system Lg = b is a lower triangular system g 1 = b 1 2,1 g 1 + g 2 = b 2 3,1 g 1 + 3,2 g 2 + g 3 =. b 3 n,1 g 1 + n,n 1 g n 1 + g n = b n We solve it by forward substitution. Then we solve the upper triangular system Ux = g by back substitution.

8 VARIANTS OF GAUSSIAN ELIMINATION If no partial pivoting is needed, then we can look for a factorization A = LU without going thru the Gaussian elimination process. For example, suppose A is 4 4. We write = a 1,1 a 1,2 a 1,3 a 1,4 a 2,1 a 2,2 a 2,3 a 2,4 a 3,1 a 3,2 a 3,3 a 3,4 a 4,1 a 4,2 a 4,3 a 4, , ,1 3, ,1 4,2 4,3 1 u 1,1 u 1,2 u 1,3 u 1,4 0 u 2,2 u 2,3 u 2,4 0 0 u 3,3 u 3, u 4,4 To find the elements n i,j o and n ui,j o, we multiply the right side matrices L and U and match the results with the corresponding elements in A.

9 Multiplying the first row of L times all of the columns of U leads to u 1,j = a 1,j, j =1, 2, 3, 4 Then multiplying rows 2, 3, 4 times the first column of U yields i,1 u 1,1 = a i,1, i =2, 3, 4 and we can solve for n 2,1, 3,1, 4,1 o. We can continue this process, finding the second row of U and then the second column of L, and so on. For example, to solve for 4,3,weneedtosolveforitin 4,1 u 1,3 + 4,2 u 2,3 + 4,3 u 3,3 = a 4,3 Why do this? A hint of an answer is given by this last equation. If we had an n n matrix A, thenwe would find n,n 1 by solving for it in the equation n,1 u 1,n 1 + n,2 u 2,n n,n 1 u n 1,n 1 = a n,n 1 n,n 1 = a n,n 1 h n,1 u 1,n n,n 2 u n 2,n 1 i u n 1,n 1

10 Embedded in this formula we have a dot product. This is in fact typical of this process, with the length of the inner products varying from one position to another. Recalling 2.4 and the discussion of dot products, we canevaluatethislastformulabyusingahigherprecision arithmetic and thus avoid many rounding errors. This leads to a variant of Gaussian elimination in which there are far fewer rounding errors. With ordinary Gaussian elimination, the number of rounding errors is proportional to n 3. This reduces the number of rounding errors, with the number now being proportional to only n 2.Thiscanleadtomajor increases in accuracy, especially for matrices A which are very sensitive to small changes.

11 TRIDIAGONAL MATRICES A = b 1 c a 2 b 2 c a 3 b 3 c a n 1 b n 1 c n 1 0 a n b n These occur very commonly in the numerical solution of partial differential equations, as well as in other applications (e.g. computing interpolating cubic spline functions). We factor A = LU, as before. But now L and U take very simple forms. Before proceeding, we note with an example that the same may not be true of the matrix inverse.

12 EXAMPLE Define an n n tridiagonal matrix A = Then A 1 is given by ³ A n 1 n i,j =max{i, j} Thus the sparse matrix A can (and usually does) have a dense inverse.

13 We factor A = LU, with L = U = α α α n α n 1 β 1 c β 2 c β 3 c β n 1 c n β n Multiply these and match coefficients with A to find {α i,γ i }.

14 By doing a few multiplications of rows of L times columns of U, we obtain the general pattern as follows. β 1 = b 1 :row1oflu α 2 β 1 = a 2, α 2 c 1 + β 2 = b 2 :row2oflu. α n β n 1 = a n, α n c n 1 + β n = b n :rown of LU These are straightforward to solve. α j = β 1 = b 1 a j, β j 1 β j = b j α j c j 1, j =2,...,n

15 To solve the linear system Ax = f or LUx = f instead solve the two triangular systems Lg = f, Ux = g Solving Lg = f: Solving Ux = g: g 1 = f 1 g j = f j α j g j 1, x n = g n β n j =2,...,n x j = g j c j x j+1 β j, j = n 1,...,1 See the numerical example on page 278.

16 OPERATIONS COUNT Factoring A = LU. Additions: n 1 Multiplications: n 1 Divisions: n 1 Solving Lz = f and Ux = z: Additions: 2n 2 Multiplications: 2n 2 Divisions: n Thus the total number of arithmetic operations is approximately 3n to factor A; andittakesabout5n to solve the linear system using the factorization of A. If we had A 1 at no cost, what would it cost to compute x = A 1 f? x i = nx ³ A 1 j=1 i,j f j, i =1,...,n

17 MATLAB MATRIX OPERATIONS To obtain the LU-factorization of a matrix, including the use of partial pivoting, use the Matlab command lu. In particular, [L, U, P ]=lu(x) returns the lower triangular matrix L, upper triangular matrix U, and permutation matrix P so that PX = LU