Direct Methods for Solving Linear Systems. Matrix Factorization
|
|
|
- Brendan Greer
- 8 years ago
- Views:
Transcription
1 Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011 Brooks/Cole, Cengage Learning
2 Outline 1 Computation Cost Rationale & Basic Solution Strategy Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 2 / 46
3 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 2 / 46
4 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 2 / 46
5 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix 4 The LU Factorization Algorithm Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 2 / 46
6 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix 4 The LU Factorization Algorithm 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 2 / 46
7 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix 4 The LU Factorization Algorithm 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 3 / 46
8 Matrix Factorization Background Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 4 / 46
9 Matrix Factorization Background Gaussian elimination is the principal tool in the direct solution of linear systems of equations. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 4 / 46
10 Matrix Factorization Background Gaussian elimination is the principal tool in the direct solution of linear systems of equations. We will now see that the steps used to solve a system of the form Ax = b can be used to factor a matrix. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 4 / 46
11 Matrix Factorization Background Gaussian elimination is the principal tool in the direct solution of linear systems of equations. We will now see that the steps used to solve a system of the form Ax = b can be used to factor a matrix. The factorization is particularly useful when it has the form A = LU, where L is lower triangular and U is upper triangular. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 4 / 46
12 Matrix Factorization Background Gaussian elimination is the principal tool in the direct solution of linear systems of equations. We will now see that the steps used to solve a system of the form Ax = b can be used to factor a matrix. The factorization is particularly useful when it has the form A = LU, where L is lower triangular and U is upper triangular. Although not all matrices have this type of representation, many do that occur frequently in the application of numerical techniques. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 4 / 46
13 Matrix Factorization Computational Cost Considerations Gaussian elimination applied to an arbitrary linear system Ax = b requires O(n 3 /3) arithmetic operations to determine x. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 5 / 46
14 Matrix Factorization Computational Cost Considerations Gaussian elimination applied to an arbitrary linear system Ax = b requires O(n 3 /3) arithmetic operations to determine x. However, to solve a linear system that involves an upper-triangular system requires only backward substitution, which takes O(n 2 ) operations. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 5 / 46
15 Matrix Factorization Computational Cost Considerations Gaussian elimination applied to an arbitrary linear system Ax = b requires O(n 3 /3) arithmetic operations to determine x. However, to solve a linear system that involves an upper-triangular system requires only backward substitution, which takes O(n 2 ) operations. The number of operations required to solve a lower-triangular systems is similar. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 5 / 46
16 Matrix Factorization Solution Strategy Suppose that A has been factored into the triangular form A = LU, where L is lower triangular and U is upper triangular. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 6 / 46
17 Matrix Factorization Solution Strategy Suppose that A has been factored into the triangular form A = LU, where L is lower triangular and U is upper triangular. Then we can solve for x more easily by using a two-step process: Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 6 / 46
18 Matrix Factorization Solution Strategy Suppose that A has been factored into the triangular form A = LU, where L is lower triangular and U is upper triangular. Then we can solve for x more easily by using a two-step process: First we let y = Ux and solve the lower triangular system Ly = b for y. Since L is triangular, determining y from this equation requires only O(n 2 ) operations. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 6 / 46
19 Matrix Factorization Solution Strategy Suppose that A has been factored into the triangular form A = LU, where L is lower triangular and U is upper triangular. Then we can solve for x more easily by using a two-step process: First we let y = Ux and solve the lower triangular system Ly = b for y. Since L is triangular, determining y from this equation requires only O(n 2 ) operations. Once y is known, the upper triangular system Ux = y requires only an additional O(n 2 ) operations to determine the solution x. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 6 / 46
20 Matrix Factorization Solution Strategy Suppose that A has been factored into the triangular form A = LU, where L is lower triangular and U is upper triangular. Then we can solve for x more easily by using a two-step process: First we let y = Ux and solve the lower triangular system Ly = b for y. Since L is triangular, determining y from this equation requires only O(n 2 ) operations. Once y is known, the upper triangular system Ux = y requires only an additional O(n 2 ) operations to determine the solution x. Solving a linear system Ax = b in factored form means that the number of operations needed to solve the system Ax = b is reduced from O(n 3 /3) to O(2n 2 ). Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 6 / 46
21 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix 4 The LU Factorization Algorithm 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 7 / 46
22 Matrix Factorization Constructing L & U First, suppose that Gaussian elimination can be performed on the system Ax = b without row interchanges. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 8 / 46
23 Matrix Factorization Constructing L & U First, suppose that Gaussian elimination can be performed on the system Ax = b without row interchanges. With the notation used earlier, this is equivalent to having nonzero pivot elements a (i) ii, for each i = 1, 2,...,n. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 8 / 46
24 Matrix Factorization Constructing L & U First, suppose that Gaussian elimination can be performed on the system Ax = b without row interchanges. With the notation used earlier, this is equivalent to having nonzero pivot elements a (i) ii, for each i = 1, 2,...,n. The first step in the Gaussian elimination process consists of performing, for each j = 2, 3,..., n, the operations (E j m j,1 E 1 ) (E j ), where m j,1 = a(1) j1 a (1) 11 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 8 / 46
25 Matrix Factorization Constructing L & U First, suppose that Gaussian elimination can be performed on the system Ax = b without row interchanges. With the notation used earlier, this is equivalent to having nonzero pivot elements a (i) ii, for each i = 1, 2,...,n. The first step in the Gaussian elimination process consists of performing, for each j = 2, 3,..., n, the operations (E j m j,1 E 1 ) (E j ), where m j,1 = a(1) j1 a (1) 11 These operations transform the system into one in which all the entries in the first column below the diagonal are zero. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 8 / 46
26 Matrix Factorization: Constructing L & U (Cont d) The system of operations in (E j m j,1 E 1 ) (E j ), where m j,1 = a(1) j1 a (1) 11 can be viewed in another way. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 9 / 46
27 Matrix Factorization: Constructing L & U (Cont d) The system of operations in (E j m j,1 E 1 ) (E j ), where m j,1 = a(1) j1 a (1) 11 can be viewed in another way. It is simultaneously accomplished by multiplying the original matrix A on the left by the matrix m M (1) = m n Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 9 / 46
28 Matrix Factorization: Constructing L & U (Cont d) The system of operations in (E j m j,1 E 1 ) (E j ), where m j,1 = a(1) j1 a (1) 11 can be viewed in another way. It is simultaneously accomplished by multiplying the original matrix A on the left by the matrix m M (1) = m n This is called the first Gaussian transformation matrix. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 9 / 46
29 Matrix Factorization Constructing L & U (Cont d) We denote the product of this matrix with A (1) A by A (2) and with b by b (2), Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 10 / 46
30 Matrix Factorization Constructing L & U (Cont d) We denote the product of this matrix with A (1) A by A (2) and with b by b (2), so A (2) x = M (1) Ax = M (1) b = b (2) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 10 / 46
31 Matrix Factorization Constructing L & U (Cont d) We denote the product of this matrix with A (1) A by A (2) and with b by b (2), so A (2) x = M (1) Ax = M (1) b = b (2) In a similar manner we construct M (2), the identity matrix with the entries below the diagonal in the second column replaced by the negatives of the multipliers m j,2 = a(2) j2 a (2) 22. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 10 / 46
32 Matrix Factorization Constructing L & U (Cont d) The product of M (2) with A (2) has zeros below the diagonal in the first two columns, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 11 / 46
33 Matrix Factorization Constructing L & U (Cont d) The product of M (2) with A (2) has zeros below the diagonal in the first two columns, and we let A (3) x = M (2) A (2) x = M (2) M (1) Ax = M (2) M (1) b = b (3) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 11 / 46
34 Matrix Factorization Constructing L & U (Cont d) In general, with A (k) x = b (k) already formed, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 12 / 46
35 Matrix Factorization Constructing L & U (Cont d) In general, with A (k) x = b (k) already formed, multiply by the kth Gaussian transformation matrix M (k). = m..... k+1,k m n,k Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 12 / 46
36 Matrix Factorization Constructing L & U (Cont d) to obtain A (k+1) x = M (k) A (k) x Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
37 Matrix Factorization Constructing L & U (Cont d) to obtain A (k+1) x = M (k) A (k) x = M (k) M (1) Ax Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
38 Matrix Factorization Constructing L & U (Cont d) to obtain A (k+1) x = M (k) A (k) x = M (k) M (1) Ax = M (k) b (k) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
39 Matrix Factorization Constructing L & U (Cont d) to obtain A (k+1) x = M (k) A (k) x = M (k) M (1) Ax = M (k) b (k) = b (k+1) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
40 Matrix Factorization Constructing L & U (Cont d) to obtain A (k+1) x = M (k) A (k) x = M (k) M (1) Ax = M (k) b (k) = b (k+1) = M (k) M (1) b Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 13 / 46
41 Matrix Factorization Constructing L & U (Cont d) The process ends with the formation of A (n) x = b (n), where A (n) is the upper triangular matrix a (1) 11 a (1) 12 a (1) 1n 0 a (2)... A (n) 22. = (n 1) a n 1,n 0 0 a (n) n,n Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 14 / 46
42 Matrix Factorization Constructing L & U (Cont d) The process ends with the formation of A (n) x = b (n), where A (n) is the upper triangular matrix a (1) 11 a (1) 12 a (1) 1n 0 a (2)... A (n) 22. = (n 1) a n 1,n 0 0 a (n) n,n given by A (n) = M (n 1) M (n 2) M (1) A Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 14 / 46
43 Matrix Factorization Constructing L & U (Cont d) This process forms the U = A (n) portion of the matrix factorization A = LU. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
44 Matrix Factorization Constructing L & U (Cont d) This process forms the U = A (n) portion of the matrix factorization A = LU. To determine the complementary lower triangular matrix L, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
45 Matrix Factorization Constructing L & U (Cont d) This process forms the U = A (n) portion of the matrix factorization A = LU. To determine the complementary lower triangular matrix L, first recall the multiplication of A (k) x = b (k) by the Gaussian transformation of M (k) used to obtain: A (k+1) x = M (k) A (k) x = M (k) b (k) = b (k+1), Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
46 Matrix Factorization Constructing L & U (Cont d) This process forms the U = A (n) portion of the matrix factorization A = LU. To determine the complementary lower triangular matrix L, first recall the multiplication of A (k) x = b (k) by the Gaussian transformation of M (k) used to obtain: A (k+1) x = M (k) A (k) x = M (k) b (k) = b (k+1), where M (k) generates the row operations (E j m j,k E k ) (E j ), for j = k + 1,...,n. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 15 / 46
47 Matrix Factorization Constructing L & U (Cont d) To reverse the effects of this transformation and return to A (k) requires that the operations (E j + m j,k E k ) (E j ) be performed for each j = k + 1,...,n. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 16 / 46
48 Matrix Factorization Constructing L & U (Cont d) To reverse the effects of this transformation and return to A (k) requires that the operations (E j + m j,k E k ) (E j ) be performed for each j = k + 1,...,n. This is equivalent to multiplying by [ M (k)] 1 : [ L (k) = M (k)] 1. = m..... k+1,k m n,k Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 16 / 46
49 Matrix Factorization Constructing L & U (Cont d) The lower-triangular matrix L in the factorization of A, then, is the product of the matrices L (k) : m L = L (1) L (2) L (n 1) = m n1 m n,n 1 1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 17 / 46
50 Matrix Factorization Constructing L & U (Cont d) The lower-triangular matrix L in the factorization of A, then, is the product of the matrices L (k) : m L = L (1) L (2) L (n 1) = m n1 m n,n 1 1 since the product of L with the upper-triangular matrix U = M (n 1) M (2) M (1) A gives Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 17 / 46
51 Matrix Factorization Constructing L & U (Cont d) LU = L (1) L (2) L (n 3) L (n 2) L (n 1) M (n 1) M (n 2) M (n 3) M (2) M (1) A Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
52 Matrix Factorization Constructing L & U (Cont d) LU = L (1) L (2) L (n 3) L (n 2) L (n 1) M (n 1) M (n 2) M (n 3) M (2) M (1) A = [M (1) ] 1 [M (2) ] 1 [M (n 2) ] 1 [M (n 1) ] 1 M (n 1) M (n 2) M (2) M (1) A Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
53 Matrix Factorization Constructing L & U (Cont d) LU = L (1) L (2) L (n 3) L (n 2) L (n 1) M (n 1) M (n 2) M (n 3) M (2) M (1) A = [M (1) ] 1 [M (2) ] 1 [M (n 2) ] 1 [M (n 1) ] 1 M (n 1) M (n 2) M (2) M (1) A = A Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
54 Matrix Factorization Constructing L & U (Cont d) LU = L (1) L (2) L (n 3) L (n 2) L (n 1) M (n 1) M (n 2) M (n 3) M (2) M (1) A = [M (1) ] 1 [M (2) ] 1 [M (n 2) ] 1 [M (n 1) ] 1 M (n 1) M (n 2) M (2) M (1) A = A We now state a theorem which follows from these observations. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 18 / 46
55 Matrix Factorization Theorem If Gaussian elimination can be performed on the linear system Ax = b without row interchanges, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 19 / 46
56 Matrix Factorization Theorem If Gaussian elimination can be performed on the linear system Ax = b without row interchanges, then the matrix A can be factored into the product of a lower-triangular matrix L and an upper-triangular matrix U, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 19 / 46
57 Matrix Factorization Theorem If Gaussian elimination can be performed on the linear system Ax = b without row interchanges, then the matrix A can be factored into the product of a lower-triangular matrix L and an upper-triangular matrix U, that is, A = LU, where m ji = a (i) ji /a (i) ii, U = a (1) 11 a (1) 12 a (1) 1n 0 a (2) a (n 1) n 1,n 0 0 a (n) n,n m L = m n1 m n,n 1 1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 19 / 46
58 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix 4 The LU Factorization Algorithm 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 20 / 46
59 Matrix Factorization Example (a) Determine the LU factorization for matrix A in the linear system Ax = b, where A = and b = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 21 / 46
60 Matrix Factorization Example (a) Determine the LU factorization for matrix A in the linear system Ax = b, where A = and b = (b) Then use the factorization to solve the system x 1 + x 2 + 3x 4 = 8 2x 1 + x 2 x 3 + x 4 = 7 3x 1 x 2 x 3 + 2x 4 = 14 x 1 + 2x 2 + 3x 3 x 4 = 7 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 21 / 46
61 Matrix Factorization: 4 4 Example Part (a) Solution (1/2) The original system was considered under Gaussian Elimination Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 22 / 46
62 Matrix Factorization: 4 4 Example Part (a) Solution (1/2) The original system was considered under Gaussian Elimination where we saw that the sequence of operations (E 2 2E 1 ) (E 2 ) (E 3 3E 1 ) (E 3 ) (E 4 ( 1)E 1 ) (E 4 ) (E 3 4E 2 ) (E 3 ) (E 4 ( 3)E 2 ) (E 4 ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 22 / 46
63 Matrix Factorization: 4 4 Example Part (a) Solution (1/2) The original system was considered under Gaussian Elimination where we saw that the sequence of operations (E 2 2E 1 ) (E 2 ) (E 3 3E 1 ) (E 3 ) (E 4 ( 1)E 1 ) (E 4 ) (E 3 4E 2 ) (E 3 ) (E 4 ( 3)E 2 ) (E 4 ) converts the system to the triangular system x 1 + x 2 + 3x 4 = 4 x 2 x 3 5x 4 = 7 3x x 4 = 13 13x 4 = 13 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 22 / 46
64 Matrix Factorization: 4 4 Example Part (a) Solution (2/2) The multipliers m ij and the upper triangular matrix produce the factorization A = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 23 / 46
65 Matrix Factorization: 4 4 Example Part (a) Solution (2/2) The multipliers m ij and the upper triangular matrix produce the factorization A = = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 23 / 46
66 Matrix Factorization: 4 4 Example Part (a) Solution (2/2) The multipliers m ij and the upper triangular matrix produce the factorization A = = = LU Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 23 / 46
67 Matrix Factorization: 4 4 Example Part (b) Solution (1/3) To solve Ax = LUx = = x 1 x 2 x 3 x 4 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 24 / 46
68 Matrix Factorization: 4 4 Example Part (b) Solution (1/3) To solve Ax = LUx = = we first introduce the substitution y = Ux x 1 x 2 x 3 x 4 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 24 / 46
69 Matrix Factorization: 4 4 Example Part (b) Solution (1/3) To solve Ax = LUx = = we first introduce the substitution y = Ux. Then b = L(Ux) = Ly. x 1 x 2 x 3 x 4 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 24 / 46
70 Matrix Factorization: 4 4 Example Part (b) Solution (2/3) First, solve Ly = b (where y = Ux: Ly = y 1 y 2 y 3 y 4 = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 25 / 46
71 Matrix Factorization: 4 4 Example Part (b) Solution (2/3) First, solve Ly = b (where y = Ux: Ly = y 1 y 2 y 3 y 4 = This system is solved for y by a simple forward-substitution process: y 1 = 8 2y 1 + y 2 = 7 y 2 = 7 2y 1 = 9. 3y 1 + 4y 2 + y 3 = 14 y 3 = 14 3y 1 4y 2 = 26 y 1 3y 2 + y 4 = 7 y 4 = 7 + y 1 + 3y 2 = 26 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 25 / 46
72 Matrix Factorization: 4 4 Example Part (b) Solution (3/3) We then solve Ux = y for x, the solution of the original system; Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 26 / 46
73 Matrix Factorization: 4 4 Example Part (b) Solution (3/3) We then solve Ux = y for x, the solution of the original system; that is, x x x 3 = x 4 26 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 26 / 46
74 Matrix Factorization: 4 4 Example Part (b) Solution (3/3) We then solve Ux = y for x, the solution of the original system; that is, x x x 3 = x 4 26 Using backward substitution we obtain x 4 = 2, x 3 = 0, x 2 = 1, x 1 = 3. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 26 / 46
75 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix 4 The LU Factorization Algorithm 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 27 / 46
76 LU Factorization Algorithm (1/3) To factor the n n matrix A = [a ij ] into the product of the lower-triangular matrix L = [l ij ] and the upper-triangular matrix U = [u ij ]; that is, A = LU, where the main diagonal of either L or U consists of all ones: Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 28 / 46
77 LU Factorization Algorithm (1/3) To factor the n n matrix A = [a ij ] into the product of the lower-triangular matrix L = [l ij ] and the upper-triangular matrix U = [u ij ]; that is, A = LU, where the main diagonal of either L or U consists of all ones: INPUT dimension n; the entries a ij, 1 i, j n of A; the diagonal l 11 = = l nn = 1 of L or the diagonal u 11 = = u nn = 1 of U. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 28 / 46
78 LU Factorization Algorithm (1/3) To factor the n n matrix A = [a ij ] into the product of the lower-triangular matrix L = [l ij ] and the upper-triangular matrix U = [u ij ]; that is, A = LU, where the main diagonal of either L or U consists of all ones: INPUT dimension n; the entries a ij, 1 i, j n of A; the diagonal l 11 = = l nn = 1 of L or the diagonal u 11 = = u nn = 1 of U. OUTPUT the entries l ij, 1 j i, 1 i n of L and the entries, u ij, i j n, 1 i n of U. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 28 / 46
79 LU Factorization Algorithm (2/3) Step 1 Select l 11 and u 11 satisfying l 11 u 11 = a 11 If l 11 u 11 = 0 then OUTPUT ( Factorization impossible ) STOP Step 2 For j = 2,...,n set u 1j = a 1j /l 11 (First row of U) l j1 = a j1 /u 11 (First column of L) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
80 LU Factorization Algorithm (2/3) Step 1 Select l 11 and u 11 satisfying l 11 u 11 = a 11 If l 11 u 11 = 0 then OUTPUT ( Factorization impossible ) STOP Step 2 For j = 2,...,n set u 1j = a 1j /l 11 (First row of U) l j1 = a j1 /u 11 (First column of L) Step 3 For i = 2,...,n 1 do Steps 4 and 5: Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
81 LU Factorization Algorithm (2/3) Step 1 Select l 11 and u 11 satisfying l 11 u 11 = a 11 If l 11 u 11 = 0 then OUTPUT ( Factorization impossible ) STOP Step 2 For j = 2,...,n set u 1j = a 1j /l 11 (First row of U) l j1 = a j1 /u 11 (First column of L) Step 3 For i = 2,...,n 1 do Steps 4 and 5: Step 4 Select l ii and u ii satisfying l ii u ii = a ii i 1 k=1 l iku ki If l ii u ii = 0 then OUTPUT ( Factorization impossible ) STOP Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
82 LU Factorization Algorithm (2/3) Step 1 Select l 11 and u 11 satisfying l 11 u 11 = a 11 If l 11 u 11 = 0 then OUTPUT ( Factorization impossible ) STOP Step 2 For j = 2,...,n set u 1j = a 1j /l 11 (First row of U) l j1 = a j1 /u 11 (First column of L) Step 3 For i = 2,...,n 1 do Steps 4 and 5: Step 4 Step 5 Select l ii and u ii satisfying l ii u ii = a ii i 1 k=1 l iku ki If l ii u ii = 0 then OUTPUT ( Factorization impossible ) STOP For j = i + 1,...,n set u ij = 1 l ii [ a ij i 1 k=1 l iku kj ] l ji = 1 u ii [ a ji i 1 k=1 l jku ki ] (ith row of U) (ith column of L) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 29 / 46
83 LU Factorization Algorithm (3/3) Step 6 Select l nn and u nn satisfying l nn u nn = a nn n 1 k=1 l nku kn (Note: If l nn u nn = 0, then A = LU but A is singular) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 30 / 46
84 LU Factorization Algorithm (3/3) Step 6 Select l nn and u nn satisfying l nn u nn = a nn n 1 k=1 l nku kn (Note: If l nn u nn = 0, then A = LU but A is singular) Step 7 OUTPUT (l ij for j = 1,...,i and i = 1,...,n) OUTPUT (u ij for j = i,..., n and i = 1,...,n) STOP Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 30 / 46
85 Matrix Factorization Using the LU Factorization to solve Ax = b Once the matrix factorization is complete, Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
86 Matrix Factorization Using the LU Factorization to solve Ax = b Once the matrix factorization is complete, the solution to a linear system of the form Ax = LUx = b Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
87 Matrix Factorization Using the LU Factorization to solve Ax = b Once the matrix factorization is complete, the solution to a linear system of the form Ax = LUx = b is found by first letting y = Ux Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
88 Matrix Factorization Using the LU Factorization to solve Ax = b Once the matrix factorization is complete, the solution to a linear system of the form Ax = LUx = b is found by first letting and solving for y. y = Ux Ly = b Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 31 / 46
89 Matrix Factorization Using the LU Factorization (Cont d) Since L is lower triangular, we have y 1 = b 1 l 11 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 32 / 46
90 Matrix Factorization Using the LU Factorization (Cont d) Since L is lower triangular, we have y 1 = b 1 and, for each l 11 i = 2, 3,...,n, y i = 1 i 1 b i l ij y j l ii j=1 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 32 / 46
91 Matrix Factorization Using the LU Factorization (Cont d) Since L is lower triangular, we have y 1 = b 1 and, for each l 11 i = 2, 3,...,n, y i = 1 i 1 b i l ij y j l ii After y is found by this forward-substitution process, the upper-triangular system Ux = y is solved for x by backward substitution using the equations x n = y n u nn and x i = 1 u ii j=1 y i n j=i+1 u ij x j Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 32 / 46
92 Outline 1 Computation Cost Rationale & Basic Solution Strategy 2 Constructing the Matrix Factorization 3 Example: LU Factorization of a 4 4 Matrix 4 The LU Factorization Algorithm 5 Permutation Matrices for Row Interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 33 / 46
93 Matrix Factorization: Permutation Matrices Limitations of the LU Factorization Algorithm We assumed that Ax = b can be solved using Gaussian elimination without row interchanges. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 34 / 46
94 Matrix Factorization: Permutation Matrices Limitations of the LU Factorization Algorithm We assumed that Ax = b can be solved using Gaussian elimination without row interchanges. From a practical standpoint, this factorization is useful only when row interchanges are not required to control round-off error. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 34 / 46
95 Matrix Factorization: Permutation Matrices Limitations of the LU Factorization Algorithm We assumed that Ax = b can be solved using Gaussian elimination without row interchanges. From a practical standpoint, this factorization is useful only when row interchanges are not required to control round-off error. We will now consider the modifications that must be made when row interchanges are required. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 34 / 46
96 Matrix Factorization: Permutation Matrices We begin with the introduction of a class of matrices that are used to rearrange, or permute, rows of a given matrix. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 35 / 46
97 Matrix Factorization: Permutation Matrices We begin with the introduction of a class of matrices that are used to rearrange, or permute, rows of a given matrix. Permutation Matrix An n n permutation matrix P = [p ij ] is a matrix obtained by rearranging the rows of I n, the identity matrix. This gives a matrix with precisely one nonzero entry in each row and in each column, and each nonzero entry is a 1. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 35 / 46
98 Matrix Factorization: Permutation Matrices Example The matrix is a 3 3 permutation matrix. P = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 36 / 46
99 Matrix Factorization: Permutation Matrices Example The matrix P = is a 3 3 permutation matrix. For any 3 3 matrix A, multiplying on the left by P has the effect of interchanging the second and third rows of A: PA = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 36 / 46
100 Matrix Factorization: Permutation Matrices Example The matrix P = is a 3 3 permutation matrix. For any 3 3 matrix A, multiplying on the left by P has the effect of interchanging the second and third rows of A: PA = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 = a 11 a 12 a 13 a 31 a 32 a 33 a 21 a 22 a 23 Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 36 / 46
101 Matrix Factorization: Permutation Matrices Example The matrix P = is a 3 3 permutation matrix. For any 3 3 matrix A, multiplying on the left by P has the effect of interchanging the second and third rows of A: PA = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 = a 11 a 12 a 13 a 31 a 32 a 33 a 21 a 22 a 23 Similarly, multiplying A on the right by P interchanges the second and third columns of A. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 36 / 46
102 Matrix Factorization: Permutation Matrices Two useful properties of permutation matrices (1/2) Suppose k 1,...,k n is a permutation of the integers 1,...,n Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 37 / 46
103 Matrix Factorization: Permutation Matrices Two useful properties of permutation matrices (1/2) Suppose k 1,...,k n is a permutation of the integers 1,...,n and the permutation matrix P = (p ij ) is defined by p ij = { 1, if j = k i 0, otherwise Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 37 / 46
104 Matrix Factorization: Permutation Matrices Two useful properties of permutation matrices (2/2) Then PA permutes the rows of A; that is, a k1 1 a k1 2 a k1 n a k2 1 a k2 2 a k2 n PA = a kn1 a kn2 a knn Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 38 / 46
105 Matrix Factorization: Permutation Matrices Two useful properties of permutation matrices (2/2) Then PA permutes the rows of A; that is, a k1 1 a k1 2 a k1 n a k2 1 a k2 2 a k2 n PA = a kn1 a kn2 a knn P 1 exists and P 1 = P t. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 38 / 46
106 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination Earlier, we saw that for any nonsingular matrix A, the linear system Ax = b can be solved by Gaussian elimination, with the possibility of row interchanges. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 39 / 46
107 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination Earlier, we saw that for any nonsingular matrix A, the linear system Ax = b can be solved by Gaussian elimination, with the possibility of row interchanges. If we knew the row interchanges that were required to solve the system by Gaussian elimination, we could arrange the original equations in an order that would ensure that no row interchanges are needed. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 39 / 46
108 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination Earlier, we saw that for any nonsingular matrix A, the linear system Ax = b can be solved by Gaussian elimination, with the possibility of row interchanges. If we knew the row interchanges that were required to solve the system by Gaussian elimination, we could arrange the original equations in an order that would ensure that no row interchanges are needed. Hence there is a rearrangement of the equations in the system that permits Gaussian elimination to proceed without row interchanges. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 39 / 46
109 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination (Cont d) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 40 / 46
110 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination (Cont d) This implies that for any nonsingular matrix A, a permutation matrix P exists for which the system PAx = Pb can be solved without row interchanges. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 40 / 46
111 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination (Cont d) This implies that for any nonsingular matrix A, a permutation matrix P exists for which the system PAx = Pb can be solved without row interchanges. As a consequence, this matrix PA can be factored into PA = LU, where L is lower triangular and U is upper triangular. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 40 / 46
112 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination (Cont d) This implies that for any nonsingular matrix A, a permutation matrix P exists for which the system PAx = Pb can be solved without row interchanges. As a consequence, this matrix PA can be factored into PA = LU, where L is lower triangular and U is upper triangular. Because P 1 = P t, this produces the factorization A = P 1 LU = (P t L)U. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 40 / 46
113 Matrix Factorization: Permutation Matrices Permutation Matrices & Gaussian Elimination (Cont d) This implies that for any nonsingular matrix A, a permutation matrix P exists for which the system PAx = Pb can be solved without row interchanges. As a consequence, this matrix PA can be factored into PA = LU, where L is lower triangular and U is upper triangular. Because P 1 = P t, this produces the factorization A = P 1 LU = (P t L)U. The matrix U is still upper triangular, but P t L is not lower triangular unless P = I. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 40 / 46
114 Matrix Factorization: Permutation Matrices Example Determine a factorization in the form A = (P t L)U for the matrix A = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 41 / 46
115 Matrix Factorization: Permutation Matrices Example Determine a factorization in the form A = (P t L)U for the matrix A = Note The matrix A cannot have an LU factorization because a 11 = 0. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 41 / 46
116 Matrix Factorization: Permutation Matrices Solution (1/4) However, using the row interchange (E 1 ) (E 2 ), Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 42 / 46
117 Matrix Factorization: Permutation Matrices Solution (1/4) However, using the row interchange (E 1 ) (E 2 ), followed by (E 3 + E 1 ) (E 3 ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 42 / 46
118 Matrix Factorization: Permutation Matrices Solution (1/4) However, using the row interchange (E 1 ) (E 2 ), followed by (E 3 + E 1 ) (E 3 ) and (E 4 E 1 ) (E 4 ), Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 42 / 46
119 Matrix Factorization: Permutation Matrices Solution (1/4) However, using the row interchange (E 1 ) (E 2 ), followed by (E 3 + E 1 ) (E 3 ) and (E 4 E 1 ) (E 4 ), produces Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 42 / 46
120 Matrix Factorization: Permutation Matrices Solution (1/4) However, using the row interchange (E 1 ) (E 2 ), followed by (E 3 + E 1 ) (E 3 ) and (E 4 E 1 ) (E 4 ), produces Then, the row interchange (E 2 ) (E 4 ), Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 42 / 46
121 Matrix Factorization: Permutation Matrices Solution (1/4) However, using the row interchange (E 1 ) (E 2 ), followed by (E 3 + E 1 ) (E 3 ) and (E 4 E 1 ) (E 4 ), produces Then, the row interchange (E 2 ) (E 4 ), followed by (E 4 + E 3 ) (E 4 ), Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 42 / 46
122 Matrix Factorization: Permutation Matrices Solution (1/4) However, using the row interchange (E 1 ) (E 2 ), followed by (E 3 + E 1 ) (E 3 ) and (E 4 E 1 ) (E 4 ), produces Then, the row interchange (E 2 ) (E 4 ), followed by (E 4 + E 3 ) (E 4 ), gives the matrix U = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 42 / 46
123 Matrix Factorization: Permutation Matrices Solution (2/4) The permutation matrix associated with the row interchanges Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 43 / 46
124 Matrix Factorization: Permutation Matrices Solution (2/4) The permutation matrix associated with the row interchanges (E 1 ) (E 2 ) and (E 2 ) (E 4 ) Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 43 / 46
125 Matrix Factorization: Permutation Matrices Solution (2/4) The permutation matrix associated with the row interchanges (E 1 ) (E 2 ) and (E 2 ) (E 4 ) is P = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 43 / 46
126 Matrix Factorization: Permutation Matrices Solution (2/4) The permutation matrix associated with the row interchanges (E 1 ) (E 2 ) and (E 2 ) (E 4 ) is P = and PA = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 43 / 46
127 Matrix Factorization: Permutation Matrices Solution (3/4) Gaussian elimination is performed on PA using the same operations as on A, except without the row interchanges. Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 44 / 46
128 Matrix Factorization: Permutation Matrices Solution (3/4) Gaussian elimination is performed on PA using the same operations as on A, except without the row interchanges. That is, (E 2 E 1 ) (E 2 ), (E 3 + E 1 ) (E 3 ), followed by (E 4 + E 3 ) (E 4 ). Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 44 / 46
129 Matrix Factorization: Permutation Matrices Solution (3/4) Gaussian elimination is performed on PA using the same operations as on A, except without the row interchanges. That is, (E 2 E 1 ) (E 2 ), (E 3 + E 1 ) (E 3 ), followed by (E 4 + E 3 ) (E 4 ). The nonzero multipliers for PA are consequently, m 21 = 1, m 31 = 1, and m 43 = 1, and the LU factorization of PA is PA = = LU Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 44 / 46
130 Matrix Factorization: Permutation Matrices Solution (4/4) Multiplying by P 1 = P t produces the factorization A = P 1 (LU) = P t (LU) = (P t L)U Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 45 / 46
131 Matrix Factorization: Permutation Matrices Solution (4/4) Multiplying by P 1 = P t produces the factorization A = P 1 (LU) = P t (LU) = (P t L)U = Numerical Analysis (Chapter 6) Matrix Factorization R L Burden & J D Faires 45 / 46
132 Questions?
Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).
MAT 2 (Badger, Spring 202) LU Factorization Selected Notes September 2, 202 Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix
Factorization Theorems
Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization
7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix
7. LU factorization EE103 (Fall 2011-12) factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm effect of rounding error sparse
7 Gaussian Elimination and LU Factorization
7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method
Solution of Linear Systems
Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start
SOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
6. Cholesky factorization
6. Cholesky factorization EE103 (Fall 2011-12) triangular matrices forward and backward substitution the Cholesky factorization solving Ax = b with A positive definite inverse of a positive definite matrix
Elementary Matrices and The LU Factorization
lementary Matrices and The LU Factorization Definition: ny matrix obtained by performing a single elementary row operation (RO) on the identity (unit) matrix is called an elementary matrix. There are three
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +
Solving Systems of Linear Equations
LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how
Solving Linear Systems, Continued and The Inverse of a Matrix
, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing
Lecture 3: Finding integer solutions to systems of linear equations
Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture
by the matrix A results in a vector which is a reflection of the given
Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that
Notes on Determinant
ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without
Solving Linear Systems of Equations. Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.
Solving Linear Systems of Equations Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab:
Using row reduction to calculate the inverse and the determinant of a square matrix
Using row reduction to calculate the inverse and the determinant of a square matrix Notes for MATH 0290 Honors by Prof. Anna Vainchtein 1 Inverse of a square matrix An n n square matrix A is called invertible
Solving Systems of Linear Equations
LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how
MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
Systems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
8 Square matrices continued: Determinants
8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You
Lecture 1: Systems of Linear Equations
MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables
Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.
Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(
MAT188H1S Lec0101 Burbulla
Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u
1 VECTOR SPACES AND SUBSPACES
1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such
a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given
University of Lille I PC first year list of exercises n 7. Review
University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients
CONTROLLABILITY. Chapter 2. 2.1 Reachable Set and Controllability. Suppose we have a linear system described by the state equation
Chapter 2 CONTROLLABILITY 2 Reachable Set and Controllability Suppose we have a linear system described by the state equation ẋ Ax + Bu (2) x() x Consider the following problem For a given vector x in
10.2 ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS. The Jacobi Method
578 CHAPTER 1 NUMERICAL METHODS 1. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS As a numerical technique, Gaussian elimination is rather unusual because it is direct. That is, a solution is obtained after
LINEAR ALGEBRA. September 23, 2010
LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................
1 Introduction to Matrices
1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns
Matrix Differentiation
1 Introduction Matrix Differentiation ( and some other stuff ) Randal J. Barnes Department of Civil Engineering, University of Minnesota Minneapolis, Minnesota, USA Throughout this presentation I have
Methods for Finding Bases
Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,
The Characteristic Polynomial
Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem
CS3220 Lecture Notes: QR factorization and orthogonal transformations
CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss
Operation Count; Numerical Linear Algebra
10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floating-point
Math 312 Homework 1 Solutions
Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please
1 Solving LPs: The Simplex Algorithm of George Dantzig
Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.
13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.
3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms
Reduced echelon form: Add the following conditions to conditions 1, 2, and 3 above:
Section 1.2: Row Reduction and Echelon Forms Echelon form (or row echelon form): 1. All nonzero rows are above any rows of all zeros. 2. Each leading entry (i.e. left most nonzero entry) of a row is in
DATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
Unit 18 Determinants
Unit 18 Determinants Every square matrix has a number associated with it, called its determinant. In this section, we determine how to calculate this number, and also look at some of the properties of
Row Echelon Form and Reduced Row Echelon Form
These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation
1.2 Solving a System of Linear Equations
1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables
Similar matrices and Jordan form
Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive
MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).
MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0
Linear Algebra Notes
Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note
Similarity and Diagonalization. Similar Matrices
MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that
Section 6.1 - Inner Products and Norms
Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,
1 Sets and Set Notation.
LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most
9.2 Summation Notation
9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a
SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89. by Joseph Collison
SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89 by Joseph Collison Copyright 2000 by Joseph Collison All rights reserved Reproduction or translation of any part of this work beyond that permitted by Sections
Lecture 4: Partitioned Matrices and Determinants
Lecture 4: Partitioned Matrices and Determinants 1 Elementary row operations Recall the elementary operations on the rows of a matrix, equivalent to premultiplying by an elementary matrix E: (1) multiplying
MATRICES WITH DISPLACEMENT STRUCTURE A SURVEY
MATRICES WITH DISPLACEMENT STRUCTURE A SURVEY PLAMEN KOEV Abstract In the following survey we look at structured matrices with what is referred to as low displacement rank Matrices like Cauchy Vandermonde
Question 2: How do you solve a matrix equation using the matrix inverse?
Question : How do you solve a matrix equation using the matrix inverse? In the previous question, we wrote systems of equations as a matrix equation AX B. In this format, the matrix A contains the coefficients
December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS
December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation
Orthogonal Bases and the QR Algorithm
Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries
RESULTANT AND DISCRIMINANT OF POLYNOMIALS
RESULTANT AND DISCRIMINANT OF POLYNOMIALS SVANTE JANSON Abstract. This is a collection of classical results about resultants and discriminants for polynomials, compiled mainly for my own use. All results
Solving Systems of Linear Equations Using Matrices
Solving Systems of Linear Equations Using Matrices What is a Matrix? A matrix is a compact grid or array of numbers. It can be created from a system of equations and used to solve the system of equations.
1 Determinants and the Solvability of Linear Systems
1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely side-stepped
The Determinant: a Means to Calculate Volume
The Determinant: a Means to Calculate Volume Bo Peng August 20, 2007 Abstract This paper gives a definition of the determinant and lists many of its well-known properties Volumes of parallelepipeds are
Solution to Homework 2
Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if
Inner Product Spaces and Orthogonality
Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,
Arithmetic and Algebra of Matrices
Arithmetic and Algebra of Matrices Math 572: Algebra for Middle School Teachers The University of Montana 1 The Real Numbers 2 Classroom Connection: Systems of Linear Equations 3 Rational Numbers 4 Irrational
Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur
Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards
Linear Codes. Chapter 3. 3.1 Basics
Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length
15.062 Data Mining: Algorithms and Applications Matrix Math Review
.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop
Chapter 3. Distribution Problems. 3.1 The idea of a distribution. 3.1.1 The twenty-fold way
Chapter 3 Distribution Problems 3.1 The idea of a distribution Many of the problems we solved in Chapter 1 may be thought of as problems of distributing objects (such as pieces of fruit or ping-pong balls)
MATH2210 Notebook 1 Fall Semester 2016/2017. 1 MATH2210 Notebook 1 3. 1.1 Solving Systems of Linear Equations... 3
MATH0 Notebook Fall Semester 06/07 prepared by Professor Jenny Baglivo c Copyright 009 07 by Jenny A. Baglivo. All Rights Reserved. Contents MATH0 Notebook 3. Solving Systems of Linear Equations........................
Linear Algebra Notes for Marsden and Tromba Vector Calculus
Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of
Lecture 2 Matrix Operations
Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrix-vector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or
MAT 200, Midterm Exam Solution. a. (5 points) Compute the determinant of the matrix A =
MAT 200, Midterm Exam Solution. (0 points total) a. (5 points) Compute the determinant of the matrix 2 2 0 A = 0 3 0 3 0 Answer: det A = 3. The most efficient way is to develop the determinant along the
Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.
Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry
1 0 5 3 3 A = 0 0 0 1 3 0 0 0 0 0 0 0 0 0 0
Solutions: Assignment 4.. Find the redundant column vectors of the given matrix A by inspection. Then find a basis of the image of A and a basis of the kernel of A. 5 A The second and third columns are
α = u v. In other words, Orthogonal Projection
Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v
2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system
1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables
DETERMINANTS TERRY A. LORING
DETERMINANTS TERRY A. LORING 1. Determinants: a Row Operation By-Product The determinant is best understood in terms of row operations, in my opinion. Most books start by defining the determinant via formulas
LU Factoring of Non-Invertible Matrices
ACM Communications in Computer Algebra, LU Factoring of Non-Invertible Matrices D. J. Jeffrey Department of Applied Mathematics, The University of Western Ontario, London, Ontario, Canada N6A 5B7 Revised
I. GROUPS: BASIC DEFINITIONS AND EXAMPLES
I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called
The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression
The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression The SVD is the most generally applicable of the orthogonal-diagonal-orthogonal type matrix decompositions Every
Solutions to Math 51 First Exam January 29, 2015
Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not
Math 215 HW #6 Solutions
Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T
Mathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
160 CHAPTER 4. VECTOR SPACES
160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results
Chapter 6. Orthogonality
6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be
26. Determinants I. 1. Prehistory
26. Determinants I 26.1 Prehistory 26.2 Definitions 26.3 Uniqueness and other properties 26.4 Existence Both as a careful review of a more pedestrian viewpoint, and as a transition to a coordinate-independent
MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.
MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column
-4- Algorithm Fast LU Decomposition
Appears in SIGSAM Bulletin 22(2):4-49 (April 988). Analysis of the Binary Complexity of Asymptotically Fast Algorithms for Linear System Solving* Introduction Brent Gregory and Erich Kaltofen Department
Linear Algebra: Determinants, Inverses, Rank
D Linear Algebra: Determinants, Inverses, Rank D 1 Appendix D: LINEAR ALGEBRA: DETERMINANTS, INVERSES, RANK TABLE OF CONTENTS Page D.1. Introduction D 3 D.2. Determinants D 3 D.2.1. Some Properties of
Lecture Notes 2: Matrices as Systems of Linear Equations
2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably
The world s largest matrix computation. (This chapter is out of date and needs a major overhaul.)
Chapter 7 Google PageRank The world s largest matrix computation. (This chapter is out of date and needs a major overhaul.) One of the reasons why Google TM is such an effective search engine is the PageRank
Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws.
Matrix Algebra A. Doerr Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Some Basic Matrix Laws Assume the orders of the matrices are such that
Iterative Methods for Solving Linear Systems
Chapter 5 Iterative Methods for Solving Linear Systems 5.1 Convergence of Sequences of Vectors and Matrices In Chapter 2 we have discussed some of the main methods for solving systems of linear equations.
Inner product. Definition of inner product
Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product
Orthogonal Projections
Orthogonal Projections and Reflections (with exercises) by D. Klain Version.. Corrections and comments are welcome! Orthogonal Projections Let X,..., X k be a family of linearly independent (column) vectors
Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances
Torgerson s Classical MDS derivation: 1: Determining Coordinates from Euclidean Distances It is possible to construct a matrix X of Cartesian coordinates of points in Euclidean space when we know the Euclidean
7.4. The Inverse of a Matrix. Introduction. Prerequisites. Learning Style. Learning Outcomes
The Inverse of a Matrix 7.4 Introduction In number arithmetic every number a 0 has a reciprocal b written as a or such that a ba = ab =. Similarly a square matrix A may have an inverse B = A where AB =
Chapter 2 Determinants, and Linear Independence
Chapter 2 Determinants, and Linear Independence 2.1 Introduction to Determinants and Systems of Equations Determinants can be defined and studied independently of matrices, though when square matrices
Vector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty
Lecture 5: Singular Value Decomposition SVD (1)
EEM3L1: Numerical and Analytical Techniques Lecture 5: Singular Value Decomposition SVD (1) EE3L1, slide 1, Version 4: 25-Sep-02 Motivation for SVD (1) SVD = Singular Value Decomposition Consider the system