Chapter 6: Solving Linear Equations

Transcription

1 134 Systems of Linear Equations Chapter 6: Solving Linear Equations Problem here is to determine values for n variables that simultaneously satisfy a st of linear equations. ( x k, k 1, 2,, n) a 11 x 1 + a 2 x a 1n x n b 1 a 21 x 1 + a 22 x a 2n x n b 2 a m1 x 1 + a m2 x a mn x n b m Such problems frequently occur in engineering and scientific applications, often involving hundreds or thousands of variables. Thus, we need efficient methods for obtaining a solution. Typical examples are network problems involving electrical circuits, fluid flow, or mechanical forces. Variables here are physical quantities such as electrical current, velocities, mechanical stress or strain, and coordinate displacement values. For instance, in the following circuit, we can write a set of equations to solve for the current through the various resistance components, given the resistance and voltage values. In some cases, as in the circuit above, we may have more equations than variables

2 135 (m < n); and sometimes we may have fewer equations than variables (m < n). Usually, however, m n. Systems of Linear Equations. The system of simultaneous, linear equations can be expressed in mtrix form as A X B or a 11 a 12 a 1n x 1 b 1 a 21 a 22 a 2n x 2 b 2 a m1 a m2 a mn x n b n where A is the m by n matrix of coefficients, X is an n-element column vector (i.e., n by 1 matrix), and B is an element column vector. If m > n (more equations than variables), the system is said to be overdetermined. If (m < n) (more variables than equations), the system is said to be undetermined. If m n, we have a determined system. With an underdetermined system, we can always obtain a solution by choosing values for some of the x k. But overdetermined and determined systems may or may not have solutions. Also, if all b k are zero, the set of equations is called a homogeneous system, and we have at least one solution: x k 0 for all k.

3 136 For a determined system (mn), the set of linear equations has a unique solution if the determinant of A (now an n x n square matrix) is nonzero. A 0 Then A is said to be a nonsingular matrix. Cramer s Rule For a system of n liknear equations with n variables and with express the solution as A 0, we can x k A k k 1, 2,, n A where A k determinant of A with kth column replaced by the elements of matrix B. Example: x 1 + 2x 1 4 3x 1 + 4x 2 1 A 1 2 B A have unique solution A A with solution x x

4 137 For a determined system of n linear equations AX B, if be a singular matrix. A 0, then A is said to In this case, we have two possibilities: (1) Either no solutions exist (some A k 0 ). (2) Or an infinite number of solutions exist (all A k 0 ). In the formulation of a physical problem, we are usually looking for a unique solution. If we find that A 0, we made an error in our formulation. Examples - Singular Matrices x 1 + x 2 1 x 1 + x 2 0 Here, A Also, since A 1 1 and A 2 1 (some A k 0 ), no solution exist. 0 Geometric interpretation: x 2 parallel lines -- same slope, different y-intercept x 1

5 138 x 1 + 2x 2 1 3x 1 + 6x 2 3 Here, A For this example, A 1 0 and A 2 0 (all A k 0 ), so we have an infinite number of solutions. i.e., one equation is simply a multiple of the other. Geometric interpretation: x 2 both equations represent same line x 1 x 1 + x x 1 + x 2 1 A Although A is not 0, it is small compared to size of coefficients (all near 1). This set of equations is called ill-conditioned since the solution (x 1 0, x 2 1) is susceptible to roundoff errors. Geometric interpretation: x 2 both lines have nearly same slope x 1

6 139 Since Cramer s Rule requires the evaluation of n x n determinants, this is a very inefficient approach for problems with more than 3 or 4 equations. Brute-force evaluation of a single n x n determinant, with n > 3, requires more than 1.5n! multiplications. Moreover, a system of n equations has n+1 determinants to evaluate. As an example, for a system of 10 equations, the number of multiplications for each of the 11 determinants is over ! 1.5( 2, 628, 800) 5, 443, 200 More efficient methods for solving systems of linear equations are based on rearranging and eliminating terms in the equations so as to reduce the number of multiplications. With these methods, we can reduce the number of computations to approximately n 3 multiplications to solve a system of n linear equations with n unknowns.

7 140 Gaussian Elimination With Back Substitution An efficient method for solving a set of n equations with m variables. Commonly used in math packages such as Mathematica. This method is based on the fact that we can add combinations of equations so as to successively eliminate terms. In Gaussian Elimination, we take each equation in turn, select an x k, and reduce the coefficients for that x to 0 in all following equations. This converts matrix A to an upper diagonal matrix. Then back substitution is used to solve for all the x values in turn, starting with x n. For example, given: x 1 + 2x 2 4 3x 1 + 4x 2 1 we multiply the first equation by -3, then add the two equations to eliminate the coefficient for x 1 in the second equation: x 1 + 2x x 2 13 (The steps to this point are often referred to as Gaussian Elimination.) Then we solve the last equation to obtain x 2-13/2 and back substitute this value in the first equation to obtain x 1 9

8 141 Similarly, for three equations and three unknowns E 1 E 2 E 3 2x 1 + x 2 + 2x 3 2 3x 1 x 2 + x 3 1 x 1 + 3x 2 + 4x 3 3 We eliminate coefficients in the first column in equations 2 and 3 with the calculations 3 1 E 2 E E, E 2 3 E E 2 1 to obtain 2x 1 + x 2 + 2x x 2 2 2x x x 3 4 Multiplying equation E 2 by (-2/5) (-7/2) and adding to E 3, we have the final set of reduced equations. 2x 1 + x 2 + 2x x 2 2 2x x Then we solve the last equation and back substitute in successive equations to obtain x , x , x

9 142 Summary: Gaussian Elimination with Back Substitution For an n x n system of linear equations a 11 a 12 a 1n x 1 b 1 a 21 a 22 a 2n a n1 a n2 a nn we successively select pivot rows and zero out pivot column coefficients in the following rows to produce the reduced set: x 2 x n b 2 b n a' 11 a' 12 a' 1n x 1 b' 1 0 a' 22 a' 2n x 2 b' a' nn On the first pass, a 11 is chosen as the pivot element and row 1 is multiplied by -a 21 /a 11 and then added to the second row. Similarly, for zeroing out coefficients in other rows below row 1. On the next pass, row 2 is chosen as the pivot row, etc. Next, the solution is obtained by back substitution: x n b' n x n b' a' nn n 1 x k b' a' k a' kj x j k j k + 1 In general, the method can be applied to any set of m x n equations and can detect systems with no solutions or multiple solutions.

10 143 Pseudocode Algorithm: Gaussian Elimination The following routine assumes an n x n system with A 0 and all diagonal elements of A are nonzero (a n 0, k 1,..., n) Input n, A, B Do pivotrow 1, n-1 (* n-1 passes *) pivotcol pivotrow pivot A(pivotRow, pivotcol) Do row pivotrow+1, n factor A(row, pivotcol) / pivot Do col pivotcol, n A(row, col) A(row, col) - factor * A(pivotRow, col) EndDo B(row) B(row) - factor * B(pivotRow) EndDo EndDo (How woud you write the next code section to accomplish back substitution?) The above algorithm converts matrix A into a lower-triangular matrix. If we wanted to same A, we could first copy it into a new matrix. We could also combine A and B into a single n x n+1 matrix A, with B as the last column of A, then apply the above algorithm to input matrix A. A variation of the above algorithm (Gauss-Jordan Method) can be used to efficiently compute the inverse of an n x n matrix A. First, combine A with identity matrix to form a 2n x 2n matrix. Then, converting the A part of this matrix to the identity matrix, produces the inverse of A in the other half of the matrix. Example A I I A 1

11 144 Extensions to Gaussian Elimination Algorithm 1. Select max element in each row as pivot element. Use index parameter to keep track of selected pivot cols. 2. If any selected pivot has a value near 0, terminate and label matrix as ill-conditioned. 3. Extend to general set of m equations with n variables. 4. Check solution in original equations to test accuracy of calculations. LU Decomposition A somewhat faster method than Gaussian Elimination. Also is much more efficient if we want to solve the set of equations with different sets of values for matrix B. For any n x n matrix A, we can always find two triangular matrices such that A LU where L is a lower-triangular matrix with all diagonal elements equal to 1, and U is an upper triangular matrix. Then the solution to AX B or LU X B can be obtained efficiently with the following two steps: LY B UX Y (first find matrix Y, then solve for X)

12 145 LU Decomposition Example: Given the equations: x 1 x2 2 1 We can write matrix A as Then we first solve the lower-triangular equation using foward substitution: LY B y 1 2 y 2 1 yielding: y 1, 2, y 2 3 Next, we solve the upper-triangular equation using back substitution: UX Y x 1 x 2 23 yielding: x 2 1, x 2 2 3

13 146 General LU Decomposition Algorithm: 1. We factor matrix A into triangular components: a 11 a 12 a 1n a 21 a 22 a 2n l u 11 u 12 u 1n 0 u 22 u 2n a n1 a n2 a nn l n1 l n u nn using the following calculations to obtain the elements of the lower- and upper-trangular matrices: for j 1,..., n for i 1, 2,..., j i 1 u ij a ij l ik u kj (summation term 0 whe i1) k 1 for i j + 1, j + 2,..., n j 1 1 l ij a u ij ij l ik u kj k 1 (values for the matrix elements on the right side of the above calculations are known) 2. Obtain y values using forward substitution: k 1 y 1 b 1, y k b k l kj y j k 2, 3,, n j 1 3. Obtain solution (x values) using back substitution: n y n 1 x n , x u k y nn u k u kj x j kk j k + 1 k n 1, n 2,, 1

14 147 LU Decomposition also provides an efficient method for evaluation of determinants. Thus, if we decompose matrix A as A LU then the determinant of A is simply n A u kk u 11 u 22 u nn k 1 Gauss-Seidel Iterative Method Can be used when some pivot elements are too small or when many coefficients are zero (sparse matrix). Start with an initial gauss for variables, then repeatedly calculate successive approximations until difference between successive values is small enough (converges). b 1 a 12 x 2 a 13 x 3 a 1n x n x a 11 i.e., etc. b 2 a 21 x 1 a 23 x 3 a 2n x n x a 22 Assuming a kk 0 k, values will converge if A is diagonally dominant : a kk > a kj (for each row) k k If above conditions are not satisfied, it may be possible to rearrange rows and columns so as to guarantee convergence.

15 148 Solving Linear Systems in Mathematica One method for solving a set of linear equations is to write the equations explicitly in the Solve function. E.g., Solve[{x1+2 x2-4, 3 x1+4 x21}, {x1,x2}] {{x1 -> 9, x2 -> - (13/2)}} (Note: solution given as set of transformation rules.) In general it is more convenient to deal with the coefficient matrix, particularly with many equations or when the number of equations is determined by the solution of a more general problem. LinearSolve[a,b] - a is an m x n matrix, and b is an m-element vector Examples: a{{1,2}, {3,4}}; b{-4,1; LinearSolve[a,b] {9, -(13/2)} (Solution here is given as a list of values.) A singular matrix with no solutions yields the message: a{{1,1}, {1,1}}; b{1,0}; LinearSolve[a,b] LinearSolve::nosol: Linear equation encountered which has so solution. LinearSolve[{{1, 1}, {1, 1}}, {1, 0}]

16 149 But a singular matrix with an infinite number of solutions may get us one of the solutions: a{{1,2}, {3,6}}; b{1,3}; LinearSolve[a,b] {1, 0} The function LinearSolve will try to find a solution even if too many variables or too many equations are specified (probably using Gaussian Elimination). Examples: a{{1,2,3}, {3,4,4}}; b{-4,1}; LinearSolve[a,b] {9, - (13/2), 0} (3 variables, 2 equations) a{{1,1}}; b{1}; LinearSolve[a,b] {1, 0} (2 variables, 1 equation) a{{1,2}, {3,4}, {5,6}}; b{-4,1,9}; LinearSolve[a,b] LinearSolve::nosol: Linear equation encountered which has no solution. LinearSolve[{{1, 2}, {3, 4}, {5, 6}}, {-4, 1, 9}}