Gaussian Elimination

Transcription

1 Gaussian Elimination Simplest example Gaussian elimination as multiplication by elementary lower triangular and permutation matrices Lower/Upper triangular, Permutation matrices. Invariance properties. Solvability vs. no solution. Solve a linear system x + x 3 = 1, x 1 + x + 3x 3 =, ( ) x 1 + x = 3, Main observations: 1. A row-echelon system, for example x 1 + x + x 3 = 1, x + 3x 3 =, x 3 = 3, is easy to solve by back substitution. That is solve the last equation for x 3, then substitute x 3 into the second equation and solve for x, then substitute x and x 3 into the first equation and solve for x 1. In a row-echelon matrix for two successive rows the leading nonzero entry in the higher row appears farther to the left than the leading nonzero entry in the lower row: Permutation of equations does not change the solution vector x. 3. If x satisfies each of the three equations, then then x satisfies linear combinations of these equations. For example, multiply the second equation in ( ) by 1/ and subtract from the last, then x must satisfy 1/x 1 + 3/x 3 =. 1

2 Gaussian elimination is an algorithm that by linear combinations and permutation of rows converts every matrix to a row-echelon form. Solution to the simplest example. Rewrite the system in the matrix form where Ax = b, A = , b = Form an augmented matrix pertaining to the system A b = Permute the first and the second row of the augmented matrix Subtract the first row from the last row Subtract the second row from the last row Solve by back substitution 4 x = 1. 0 Check that the found x solves the original system. Left multiplication of A by an elementary permutation matrix P ij, for example P 3 =

3 permutes the ith and the jth row of A Left multiplication of A by an elementary lower triangular (transvection) matrix L ij,λ,i < j for example L 3,.1 == adds the ith row, multiplied by λ to the jth row. Gaussian elimination is multiplication by permutation and lower triangular matrices, for example L 3, 1 L 13, 1 P 1 A b = It is useful to study class properties of elementary matrices! 1. For a pair of elementary permutation and lower triangular matrices P and L there exists another pair of such matrices L and P so that L P = P L if L corresponds to an elimination step in the Gaussian method, P corresponds to a permutation step in the Gaussian method, the elimination step precedes the permutation step.. A product of lower triangular matrices is a lower triangular matrix, where a (general) lower triangular matrix is L ij,λ L ij, λ = I, where I is the identity matrix I = A product of permutation matrices is a permutation matrix, where a (general) permutation matrix in each row/column has only one 1, and the rest is 0. 3

4 5. P ij P ij = I. Note: In general part is not true. Counterexample: let 0 1 P =, and L = λ 1 then A = P L = λ 1 cannot be represented as A = L P. Theorem LP A = E, where P is a permutation matrix, L is a lower triangular matrix, E has a row-echelon form. Multiplication is simple, inverses is a delicate matter! Definition A matrix B is a left inverse of a matrix A if BA = I. A matrix B is a right inverse of a matrix A if AB = I. Note It is possible that left/right inverse exists, but right/left does not- think of rectangular (nonsquare) matrices. Lemma If B, a right inverse of A, exists then there is at least one solution of Ax = b. Proof: take x = Bb, then Ax = ABb = b. Lemma If B, a left inverse of A, exists then there is at most one solution of Ax = b. Proof: Suppose Ax 1 = b and Ax = b, then x 1 = BAx 1 = Bb = BAx = x. Note Left inverses are useful for understanding least squares method. Left inverse might exist, but there may be no solutions: Left inverse: A b = B = ( ) 0 0 Note Left inverse can be defined as a (linear) map B such that for any x if y = Ax then By = x. Right inverse can be defined as a (linear) map B such that for any y if By = x then y = Ax. Definition A matrix, denoted by A 1 is the inverse of a matrix A, if A 1 A = AA 1 = I. The inverse matrix is unique: Suppose B and C are two different inverses of A, then B = B(AC) = (BA)C = C. Note The inverse A 1 also can be defined as a (linear) map. Lemma If A 1 exists, then there the solution of Ax = b is unique. Note If a left/right inverse of A exists, then A is an injective/surjective map. If the (dual) inverse of A exists then A is a bijective map. Theorem A = P LE, where P is a permutation matrix, L is a lower triangular matrix, E has a row-echelon form. A 1 = E 1 L 1 P 1 if and only if E 1 exists. 4

5 Theorem E 1 exists if and only if E = DU, where in the diagonal matrix λ 0 λ D = 0 0 λ λ λ n all λ i 0 and U is the upper triangular matrix U = Proof: if E = DU, then for the problem Ax = y for any y by back substitution we can find the unique solution x. E DU if case 1: there are two consequtive rows of E such that the leading nonzero entry in the higher row appears two or more entries farther to the left than the leading nonzero entry in the lower row , ,, or case E has nonzero entries on the diagonal, but it has more rows than columns: , 0 1, or case 3 E has nonzero entries on the diagonal, but it has more columns than rows: In case 1, by backward elimination there either no solutions (left) or there are more than 1 solution (center, right). In the case there either no solutions (left) or there is a unique solution (right). In the third case, there are (always) more than 1 solution. 5