Systems of Linear Equations

Transcription

1 Systems of Linear Equations 01 Definitions Recall that if A R m n and B R m p, then the augmented matrix [A B] R m n+p is the matrix [A B], that is the matrix whose first n columns are the columns of A, and whose last p columns are the columns of B Typically we consider B = R m 1 R m, a column vector We also recall that a matrix A R m n is said to be in reduced row echelon form if, counting from the topmost row to the bottom-most, 1 any row containing a nonzero entry precedes any row in which all the entries are zero (if any) 2 the first nonzero entry in each row is the only nonzero entry in its column 3 the first nonzero entry in each row is 1 and it occurs in a column to the right of the first nonzero entry in the preceding row Example 01 The following matrices are not in reduced echelon form because they all fail some part of 3 (the first one also fails 2): ( ) A matrix that is in reduced row echelon form is: A system of m linear equations in n unknowns is a set of m equations, numbered from 1 to m going down, each in n variables x i which are multiplied by coefficients a ij F, whose sum equals some b j R: a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 (S) a m1 x 1 + a m2 x a mn x n = b m If we condense this to matrix notation by writing x = (x 1,, x n ), b = (b 1,, b m ) and A R m n, the coefficient matrix of the system, the matrix whose elements are the coefficients a ij of the variables in (S), then we can write (S) as (S) Ax = b noting, of course, that b and x are to be treated as column vectors here by associating R n with R n 1 If b = 0 the system (S) is said to be homogeneous, while if b 0 it is said to be nonhomogeneous Every nonhomogeneous system Ax = b has an associated or corresponding homogeneous system Ax = 0 Furthermore, each system Ax = b, homogeneous or not, has an associated or corresponding augmented matrix is the [A b] R m n+1 A solution to a system of linear equations Ax = b is an n-tuple s = (s 1,, s n ) R n satisfying As = b The solution set of Ax = b is denoted here by K A system is either consistent, by which 1

2 rank(aq) = dim ( im(t AQ ) ) = dim ( im(t P T A ) ) = dim ( im(t P T A ) ) = dim(im(t A )) = rank(a) we mean K, or inconsistent, by which we mean K = Two systems of linear equations are called equivalent if they have the same solution set For example the systems Ax = b and Bx = c, where [B c] = rref([a b]) are equivalent (we prove this below) 02 Preliminaries Remark 02 Note that we here use a different (and more standard) definition of rank of a matrix, namely we define rank A to be the dimension of the image space of A, rank A := dim(im A) We will see below that this definition is equivalent to the one in Bretscher s Linear Algebra With Applications (namely, the number of leading 1s in rref(a)) Theorem 03 If A R m n, P R m m and Q R n n, with P and Q invertible, then (1) rank(aq) = rank(a) (2) rank(p A) = rank(a) (3) rank(p AQ) = rank(a) Proof: (1) If Q is invertible then the associated linear map T Q is invertible, and so bijective, so that im T Q = T Q (R n ) = R n Consequently so that im(t AQ ) = im(t A T Q ) = T A ( im(tq ) ) = T A (R n ) = im(t A ) rank(aq) = dim ( im(t AQ ) ) = dim ( im(t A ) ) = rank(a) (2) Again, since T P is invertible, and hence bijective, because P is, we must have Thus, dim ( im(t P T A )) ) = dim(im(t A )) (3) This is just a combination of (1) and (2): rank(p AQ) = rank(aq) = rank(a) Corollary 04 Elementary row and column operations on a matrix are rank-preserving Proof: If B is obtained from A by an elementary row operation, there exists an elementary matrix E such that B = EA Since elementary matrices are invertible, the previous theorem implies rank(b) = rank(ea) = rank(a) A similar argument applies to column operations Theorem 05 A linear transformation T L(R n, R m ) is injective iff ker(t ) = {0} Proof: If T is injective and x ker(t ), then T (x) = 0 = T (0), so that x = 0, whence ker(t ) = {0} Conversely, if ker(t ) = {0} and T (x) = T (y), then, 0 = T (x) T (y) = T (x y) = x y = 0 or x = y, and so T is injective 2

3 Theorem 06 A linear transformation T L(R n, R m ) is injective iff it carries linearly independent sets into linearly independent sets Proof: If T is injective, then ker T = {0}, and if v 1,, v k R n are linearly independent, then for all a 1,, a k R we have a 1 v a k v k = 0 = a 1 = = a k = 0 Consequently, if a 1 T (v 1 ) + + a k T (v k ) = 0 then, since a 1 T (v 1 ) + + a k T (v k ) = T (a 1 v a k v k ), we must have a 1 v a k v k ker T, or a 1 v a k v k = 0, and so a 1 = = a k = 0 whence T (v 1 ),, T (v n ) R m are linearly independent Conversely, if T carries linearly independent sets into linearly independent sets, let β = {v 1,, v n } be a basis for R n and suppose T (u) = T (v) for some u, v R n Since u = a 1 v a n v n and v = b 1 v b n v n for unique a i, b i R, we have 0 = T (u) T (v) = T (u v) = T ( (a 1 b 1 )v (a n b n )v n ) = (a 1 b 1 )T (v 1 ) + + (a n b n )T (v n ) so that, by the linear independece of T (v 1 ),, T (v n ), we have a i b i = 0 for all i, and so a i = b i for all i, and so u = v by the uniqueness of expressions of vectors as linear combinations of basis vectors Thus, T (u) = T (v) = u = v, which shows that T is injective 03 Important Results Theorem 07 The solution set K of any system Ax = b of m linear equations in n unknowns is an affine space, namely a coset of ker(t A ) represented by a particular solution s R n : Proof: If s, w K, then K = s + ker(t A ) (01) A(s w) = As Aw = b b = 0 so that s w ker(t A ) Now, let k = s w ker(t A ) Then, w = s + k s + ker(t A ) Hence K s + ker(t A ) To show the converse inclusion, suppose w s + ker(t A ) Then w = s + k for some k ker(t A ) But then Aw = A(s + k) = As + Ak = b + 0 = b so w K, and s + ker(t A ) K Thus, K = s + ker(t A ) Theorem 08 Let Ax = b be a system of n linear equations in n unknowns The system has exactly one solution, A 1 b, iff A is invertible Proof: If A is invertible, substituting A 1 b into the equation gives A(A 1 b) = (AA 1 )b = I n b = b so it is a solution If s is any other solution, then As = b, and consequently s = A 1 b, so the solution is unique Conversely, if the system has exactly one solution s, then by the previous 3

4 theorem K = s + ker(t A ) = {s}, so ker(t A ) = {0}, and T A is injective But it is also onto, because T A L(R n, R n ) takes linearly independent sets into linearly independent sets: explicitly, it takes a basis β = {v 1,, v n } to a basis T A (β) = {T A (v 1 ),, T A (v n )} (because if T (β) is linearly independent, it is a basis by virtue of having n elements) Because it is a basis, T A (β) spans R n, so that if v R n, there are a 1,, a n R such that v = a 1 T A (v 1 ) + + a n T A (v n ) = T A (a 1 v a n v n ) Letting u = a 1 v a n v n R n shows that T A (u) = v, so T A, and therefore A, is surjective, and consequently invertible Theorem 09 A system of linear equations Ax = b is consistent iff rank A = rank[a b] Proof: Obviously Ax = b is consistent iff b im T A But in this case im T A = span(a 1,, a n ) = span(a 1,, a n, b) = im T [A b] where a i are the columns of A Therefore, Ax = b is consistent iff rank A = dim ( im T A ) = dim ( im T(A b) ) = rank ( [A b] ) Corollary 010 If Ax = b is a system of m linear equations in n unknowns and it s augmented matrix [A b] is transformed into a reduced row echelon matrix [A b ] by a finite sequence of elementary row operations, then (1) Ax = b is inconsistent iff rank(a ) rank[a b ] iff [A b ] contains a row in which the only nonzero entry lies in the last column, the b column (2) Ax = b is consistent iff [A b ] contains no row in which the only nonzero entry lies in the last column Proof: If rank A rank[a b ], then rank(a ) < rank[a b ], since we could consider A as equal to [A 0], and if this matrix has r linearly independent rows, or rank r, so does A Whence if rank[a b ] rank[a 0] = rank A, it is because b contains some nonzero element in one of the bottom n r slots corresponding to the zero rows of A Hence [A b ] contains a row in which the only nonzero entry lies in the last column Thus, by the last theorem, since rank is preserved under multiplication by elementary matrices (Corollary 04), we have Ax = b is inconsistent iff rank A rank[a b] iff rank A rank[a b ] iff [A b ] contains a row in which the only nonzero entry lies in the last column Conversely, if [A b ] contains a row in which the only nonzero entry lies in the last column, then rank[a b ]) > rank[a 0] = rank A The second point follows from the previous theorem, Corollary 04, and 1 of this theorem: Ax = b is consistent iff rank A = rank A = rank[a b ] = rank[a b] iff [A b ] contains no row in which the only nonzero entry lies in the last column Theorem 011 Let Ax = b be a system of m linear equations in n unknowns If B R m m is invertible, then the system (BA)x = Bb is equivalent to Ax = b Proof: If K is the solution set for Ax = b and K is the solution set for (BA)x = Bb, then w K Aw = b = (B 1 B)b (BA)w = Bb w K so K = K 4

5 Corollary 012 If Ax = b is a system of m linear equation in n unknowns, then A x = b is equivalent to Ax = b if [A b ] is obtained from [A b] by a finite number of elementary row operations Proof: If [A b ] is obtained from [A b] by a finite number of elementary row operations, which may be executed by left-multiplying [A b] by elementary m m matrices E 1,, E p, then let B = E p E p 1 E 1, which is invertible, so that [A b ] = B[A b] = [BA Bb] Hence, since A = BA and b = Bb, A x = b is equivalent to Ax = b by the previous theorem Remark 013 (Gaussian Elimination) As a result of this corollary, we now know that Gaussian elimination transforms any system of linear equations Ax = b into its equivalent reduced row echelon form A x = b In the forward pass the augmented matrix is transformed into an upper triangular matrix in which the first nonzero entry of each row is 1, and it occurs in a column to the right of the first nonzero entry of each preceding row This is achieved by a finite number type 3 and 2 row operations/elementary matrix multiplications, since there are finitely many rows in [A b] In the backward pass or back substitution the upper triangular matrix is transformed into reduced row echelon form by making the first nonzero entry of each row the only nonzero entry of its column This is also achieved by type 3 and 2 row operations/elementary matrix multiplications Hence, by the previous corollary, we can always find m m invertible matrices B such that by multiplying the augmented matrix by it we produce an equivalent system which is in row echelon form By Theorem 010 through Corollary 012 we know that Gaussian elimination will tell us whether a system Ax = b does or does not have a solution, namely if and only if the reduced row echelon form of the augmented matrix [A b ] contains no row in which the only nonzero entry lies in the last column The next theorem tells us what to do next in order to obtain a particular solution s and, when A is not invertible, a basis for the solution set K = s + ker(t A ) Theorem 014 Let Ax = b be a consistent system of m linear equations in n unknowns, that is let rank A = rank[a b], and let the reduced row echelon form [A b ] of the augmented matrix [A b] have r m nonzero rows Then, (1) rank A = r (2) If we divide into two classes the variables appearing in the reduced row echelon form A x = b of the system, the outer variables or dependent variables, consisting of the r variables x 1 = x i1,, x ir appearing as the leftmost in one of the equations, and the inner variables or free variables consisting of the other x j, and then parametrize the inner variables x j1,, x jn r by setting x j1 = t 1,, x jn r = t n r for t 1,, t n r R, then, solving for the outer variables in terms of the inner variables and putting the resulting values of the x i in terms of t 1,, t n r back into the equation for x results in a general solution of the form x = s = s 0 + t 1 u t n r u n r Here, the constant vector s 0 is a particular solution of the system, ie s 0 K, and the set {u 1,, u n r } is a basis for ker(t A ), the solution set to the corresponding homogeneous system The procedure is illustrated below (cf also Example 017): a 11 a 1n a m1 a mn x 1 x n = b 1 b n 5

6 Gaussian elimination 1 a 12 a 1n b x a r,n r+1 a rn x = b r 0 x n x 1 = b 1 a 12x 2 a 1nx n x i2 = b 2 a 2i 2 x i2 a 1nx n outer variables in terms of inner variables x ir parametrizing the inner variables and rearranging = b r a r,n r+1x n r+1 a rnx rn x 1 b 1 + u 11 t u 1,n r t n r x j1 t 1 = x ir b r + u r1 t u r,n r t n r x jn r t n r the last of which may be written as a linear combination of 1, t 1,, t n r and condensed to x = s = s 0 + t 1 u t n r u n r Proof: (1) Since [A b ] is in reduced row echelon form, it must have r nonzero rows by the definition of reduced row echelon form, and they are clearly linearly independent, whence r = rank[a b ] = rank A = r (2) By our methods of getting s we know that any such s, for any values of t 1,, t n r, is a solution of the system A x = b, and therefore to the equivalent original system Ax = b: K = s 0 + span(u 1,, u n r ) In particular, if we set t 1 = = t n r = 0, we see that s 0 is a particular solution, ie s 0 K But by Theorem 07 we know that K = s 0 + ker T A, whence However, because rank A = r, we must have ker T A = s 0 + K = span(u 1,, u n r ) dim(ker T A ) = n rank T A = n rank A = n r Therefore, we have that {u 1,, u n r } is a basis for ker T A Theorem 015 If A R n n has rank r > 0 and B = rref(a), then (1) The number of nonzero rows in B is r (2) For each i = 1,, r, there is a column b ji of B such that b ji = e i (3) The columns of A numbered j 1,, j r, corresponding to the b ji in (2), are linearly independent (4) For all k = 1,, n, if column k of B is the linear combination d 1 e d r e r 6

7 then column k of A is the linear combination d 1 a ji + + d r a ji where the a ji are the linearly independent columns given in (3) Proof: (1) By Corollary 04 rank(a) = r = rank(b) = r, and because B is in reduced row echelon form, no nonzero row of B can be a linear combination of the others, which means B has exactly r nonzero rows (2) If r 1, the vectors e 1,, e r must occur among the columns of B by the definition of reduced row echelon form, and these we label b ji (3) Note that if there are c 1,, c r R such that c 1 a j1 + c r a jr = 0 since B is obtained from A by a finite sequence of elementary row operations, there exists an invertible m m matrix C = E p E p 1 E 1 such that CA = B, whence C(c 1 a j1 + c r a jr ) = c 1 Ca j1 + c r Ca jr = c 1 b j1 + c r b jr = c 1 e 1 + c r e r = 0 so we must have c 1 = = c r = 0, and a ji,, a ji are linearly independent (4) Note that since B has only r nonzero rows, every column of B is of the form d 1i d ri 0 0 for d 1i,, d ri F, so for those columns of B that look like d 1i e d ri e r, the corresponding columns of A must be C 1 (d 1i e d ri e r ) = d 1i C 1 e d ri C 1 e r = d 1i C 1 b 1i + + d ri C 1 b ri = d 1i a 1i + + d ri a ri Corollary 016 The reduced row echelon form of a matrix is unique Proof: This follows directly from part (4) of the previous theorem, since the d ij are determined by the columns of A, and every column a ji of A is in the span of {a j1,, a jr }, which is a linearly independent set Consequently, each column is uniquely expressed as a linear combination of {a j1,, a jr }, whence the d ij are unique Now, since C is invertible, T C is an isomorphism, so that B = T C (A) also has its columns uniquely expressed as a linear combination of T C (a j1 ),, T C (a jr ) Consequently, B is unique 7

8 04 Examples Example 017 Convert the following system of 4 linear equations in 5 unknowns 2x 1 + 3x 2 + x 3 +4x 4 9x 5 = 17 x 1 + x 2 + x 3 + x 4 3x 5 = 6 x 1 + x 2 + x 3 +2x 4 5x 5 = 8 2x 1 + 2x 2 + 2x 3 +3x 4 8x 5 = 14 by Gaussian elimination into reduced row echelon form, then parametrize the inner variables and show that the solution set of the system is K = (3, 1, 0, 2, 0) + ker A = (3, 1, 0, 2, 0) + span ( ( 2, 1, 1, 0, 0), (2, 1, 0, 2, 1) ) Solution: We use E 1, E 2 and E 3 to denote generic elementary matrices of type 1, 2 and 3, respectively E (02) row 1: 3 E 3 row 3: 2 E row 3: E1 row 1: E (03) (04) where (02) and (03) represent the forward pass, reducing (A b) to an upper triangular matrix with the first nonzero entry in each row equal to 1, while the backward pass/back substitution occurs in (04), producing the reduced row echelon form The equivalent system of linear equations corresponding to the reduced row echelon matrix is x 1 + 2x 3 2x 5 = 3 x 2 x 3 + x 5 = 1 x 4 2x 5 = 2 Now, to solve such a system, divide the variables into 2 sets, one consisting of those that appear as leftmost in one of the equations of the system, the other of the rest In this case, we divide them into {x 1, x 2, x 4 } and {x 3, x 5 } To each variable in the second set, assign a parametric value t 1, t 2, R In our case we have x 3 = t 1 and x 5 = t 2 Then solve for the variables in the first set in terms of those in the second set: x 1 = 2x 3 +2x 5 +3 = 2t 1 +2t x 2 = x 3 x 5 +1 = t 1 t x 4 = 2x 5 +2 = 2t

9 Thus an arbitrary solution is of the form x 1 2t 1 + 2t x 2 x = x 3 x 4 = t 1 t t 1 2t = t t x 5 t Note that β = 1 0, 1 1 0, s = are, respectively, a basis for ker(t A ), the homogeneous system, and a particular solution of the nonhomogeneous system Of course ker(t A ) = span(β), so the solution set for the nonhomogeneous system is K = s + ker(t A ) = {(3, 1, 0, 2, 0) + t 1 ( 2, 1, 1, 0, 0) + t 2 (2, 1, 0, 2, 1) t 1, t 2 R} For example, choosing t 1 = 2 and t 2 = 10, we have s = (19, 7, 2, 22, 10) we have = So s is indeed a solution Example 018 are linearly independent Show that the first, third and fifth columns of A = Solution: We could, of course, check directly, if we already knew that columns 1, 3 and 5 were the ones we were looking for: a, b, c R, a + 6b + 4c 0 a b c 1 0 = a + 3b + c 2a + 8b = a + 7b + 9c 0 implies that a = 4b, which implies that 8b+6b = 4c, or b = 2c, 4b+3b = c, or b = c, whence c = 2c, so c = 0, whence a = b = 0 But we might have to try ( ) 5 3 = 5! 3!2! = 10 different possible combinations of columns of A to figure out that the 1, 3, 5 combination is the right one Instead of proceeding so haphazardly, we could deduce this more simply by transforming A to reduced row echelon form and using Theorem 015: Gaussian elimination

10 which immediately shows that b 11 = e 1, b 32 = e 2 and b 53 = e 3 are our B columns, and hence a 1, a 3 and a 5 are our linearly independent A columns Note also that since b 2 = 2b 1 = 2e 1 and b 4 = 4b 1 b 3 = 4e 1 e 2, we must have, by part 4 of the theorem, that a 2 = 2a 1 and a 4 = 4a 1 a 3, which we of course have 10