Systems of Linear Equations

Transcription

1 Systems of Linear Equations Math 108A: August 5, 2008 John Douglas Moore 1 Orthogonal complements and null spaces of linear maps Linear algebra is the theory behind solving systems of linear equations, such as a 11 x 1 + a 12 x a 1n x n = b 1, a 21 x 1 + a 22 x a 2n x n = b 2, a m1 x 1 + a m2 x a mn x n = b n. (1) Here the a ij s and b i s are known constants, and we are solving for the unknowns x 1,..., x n. This system of linear equations can be written in terms of the matrix a 11 a 12 a 1n A = a 21 a 22 a 2n. a m1 a m2 a mn as Ax = b, where x 1 x = x 2 and b = b 2 x n The matrix A defines a linear map b 1 b m. T A : R n R m by T A (x) = Ax. The range of T A is just the space of vectors b for which the equation Ax = b has a solution, while the null space of T A is the space of solutions to the associated homogeneous linear system a 11 x 1 + a 12 x a 1n x n = 0, a 21 x 1 + a 22 x a 2n x n = 0, a m1 x 1 + a m2 x a mn x n = 0. (2) 1

2 The fundamental Theorem 3.4 from the text [1] states that In particular, it follows that n = dim(r n ) = dim(null(t A )) + dim(range(t A )). (3) dim(null(t A )) = 0 dim(range(t A )) = n. Thus is n = m, T A is surjective if and only if it is injective; equivalently, the system (1) has a solution for every choice of vector b mathbbr n if and only if the associated homogeneous system (??) has only the zero solution. The notion of orthogonal complement gives a nice picture of the process of finding a basis for the space of solutions to a homogeneous linear system. Indeed, we can use the dot product to write the homogeneous linear system whose solutions form the null space of T A in a particularly intuitive form. Let a 1 = (a 11, a 12,..., a 1n ), a 2 = (a 21, a 12,..., a 2n ), a m = (a m1, a m2,..., a mn ), and let W = span(a 1, a 2,..., a m ). We can then say that null(t A ) = {x R n : a 1 x = 0, a 2 x = 0,, a m x = 0} = {x R n : a x = 0 for a W }. Note that the last space on the right-hand side is simply the collection of all vectors which are perpendicular to W. We call this space the orthogonal complement to W and denote it by W. We can then write null(t A ) = W. Orthogonal Complement Theorem. If (a 1, a 2,..., a m ) is a list of vectors in R n, W = span(a 1, a 2,..., a m ) and W is the space of solutions x R n to the homogeneous linear system of equations then a 1 x = 0, a 2 x = 0,, a m x = 0, 1. dim W + dim W = n, and 2. R n = W W. Proof: By Theorem 2.10 in [1], the list (a 1, a 2,..., a m ) can be reduced to a basis (b 1, b 2,..., b k ) for W, where k = dim W. Let B be the k n matrix whose rows are the elements of the list (b 1, b 2,..., b k ) and let T B : R n R k be the corresponding linear map. Then the set of equations a 1 x = 0, a 2 x = 0,, a m x = 0 2

3 is equivalent to the set of equations b 1 x = 0, b 2 x = 0,, b k x = 0, because the space of solutions to either set of equations is just the space of vectors perpendicular to W. Thus we can write W = null(t B ). Therefore it follows from (3) that n = dim(r n ) = dim(null(t B )) + dim(range(t B )) = dim W + dim W. (4) On the other hand, if a W W, then a a = 0 and since the dot product is positive-definite, a = 0. Thus W W = {0} and Proposition 2.19 of [1] then implies that R n = W W, finishing the proof of the Theorem. Corollary. The dimension of the space W spanned by the rows of A equals the dimension of the space spanned by the columns of A. Proof: Comparison of (3) with the equation n = dim W + dim W shows that dim W = dim(range(t A )). But W is the space spanned by the rows of A, while range(t A ) is the space spanned by the columns of A. One sometimes expresses this in the following form: Row rank equals column rank. 2 Elementary row operations Although Theorem 2.10 in [1] states that the list (a 1, a 2,..., a m ) can be reduced to a basis (b 1, b 2,..., b k ), it does not give an efficient procedure for carrying out this reduction. We now describe a useful procedure, based upon the elementary row operations for finding a basis for W and W at the same time. Proposition. Let V be a vector space over a field F, v 1, v 2,..., v m a collection of elements of V. Then 1. If σ : {1, 2,..., m} {1, 2,..., m} is one-to-one and onto. Then Span(v 1, v 2,..., v m ) = Span(v σ(1), v σ(2),..., v σ(m) ). 2. If a F and a 0, then 3. If a F, then Span(v 1, v 2,..., v m ) = Span(av 1, v 2,..., v m ). Span(v 1, v 2,..., v m ) = Span(v 1 + av 2, v 2,..., v m ). As an exercise, you may want to provide a proof of this Proposition. We can change an m n matrix A to a new matrix A by one of the elementary row operations: 3

4 1. Interchange two rows in A. 2. Multiply a row by a nonzero constant c. 3. Add a constant multiple of one row to another. Each of these operations is reversible. Although they change the linear map T A, the above Proposition, shows that the elementary row operations preserve the linear subspace W of R n which is spanned by the rows of A. Moreover, since the elementary row operations transform the homogeneous linear system appearing in (2) into a homogeneous linear system with the same space of solutions, the elementary row operations preserve the null space of T A, which is of course, just the orthogonal complement to W. We apply the elementary row operations with the objective of replacing A by a matrix B in row-reduced echelon form. By definition, the matrix B is in row-reduced echelon form if it has the following properties: 1. The first nonzero entry in any row is a one. 2. If a column contains an initial one for some row all of the other entries in that column are zero. 3. If a row consists of all zeros, then it is below all of the other rows. 4. The initial one in a lower occurs to the right of the initial one in each previous row. It can be proven that any m n matrix can be put in row-reduced echelon form by elementary row operations. It is easy to see how to carry out the procedure. One starts by putting a one in the first row of the first nonzero column. We doe this by interchanging rows if necessary to get a nonzero entry in the first row of the first nonzero column, and then divide the row by this entry. We then zero out all other elements in the first nonzero column. In the submatrix obtained by removing the first row, we then apply the same procedure obtaining an initial one in a second column and zeroing out all other entries in that column. Continuing with this procedure leads to a matrix in row-reduced echelon form. Properly reformulated the procedure we have described would give a proof that any matrix can be put in row-reduced echelon form by elementary row operations. We will not carry out all of the details here. It is usually easy to work this out in special cases, and the resulting rowreduced echelon matrix makes it easy to construct a basis for the space W spanned by the rows of A and a basis for the null space W of T A. One can show that the nonzero rows of the row-reduced echelon matrix form a basis for W. indeed, suppose there are k nonzero rows b 1,..., b k in the row reduced echelon matrix B. Suppose there is a collection of elements x 1,... x k from F, such that x 1 b x k b k = 0. 4

5 Each of the equations corresponds to one of the columns of B and the equation corresponding to the j-th initial one is just the equation x j = 0. Thus x 1 b x k b k = 0 x 1 = = x k = 0, and the row vectors b 1,..., b k are linearly independent. Since they obviously span W they form a basis for W. If there are k nonzero rows and hence W has dimension k, one can solve for the k variables corresponding to the initial ones in those rows in terms of the n k variables corresponding to the other rows. The n k variables which do NOT correspond to initial ones are free variables, and can be thought of as coordinates for the space W. This gives a general solution to the original linear system, in which the free variables form the coordinates. Just as in the case of W, one can check that the list of vectors multiplying the free variables form a basis for W, that is, a basis for the null space of the linear map T A that we started with. Using the bases you have obtained it is easy to verify in any individual case that R n = W W. You can also find a basis for the range of T A, but for that you need to use the elementary column operations on the original matrix A, since the range of T A is spanned by the columns of A. Note that the columns of A are NOT preserved by the elementary row operations. References [1] S. Axler, Linear algebra done right, Second edition, Springer, New York, [2] S. Lang, Linear algebra, Second edition, Springer, New York,