Reading [SB] Ch. 11, p , Ch. 27, p

Transcription

1 Reading [SB] Ch., p , Ch. 27, p Basis. Linear Combinations A linear combination of vectors v, v 2,, v m R n with scalar coefficients α, α 2,, α m R is the vector α v + α 2 v α m v m. as The set of all linear combinations of vectors v, v 2,, v m R n is denoted L[v, v 2,, v m ] = {α v + α 2 v α m v m, α i R}. It is evident that L[v, v 2,, v m ] R n is a subspace. Example. For a one single nonzero vector v R n L[v] = {t v, t R} is the line generated or spanned by v: it passes trough the origin and has direction of v. Example. For any two nonzero vectors v, w R n is either: L[v, w] = {s v + t w, s, t R} the line generated (or spanned) by v if v and w are collinear, that is if w = k v, k R, or is the plane generated (or spanned) by v and w, which passes trough the origin, if v and w are non-collinear. Example. For any two non-collinear vectors v, w R 2 is whole R 2. L[v, w] = {s v + t w, s, t R} Example. For any three nonzero vectors u, v, w R 2 s.t. v and w are non-collinear L[v, w] = L[u, v, w] = R 2.

2 .2 Linear Dependence and Independence Definition. A sequence of vectors v, v 2,, v n is called linearly dependent if one of these vectors is linear combination of others. That is i, v i L(v,, ˆv i,, v n ). Definition. A sequence of vectors v, v 2,, v m is linearly dependent if there exist α,, α m with at last one nonzero α k s.t. α v + α 2 v α m v m =. Why these definitions are equivalent? Example. Any sequence of vectors which contains the zero vector is linearly dependent. (Why?) Example. Any sequence of vectors which contains two collinear vectors is linearly dependent. (Why?) Example. Any sequence of vectors of R 2 which consists of more then two vectors is linearly dependent. (Why?) Example. A sequence consisting of two vectors v, v 2 is linearly dependent if and only if these vectors are collinear (proportional), i.e. v 2 = k v. (Why?) Definition 2. A sequence of vectors v, v 2,, v n is called linearly independent if it is not linearly dependent. Definition 2. A sequence of vectors v, v 2,, v n is called linearly independent if non of these vectors is a linear combination of others. Definition 2. A sequence of vectors v, v 2,, v n is called linearly independent if α v + α 2 v α m v m = is possible only if all α i -s are zero. Why these definitions are equivalent? Example. The vectors e = (,, ), e 2 = (,, ), e 3 = (,, ) R 3 are linearly independent. Indeed, suppose α e + α 2 e 2 + α 3 e 3 =, this means α (,, ) + α 2 (,, ) + α 3 (,, ) = (α,, ) + (, α 2, ) + (,, α 3 ) = (α, α 2, α 3 ) = (,, ), thus α =, α 2 =, α 3 =. 2

3 .2. Linear Independence and Systems of Linear Equations How to check wether a given sequence of vectors v, v 2,, v m R n is linear dependent or independent? Let v v 2 v m v A = 2 v 22 v m, v n v 2n v mn be the matrix whose columns are v j s. Theorem A sequence of vectors v, v 2,, v m is linear independent iff the homogenous system Aα = has only zero solution α = (,, ). Example. Determine whether the sequence of vectors is linearly dependent v =, v 2 =, v 3 = Solution. We must check whether the equation c v + c 2 v 2 + c 3 v 3 = has non-all-zero solution for c, c 2, c 3. In coordinates this equation looks as a system c + c 2 + c 3 = c + c 2 + c 3 =. c + c 2 + c 3 = c + c 2 + c 3 = The matrix of the system has maximal rank 3. So there are no free variables, and the system has only zero solution. Thus this sequence of vectors is linearly independent. Example. Determine whether the sequence of vectors is linearly dependent v =, v 2 =, v 3 =.. 3

4 Solution. We must check whether the equation c v + c 2 v 2 + c 3 v 3 = has non-all-zero solution for c, c 2, c 3. In coordinates this equation looks as a system c + c 2 + c 3 = c + c 2 + c 3 =. c c 2 + c 3 = c + c 2 + c 3 = The matrix of the system has the rank 2. So there is free variable, and the system has non-zero solutions too. Thus this sequence of vectors is linearly dependent. Theorem 2 A set of vectors v, v 2,, v k in R n with k > n is linearly dependent. Proof. We look at a nonzero solution c,, c k of the equation c v + + c k v k =, or, equivalently, of the system v c + + v k c k = v 2 c + + v k2 c k = v n c + + v kn c k =. This homogenous system has k variables and n equations. Then rank n < k, so there definitely are free variables, consequently there exists nonzero solution c,, c k. Theorem 3 A set of vectors v, v 2,, v n in R n is linearly independent iff det(v v 2 v n ). Proof. We look at a nonzero solution for c,, c n of the equation c v + + c n v n =. The system which corresponds to this equation has n variables and n equations and is homogenous. So it has a non-all-zero solutions iff its determinant is zero. 4

5 .3 Span Let v,, v k be a sequence of m vectors from R n. The set of all linear combinations of these vectors L[v,, v k ] = {α v + α 2 v α k v k, α,, α k R} is called the set generated (or spanned) by the vectors v,, v k. Example. The vectors v = (,, ), v 2 = (,, ) span the xy plane (the plane given by the non-parameterized equation z = ) of R 3. Indeed, any point p = (a, b, ) of this plane is the following linear combination av + bv 2 = a(,, ) + b(,, ) = (a,, ) + (, b, ) = (a, b, ). Example. The vectors v = (, 2), v 2 = (3, 4) span whole R 2. Indeed, let s take any vector v = (a, b). Our aim is to find c, c 2 s.t. c v + c 2 v 2 = v. In coordinates this equation looks as a system { c + c 2 3 = a c 2 + c 2 4 = b. The determinant of this system, so this system has a solution for each a and b. Example. Different sequences of vectors can span the same sets. For example R 2 is spanned by each of the following sequences: (a) v = (, ), v 2 = (, ); (b) v = (, ), v 2 = (, ); (c) v = (, ), v 2 = (, ); (d) v = (, 2), v 2 = (2, ); (e) v = (, ), v 2 = (, ), v 3 = (2, 3). Check this! For a given sequence of vectors v,, v k R n form the n k matrix whose columns are v i s: here v i = (v i, v i2,, v in ). A = v v 2 v k v 2 v 22 v k v n v 2n v kn 5,

6 Theorem 4 Let v,, v k R n be a sequence of vectors. A vector b R n lies in the space L(v,, v k ) if and only if the system A c = b has a solution. Proof. Evident: A c = b means c v + + c k v k = b. Corollary A sequence of vectors v,, v k R n spans R n if and only if the system A c = b has a solution for any vector b R n. Corollary 2 A sequence of vectors v,, v k R n with k n can not span R n. Proof. In this case the matrix A has less columns than rows. Choosing appropriate b we can make rank(a b) > rank(a) (how?), this makes the system A c = b non consistent for this b..4 Basis and Dimension A sequence of vectors v,, v n R n forms a basis of R n if () they are linearly independent; (2) they span R n. Example. The vectors e = (,,, ), e 2 = (,,, ),, e n = (,,, ) form a basis of R n. Indeed, firstly they are linearly independent since the n n matrix (e e 2 e n ) is the identity, thus it s determinant is. Secondly, they span R 2 : any vector v = (x,, x n ) is the following linear combination v = x e + + x n e n. A basis v,, v n R n is called orthogonal if v i v j = for i j. This means that all vectors are perpendicular to each other: v i v j = for i j. An orthogonal basis v,, v n R n is called orthonormal if v i v i =. This means that each vector of this basis has the length. In other words: v i v j = δ i,j where δ ij is famous Kroneker s symbol δ ij = The basis e,, e n is orthonormal. { if i = j if i j. 6

7 Theorem 5 Any two non-collinear vectors of R 2 form a basis. For example e = (, ), e 2 = (, ) is a basis. Another basis is, say e = (, ), e 2 = (, ). Theorem 6 Any basis of R n contains exactly n vectors. Why? Because more than n vectors are linearly dependent, and less than n vectors can not span R n. The dimension of a vector space is defined as the number of vectors in its basis. Thus dim R n = n. Theorem 7 Let v,, v k R n and A be the matrix whose columns are v j s: v v 2 v n v A = 2 v 22 v n. v n v 2n v nn Then the following statements are equivalent (a) v,, v n are linearly independent; (b) v,, v n span R n ; (c) v,, v n is a basis of R n ; (d) det A. Example. R 3 is 3 dimensional: we have here a basis consisting of 3 vectors v = (,, ), v 2 = (,, ), v 3 = (,, ) Generally, the dimension of R n is n: it has a basis consisting of n elements e = (,,, ), e 2 = (,,, ),, e n = (,,, )..5 Subspace A subset V R n is called subspace if V is closed under vector operations - summation and scalar multiplication, that is: v, w V, c R v + w V, c v V. Example. The line x(t) = t (2, ), that is all multiples of the vector v = (2, ) which passes trough the origin is a subspace. But the line x(t) = (, ) + t (2, ) is not. 7

8 Theorem 8 Let w,, w k R n be a sequence of vectors. Then the set of all linear combinations L[w,, w k ] R n is a subspace. Why? Example. The subspace of R 3 {(a, b, ), a, b R}, which is the xy plane, has dimension 2: v = (,, ), v 2 = (,, ) is its basis. Example. Similarly, the subspace of R 3 which is the y line, has dimension : is its basis. {(a,, ), a R}, v = (,, ).5. How to find the dimension and the basis of L(v,, v k )? Let v,, v k R n be a sequence of vectors from R n, and L(v,, v k ) R n be the corresponding subspace. How can we find the dimension and basis of this subspace? Let A = v v 2 v k v 2 v 22 v k v n v 2n v kn be the matrix whose columns are v j s. Let r be the rank of this matrix and M be a corresponding main r r minor. Then the dimension of L(v,, v k ) is r and its basis consists of those v i -s, who intersect M (why?)..6 Conclusion, Let v, v 2,, v k be a sequence of vectors from R n and let A = v v 2 v k v 2 v 22 v k v n v 2n v kn 8,

9 be the matrix whose columns are v j s. Let r be the rank of this matrix. The following table shows when this sequence is linearly independent or spans R n depending on value of k: k < n k = n n < k independent r = k r = n no spans R n no r = n r = n 2 Spaces Attached to a Matrix Let A = a a n a m a mn be an m n matrix. There are three vector spaces attached to A: the column space Col(A) R m, the row space Row(A) R n and the null space Null(A) R n. 2. Column Space The column space Col(A) is defined as a subspace of R m spanned by column vectors of A, that is a a 2 a n a Col(A) = L( 2, a 22,, a n2 ). a m a m2 a nm Theorem 9 dim Col(A) = rank A. 2.. How to Find a Basis of Col(A) Just find a basic minor of A. Then all the columns that intersect this minor form a basis of Col(A). Example. Find a basis of Col(A) for A = Solution. ( ) Calculation shows that a basic minor here can be chosen as 4. So the basis of Col(A) consists of last two columns ,

10 Of course we can choose as a basic minor ( a basis of Col(A) consisting of second and third columns ). In this case we obtain 3 3, The Role of Column Space (a) The system A x = b has a solution for a particular b R m if b belongs to column space Col(A). (b) The system A x = b has a solution for every b R m if and only if rank A equals of number of equations m. (c) If A x = b has a solution for every b, then number of equations = rank A number of variables. 2.2 Row Space The row space Row(A) is defined as a subspace of R n spanned by row vectors of A, that is Row(A) = L(w, w 2,, w m ) where w,, w m are the row vectors of A: Theorem w = (a,, a n ) w m = (a m,, a mn ). dim Row(A) = ranka. So the dimensions of the column space Col(A) and the row space Row(A) both equal to rank A How to Find a Basis of Row(A) Just find a basic minor of A. Then all the rows that intersect this minor form a basis of Row(A) Example. Find a basis of Row(A) for A = ( ) 4 Calculation shows that a basic minor here can be chosen as. So 7 9 the basis of Row(A) consists of first two rows (2, 3,, 4), (2, 3, 7, 9)..

11 2.3 Null-space Previous two attached spaces Col(A) and Row(A) are defined as subspaces generated by some vectors. The third attached space Null(A) is defined as the set of all solutions of the system A x =, i.e. Null(A) = {x = (x,, x n ) R n, A x = }. But is this set a subspace? Yes, yes! But why? Theorem A subset Null(A) is a subspace. Proof. We must show that Null(A) is closed with respect to addition and scalar multiplication. Indeed, suppose x, x Null(A), that is A x =, A x =. Then A(x + x ) = A(x) + A(x ) = + =. Furthermore, let x Null(A) and c R. Then A (c x) = c (A x) = c = How to Find a Basis of Null(A) To find a basis of null-space Null(A)) just solve a system A x =, that is express basic variables in terms of free variables. As we know there are r = rank(a) basic variables, say x, x 2,, x r, and consequently n r free variables, in this case x r+, x r+2,, x n. Express the basic variables in terms of free variables, and find n r following particular solutions particular solutions which form a basis of N ull(a) v = x x 2 x r, v 2 = x 2 x 2 2 x 2 r,, v n r = x n r x n r 2 x n r r, v n r = x n r x n r 2 x n r r.

12 Example. Find a basis for the null-space of the matrix 3 4 A = Solution. First solve the homogenous system x x 2 + 3x 3 x 4 = x + 4x 2 x 3 + x 4 = 3x + 7x 2 + x 3 + x 4 = 3x + 2x 2 + 5x 3 x 4 =. Computation gives rank A = 2, so dim Null(A) = 4 rank A = 4 2 = 2, and the solution gives x = 2.2x 3 +.6x 4, x 2 =.8x 3.4x 4, x 3 = x 3, x 4 = x 4 So the general solution is x x 2 x 3 = x 4 2.2x 3 +.6x 4.8x 3.4x 4 x 3 x 4 Substituting x 3 =, x 4 = we obtain the first basis vector of null space.22.8 v =. Now substituting x 3 =, x 4 = we obtain the second basis vector of.6.4 null space v 2 = So the basis of Null(A) is, v.4 2 =. 2.4 Fundamental Theorem of Linear Algebra The column space of A, spanned by n column vectors, and the row space of A, spanned by m row vectors, have the same dimension equal to ranka. The Fundamental Theorem of Linear Algebra describes the dimension of the third subspace attached to A: Theorem 2 dim Null(A)+rank A=n. 2

13 2.5 Solutions of Systems of Linear Equations We already know how to express all solutions of homogenous system A x = : just find a basis of Null(A) v, v 2,, v n r, then any solution, since it is an element of Null(A), is a linear combination x = α v + + α n r v n r. Now turn to non-homogenous systems. Let A x = b, x R n, b R m be a system of linear equations and A x = be the corresponding homogenous system. Theorem 3 Let c be a particular solution of A x = b. Then, every other solution c of A x = b can be written as c = c + w where w is a vector from Null(A), that is a solution of homogenous system A x =. Proof. Since c and c are solutions, we have A c = b, A c = b. Let s define w = c c. Then A w = A (c c) = A c A c = b b =, so w = c c is a solution of A x =. Thus c = c + w. According to this theorem in order to know all solutions of A x = b it is enough to know one particular solution of A x = b and all solutions of A x =. Then any solution is given by {c + α v + + α n r v n rank A }. But how to find one particular solution of A x = b? Just take (for example) the following free variables x r+ =, x r+2 =,, x n = and solve x,, x r. Example. Express general solution of the system x x 2 + 3x 3 x 4 = x + 4x 2 x 3 + x 4 = 6. 3x + 7x 2 + x 3 + x 4 = 3 3x + 2x 2 + 5x 3 x 4 = 8 Solution. We already know general solution of corresponding homogenous system A x = : a basis of Null(A) is v =, v.4 2 =, 3

14 so the general solution of homogenous system is given by x.22.6 x 2 x 3 = α.8 + α.4 2. x 4 Now we need one particular solution of non-homogenous system. Take x 3 =, x 4 =, we obtain { x x 2 = x + 4x 2 = 6. This gives x = 2, x 2 =. So a particular solution is x 2 x 2 x 3 =. x 4 Finally, the general solution of nonhomogenous system is given by x x 2 x 3 = + α.8 + α.4 2. x Orthogonal Complement For a subspace V R n its orthogonal complement V R n is defined as the set of all vectors w R n that are orthogonal to every vector from V, i.e. V = {w R n, v w = for v V }. Proposition For any subspace V R n (a) V is a subspace. (b) V V = {}. (c) dim V + dim V = n. (d) (V ) = V. (e) Suppose V, W R n are subspaces, din V + din W = n and for each v V, w W one has v w =. Then W = V. Proof of (a).. Suppose w V, i.e. w v = for v V. Let us show that kw V. Indeed kw v = k(w v) = k =. 2. Suppose w, w V, i.e. w v =, w v = for v V. Let us show that w + w V. Indeed (w + w ) v = w v + w v = + =. 4

15 Theorem 4 For a matrix A (a) Row(A) = Null(A). (b) Col = NullA T. Example. In R 3, the orthogonal complement to xy plane is the z-axes. Prove it! 5

16 Exercises Exercises from [SB].2,.3,.9,.,.2,.3,.4 27., 27.2, 27.3, 27.4, 27.5, 27.6, 27.7, 27.8, , 27.3, 27.4, 27.7 Homework. Exercise Show that the vectors from.4 (b) do not span R 3 : present at last one vector which is NOT their linear combination. 3. Show that the vectors from.4 (b) are linearly dependent: find their linear combination with non-all-zero coefficients which gives the zero vector. 4. Show that if v Row(A), w Null(A) then v w =. Actually this proofs Row(A) = Null(A). 5. Exercise 27. (d). 6

17 Let v, v 2,, v m R n and A = Summary v v 2 v m v n v 2n v mn be the matrix whose columns are v j s. Linear Combinations: L[v, v 2,, v m ] = {α v + + α m v m. Linearly dependent: i, v i L(v,, ˆv i,, v m ) or (α,, α m ) (,, ) s.t. α v + α 2 v α m v m =. Linearly independent: α v + α 2 v α m v m = α k =, or Aα = has only zero solution. (v,, v k ) R n spans R n if L[v, v 2,, v m ] = R n or Aα = b has a solution for b = (b,, b n ). (v,, v k ) R n is a basis if it is lin. indep. and spans R n. n lin. indep. vectors span R n, so they form a basis. n vectors spanning R n are lin. indep., so they form a basis. Subspace V R n : v, w V, c R v + w V, c v V. Dimension and basis of L[v, v 2,, v m ]: dimension is rank A, basis - the columns intersecting main minor. Spaces Attached to a Matrix Let A = a a n a m a mn. Column space: Col(A) = L[ a a m, a 2 a m2,, a n a nm b Col(A) iff Ax = b has a solution. dim Col(A) = rank A, basis - columns that intersect main minor. Row Space: Row(A) = L[(a,, a n ),, (am,, a mn )]. dim Row(A) = rank A, basis - rows that intersect main minor. Null-space: Null(A) = {x R n, A x = }. dim Null(A) = n rank A. Basis of Null(A) - the following solutions of Ax = v = x x r, v 2 = x 2 x 2 r,, v n r = x n r x n r r, v n r = Orthogonal complement: V = {w R n, v w = for v V }. Row(A) = Null(A), Col = NullA T. 7 x n r x n r r ]..