806 Problem Set 4 Solution Due Wednesday, March 2009 at 4 pm in 2-06 Total: 75 points Problem : A is an m n matrix of rank r Suppose there are right-hand-sides b for which A x = b has no solution (a) What are all the inequalities (< or ) that must be true between m, n, and r? (b) A T y = 0 has solutions other than y = 0 Why must this be true? Solution (5 points = 0+5) (a) First of all, the rank r of a matrix is the number of column (row) pivots, it must be less than equal to m and n If the matrix were of full row rank, ie, r = m, it would imply that A x = b always has a solution; we know that this is not the case, and hence r m To sum up, the inequalities among m, n, r are r n, r < m (b) Since A T is an n m matrix, the null space N(A T ) has dimension m r, which is positive by (a) Hence, A T y = 0 has solutions other than y = 0 Problem 2: A is an m n matrix of rank r Which of the four fundamental subspaces are the same for: ( ) A (a) A and A ( ) ( ) A A A (b) and A A A ( ) ( ) A A A Explain why all three matrices A,, and must have the same rank r A A A Solution (20 points = 0+0) (a) Note that if we do invertible row operations on the matrix ( ) A kill the bottom A and get 0 ( ) A, we may A
Nullspace: Since the nullspace is invariant under row operations, the two matrices have the same nullspace Column space: Since the two matrices do not have the same number of rows, their column space must not be the same Row space: Since the row space is also invariant under row operations, the two matrices have the same row space ( ) A Left nullspace: The transpose of is ( A A T A T) Hence, the left nullspace is a subspace of R 2m However, the left nullspace is a subspace of R m They cannot be the same ( ) ( ) A A A 0 (b) Using invertible column operations, we can turn into ( ) A A A 0 A A The nullspace and the row space of are subspaces of R A A 2n, whereas the ( ) A nullspace and the row space of are subspaces of R A n They cannot be the same Since the column space and the left nullspace are invariant under column operations, the two matrices have the same column space and left nullspace REMARK: For an m n matrix of rank r, we have Fundamental space subspace of dimension Nullspace R n n r Column space R m r Row space R n r Left nullspace R m m r Problem 3: Find a basis for each of the four subspaces for 0 2 3 4 A = 0 2 4 6 0 0 0 2 Solution (25 points = 5(echelon form)+5+5+5+5) We first write A as in row-reduced echelon form 0 2 3 4 0 2 3 4 0 2 3 4 0 2 0 2 0 2 4 6 0 0 0 2 0 0 0 2 0 0 0 2 0 0 0 2 0 0 0 2 0 0 0 0 0 0 0 0 0 0 2
The second and the fourth variables are the pivots The first, third, and the fifth variables are free variables The row operation matrix E (R = EA) is 3 0 0 0 E = 0 0 0 0 0 0 0 Nullspace: The null space is as follow: 0 0 N(A) = x 0 0 + x 2 3 0 + x 5 0 0 0 0 0 = 0 0 0 2 0 2 4 3 0 0 for x, x 3, x 5 R Column space: Since the pivot columns (the second and the fourth) of A span the column space, we have 3 C(A) = a + b 4 for a, b R 0 CAUTION: We get the pivot columns by looking at R but C(A) is spanned by these columns of A, NOT these columns of R Row space: it is the same as the row space of R as R is obtained by invertible row operations So 0 0 C(A T ) = x 2 2 0 + x 0 4 0 for x 2, x 4 R 2 2 Left nullspace: It has a basis given by the rows of E for which the corresponding rows of R are all zero That is to say, we need to take the last row of E Thus, N(A T ) = a for a R Problem 4: True or false (give a reason if true, or a counterexample if false): 3
(a) A and A T have the same number of pivots (b) A and A T have the same left nullspace (c) If the C(A) = C(A T ), then A = A T (d) If A T = A, then the row space of A is the same as the column space of A Solution (20 points = 5+5+5+5) (a) True, because A and A T have the same rank, which equals to the number of pivots of the matrices (b) False In particular, if A is an m n matrix of rank r with m n, the dimension of two left nullspace will not be the same; Indeed, dim N(A T ) = m r and dim N ( (A T ) T) = n r For example, A = ( ( 0) ) The left nullspace of A is N(A T ) = 0 and the left null 0 space of A T is N(A) = a for a R (c) This is not necessarily true In particular, if A is a non-symmetric n n invertible matrix, C(A) = C(A T ) = R n (d) True, because the row space of A is C(A T ) = C( A) = C(A) is the same as the column space of A Problem 5: Use the Matlab command A = rand(0,5); to make a random 0 5 matrix A, and B = rand(5,9) to make a random 5 9 matrix B Then use the command [R,p] = rref(a*b); to find the row-reduced echelon form R and a list p of the pivot columns Using this information, give bases for the nullspace, column space, and row space of AB Solution (0 points) >> A = rand(0, 5) A = 0847 0576 06557 07060 04387 09058 09706 00357 0038 0386 0270 09572 0849 02769 07655 4
0934 04854 09340 00462 07952 06324 08003 06787 0097 0869 00975 049 07577 08235 04898 02785 0428 0743 06948 04456 05469 0957 03922 037 06463 09575 07922 06555 09502 07094 09649 09595 072 00344 07547 >> B = rand(5, 9) B = 02760 04984 0753 09593 08407 03500 0357 02858 00759 06797 09597 0255 05472 02543 0966 08308 07572 00540 0655 03404 05060 0386 0843 025 05853 07537 05308 0626 05853 0699 0493 02435 0660 05497 03804 07792 090 02238 08909 02575 09293 04733 0972 05678 09340 >> A*B ans = 09286 299 8685 77 8386 234 598 3642 3783 09837 499 3083 5080 3997 0770 533 2495 0522 3780 6044 6448 008 8204 09787 292 94 4428 2960 4439 20233 4829 24020 0544 20258 807 3699 202 429 2570 203 485 06973 5093 444 0706 089 0343 5049 05253 3908 0994 4975 2978 563 064 3029 5755 0894 5298 0059 6739 4764 3959 588 652 6404 2939 6898 0972 9498 6330 498 47 2755 25493 7674 24308 5765 2555 239 8662 26 649 7527 6739 9043 09477 9478 5730 09475 >> [R,p] = rref(a*b) R = 0000 0 0 0 0 02954-2634 -05533-073 0 0000 0 0 0-0327 34223 559 0940 0 0 0000 0 0 299-2805 -09454 05034 0 0 0 0000 0-00369 -6062-08889 -2387 5
0 0 0 0 0000-05696 30357 4608 07307 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 p = 2 3 4 5 The pivot variables are x,, x 5 and free variables x 6,, x 9 The column space is spanned by the first 5 columns of AB The row space of AB is the same as the row space of R which is generated by the first five rows of R The nullspace is given by the negative of the upper right 5 4 block together with a 4 4 identity matrix, one on top of the other In other words, it is the column space of the following matrix 02954 2634 05533 073 0327 34223 559 0940 299 2805 09454 05034 00369 6062 08889 2387 05696 30357 4608 07307 0 0 0 0 0 0 0 0 0 0 0 0 Problem 6: Explain why the following statement must be true: if a subspace S is contained in another subspace V, then the orthogonal complement V is contained in the orthogonal complement S Solution (5 points) Let x V be an element in the orthogonal complement We want to show that it is in the orthogonal complement S ; that is to say for any s S, x T s = 0 But we know that x T v = 0 for any v V and in particular, s is an element in a subspace of V So, x T s = 0 We are done 6
REMARK: Since the orthogonal complement of a subspace S is defined to be the subspace whose vectors pairs to zero with the vectors in S, the intuition here is that the larger the S is, the more restriction S has, and hence the smaller S is Problem 7: If A T A x = 0 then A x = 0 Reason: A T A x = 0 means that A x in the nullspace of A x is also in the space of A These two spaces are, so their only intersection is A x = 0 Thus, A T A has the same nullspace as A (We derive this in another way in class) Solution (5 points) A T ; column space; orthogonal Problem 8: Suppose you have two matrices V and W such that C(V ) and C(W ) are orthogonal subspaces What is V T W? Solution (5 points) It has to be zero Indeed, if we write V = ( v v n ) and W = ( w w m ), v i C(V ) and w j C(W ) and v T w v T w m V T W = v n T w v n T w m because C(V ) and C(W ) are orthogonal = 0 0 0 0 Problem 9: Suppose L is a one-dimensional subspace (a line through the origin) in R 3 Its orthogonal complement L is the perpendicular to L Then (L ) is a perpendicular to L, and in fact (L ) is the same as Solution (5 points) plane; this is because dim L + dim L = 3 line; this is because dim L + dim(l ) = 3 L It is true in general that for a subspace V of R n, (V ) = V Problem 0: Let N be a matrix whose columns are a basis for the nullspace of A Then the nullspace of N T is the space of A 7
Solution (5 points) row The condition implies that the nullspace of A is the same as the column space of N, ien(a) = C(N) We also know that the null space of N T (or the left nullspace of N) is the orthogonal complement of the column space of N, ie N(N T ) = C(N), and the row space of A is the orthogonal complement of the nullspace of A, ie C(A T ) = N(A) Hence, N(N T ) = C(N) = N(A) = C(A T ) REMARK: The condition that the columns of N form a basis is not necessary, we need only that the column space of N is the same as the nullspace of A REMARK2: For a matrix A, the column space is orthogonal to the left nullspace and the row space is orthogonal to the nullspace Problem : Let A be the matrix A = ( ) 3 6 6 4 8 8 (a) Find the projection matrix P C onto C(A) (b) Find the projection matrix P R onto the row space of A (c) Compute P C AP R Explain your result (d) For any matrix A (not necessarily the one above), with P C and P R defined as the projection matrices onto A s column and row space respectively, conclude that you would get P C AP R = Solution (20 points = 5+5+5+5) (a) The column space C(A) is spanned by the vector a = P C onto C(A) is obtained via dividing aa T by a T a, which is ( ) 3 (3 ) / 4 (3 3 + 4 4) = 4 ( ) 9/25 2/25 2/25 6/25 ( ) 3 So the projection 4 3 (b) The row space C(A T ) is spanned by the vector b = 6 So the projection 6 8
P R onto C(A T ) is bb T/ 3 b T b = 6 ( 3 6 6 ) / /9 2/9 2/9 (3 3 + 6 6 + 6 6) = 2/9 4/9 4/9 6 2/9 4/9 4/9 (c) P C AP R = = ( ) ( ) 2/9 2/9 9/25 2/25 3 6 6 /9 2/9 4/9 4/9 2/25 6/25 4 8 8 2/9 4/9 4/9 ( 3 6 6 4 8 8 ) 2/9 2/9 /9 2/9 4/9 4/9 = 2/9 4/9 4/9 ( 3 6 6 4 8 8 ) = A See the explanation below (d) A This consists of two parts: P C A = A and AP R = A First, multiplying the projection matrix P C on the left is equivalent to projecting each column of A onto the subspace C(A) This will just give back A itself Thus, P C A = A Similarly, multiplying the projection matrix P R on the right is equivalent to projecting each row of A onto the subspace C(A T ) Thus, we will get back to A An alternative way to see this is to check it for each vector x We can write any vector x as x = P R x+z, where z is in the complement of the row space, which means that x is in the null space Therefore Ax = A(P R x + z) = AP R x + Az = AP R x since Az = 0, and hence A = AP R since this is true for all x Problem 2: Find the projection matrix P onto the plane x + 2y z = 0 in two ways: (a) Choose two vectors in the plane and make them the columns of a matrix A The plane is the column space Then compute P = A(A T A) A T (b) Write a vector e that is perpendicular to that plane Compute the matrix Q = e e T / e T e that projects onto the e direction Then compute P = I Q 9
Solution (20 points = 0+0) (a) We view the plane as thenull space of the matrix ( 2 ) Then a basis 2 of this space is given by, 0 Hence we take the matrix A to be 0 2 A = 0 0 Thus, A T A = ( 2 0 0 ) 2 0 = 0 (A T A) = ( ) 5 2 and 2 2 ( ) /3 /3 /3 5/6 Then the projection matrix is 2 ( ) ( ) 5/6 /3 /6 P = A(A T A) A T = 0 /3 /3 2 0 = /3 /3 /3 /3 5/6 0 0 /6 /3 5/6 (b) Since the row space of the matrix ( 2 ) is orthogonal to the nullspace, which is the plane as explained in part (a), we may take e = 2 Thus, we have Q = ee T/ 2 / /6 /3 /6 e T e = 2 4 2 6 = /3 2/3 /3 2 /6 /3 /6 Hence, 5/6 /3 /6 P = I Q = /3 /3 /3 /6 /3 5/6 Problem 3: The nullspace of A T is to the column space C(A), so if A T b = 0 then the projection of b onto C(A) should be p = Check that P b gives this answer, where P is the projection matrix P = A(A T A) A T 0
Solution (0 points) orthogonal 0; the condition says that b is in the orthogonal complement of the column space C(A) and hence it has projection 0 on that space We calculation P b as follows P b = A(A T A) A T b = A(A T A) 0 = 0 Problem 4: Explain why one must have P 2 = P, from the definition of the projection matrix P onto the column space of a matrix A (if we take a vector b and project it to the column space to get P b, then project it again, we must get ) Check explicitly that this is true from the formula P = A(A T A) A T Solution (0 points) P b; this is because we can write P 2 b as P (P b) and the projection restricted to the column space of A is identity, so further composition of P b with P will still give P b If P = A(A T A) A T, we have P b = A(A T A) A T b P b = ( A(A T A) A T) 2 b = A(A T A) A T A(A T A) A T b ( ) = A (A T A) (A T A) (A T A) A T b = A I (A T A) A T b = P b