Solutions to Assignment 12

Transcription

1 Solutions to Assignment Math 7, Fall 6.7. Let P have the inner product given by evaluation at,,, and. Let p =, p = t and q = /(t 5). Find the best approximation to p(t) = t by polynomials in Span{p, p, q}. Let W = Span{p, p, q}. First I note that {p, p, q} is a linearly independent set (this is clear because each of the polynomials has a different degree, so obviously no non-trivial linear combination can be zero). Now p, p =, p.q = each equal zero, so {p, p, q} is an orthogonal basis., and p, q = At this point approximating p = t is just a matter of following the formula. First we compute a few dot products: 7 7 p, p = =, p, p = = 6, p, q = =, p, p = =, 7 p, p = =, and q, q = =. So now proj W (t ) = <p,p> p <p,p > + <p,p> p <p,p > + <q,p> q = 6 + t+ ( <q,q> (t 5)) = t Let T be a one-to-one linear transformation from a vector space V into R n. Show that for u, v V, the formula u, v = T (u) T (v) defines an inner product on V.

2 Well, there are some things we have to check. For all v, u R n, note that u, v = T (u) T (v) = T (v) T (u) = v, u, where the second equality is true because the dot product is commutative (theorem, page 76). Furthermore, for all v, u, w R n, u + v, w = T (u + v) T (w) = (T (u) + T (v)) T (w) = (T (u) T (w)) + (T (v) T (w)) = u, w + v, w. In this case the second inequality holds because T is a linear transformation, and the third holds because the dot product distributes (theorem, page 76). If c R and u, v R, then cu, v = T (cu) T (v) = (ct (u)) T (v) = c(t (u) T (v)) = c u, v. Here the second inequality holds because T is a linear transformation, and the third holds by theorem theorem, page 76. Finally, note that for each u R, u, u = T (u) T (u). Of course, T (u) is a vector in R n, so again by theorem page 76, we know that T (u) T (u) and equals zero if and only if T (u) =. But T is a one-to-one linear transformation. We know that T () = (see page 77) and thus if T (u) = this implies that u =. That is, of course, what we needed to prove, so we are finished Suppose 5 out of 5 data points in a weighted least-squares problem have a y-measurement that is less reliable then the others, and they are to be weighted half as much as the other points. One method is to weight the points by a factor of and the other 5 by a factor of /. A second method is to weight the points by a factor of and the other 5 by a factor of. Do the two methods produce different results? Explain. The answer is no. We can rename the data points so that the first five are the ones with suspect measurements. Then given weights w = = w 5 =, w 6 = w 5 =, and w = = w 5 = /, w 6 = w 5 =, let W and W be the diagonal matrices whose diagonal entries are the w i and the w i respectively. Note that W = W. Now, if we write A to be the original design matrix, and y to be the original design vector, solving the weighted least squares problem for Ax = y with weights w,..., w n is equivalent to solving the least squares problem for W Ax = W y, that is, it is the same as solving the system (W A) T W Ax = (W A) T W y (see page 8). This last equation can be rewritten as A T W Ax = A T W y because W is symmetric. Similarly, solving the weighted least squares problem for Ax = y with weights w,..., w n is equivalent to solving A T (W ) Ax = A T (W ) y. Both of these two systems, however, have the same solution, because A T (W ) Ax = A T (W ) Ax = A T (W ) Ax and

3 A T (W ) y = A T (W ) y = A T (W ) y. That is, multiplying both sides of A T (W ) Ax = A T (W ) y by four (which doesn t change the solution space) gives us the system A T (W ) Ax = A T (W ) y Let A = 5, v = and v =. Verity that v and v are eigenvectors of A. Then orthogonally diagonalize A. To verify that these are eigenvectors, note that Av = = v, and Av = = v. In order to orthogonally diagonalize A, we need to find another orthogonal eigenvector and its corresponding eigenvalue. Whatever this a vector is (for now we write it as v = b, it must have that v v = = v v, c or rather, that a + b + c = and a + b =. We conclude that b = a, and hence that c = a. So, take a =. Then v =. By construction this is orthogonal to each of v and v. Note that Av = = v. Because v and v are linearly independent (because there are only two it is enough to check that one is not the multiple of the other), this is a linearly independent set. So we can write P = and thus A = (here we have replaced each of v i with. v i v i ), 7..8 Show that if A is an n n symmetric matrix, then (Ax) y = x (Ay) for all y, y R n. This is a straight forward application of the properties of the dot product and

4 the fact that A = A T. Note that (x) y = (Ax) T y = x T A T y = x T (A T y) = x T (Ay) = x (Ay). 7.. Suppose A and B are both orthogonally diagonalizable and AB = BA. Explain why AB is also orthogonally diagonalizable. By theorem, page 5, it is enough to show that AB is symmetric so we need to show that (AB) T = AB. Note that by the same theorem, both A and B are symmetric, and thus AB = A T B T = (BA) T. But BA = AB, so we conclude that AB = A T B T = (BA) T = (AB) T as required Let A be the matrix of the quadratic form 9x + 7x + x 8x x + 8x x. It can be shown that the eigenvalues of A are, 9, and 5. Find an orthogonal matrix P such that the change of variable x = P x transforms x T Ax into a quadratic form with no cross-product term. Given P and the new quadratic form. To find the matrix A, we let the ith diagonal entry of A be the coefficient of x i, and the i, jth entry (for i j) be half of the coefficient of x i x j (which is the same as saying that we let it be half of the coefficient of x j x i ). Thus this quadratic form is given by x T Ax where 9 A = 7. We row reduce the system A λi = for each of the three eigenvalues of A, and obtain eigenvectors v = corresponding to λ =, / v = corresponding to λ = 9, and v = / corresponding to λ = 5. Because eigenvectors corresponding to different eigenvalues are orthogonal for symmetric matrices (theorem, page 5), we conclude that {v, v, v } is an

5 orthogonal set. So replace each of v i with v i to obtain an orthonormal set. The resulting orthonormal diagonalization of A is A = P DP T = 9. 5 So the change of variables we are looking for is y = P T x for The new quadratic form is P = v i (P y) T A(P y) = y T P T AP y = y T Dy = y + 9y + y. Note: there are other possibilities, depending on the order you choose to put the eigenvalues in D What is the largest value of the quadratic from 5x x if x T x =? [ ] Write Q(x) = 5x x. Then we know that if x =, then x T x =, and Q(x) = 5, so the maximum is at least 5. On the other hand, because x is positive, for all x such that x + x =, we have that 5x x 5x x + 8x = 5(x + x ) = 5() = 5. Thus the maximum is at most 5. We conclude that the maximum must be at exactly Show that if an n n matrix A is positive definite, then there exists a positive definite matrix B such that A = B T B. Note that A is symmetric (positive definite matrices are always symmetric because we only define this notion in the symmetric case see page 6). Thus by theorem, page 5, we know that A is diagonalizable. In particular, we can write A = P DP T where P is orthogonal and D is a diagonal matrix whose entries are the eigenvalues of A. Write λ λ D = λ n 5

6 Because A is positive definite, each of the λ i (the eigenvalues of A) are positive. Thus we can let C be the diagonal matrix whose ith diagonal entry is λ i. Thus CC = D (recall that the result of multiplying diagonal matrices is simply to multiply together the elements on their diagonals). Now let B = P CP T, and it remains to show that B T B = A. This is not difficult. Turning the crank, B T B = (P CP T ) T (P CP T ) = P TT C T P T P CP T = P CP T P CP T = P CCP T = P DP T = A, as required (we used that C T = C because C is diagonal). 6