Orthogonal Projections and Orthonormal Bases

Transcription

1 CS 3, HANDOUT -A, 3 November 04 (adjusted on 7 November 04) Orthogonal Projections and Orthonormal Bases (continuation of Handout 07 of 6 September 04)

2 Definition (Orthogonality, length, unit vectors). Two vectors v, w R n are orthogonal if v w = 0.. The length (or norm) of a vector v R n is v = v v. 3. A vector u R n is a unit vector if u =. Definition (Orthonormal vectors) The vectors u,..., u k R n are called orthonormal if:. u = = u k =,. u i u j = 0 for all i < j k.

3 Theorem 3 (Properties of orthonormal vectors). Orthonormal vectors are linearly independent.. If the vectors u,..., u n R n are orthonormal, they form a basis of R n. Theorem 4 (Orthogonal projection) Consider a vector x R n and a subspace V of R n. Then we can write x = x + x where x is in V and x is perpendicular to V, and this representation is unique. x is called the orthogonal projection of x on V. 3

4 Example of orthonormal vectors in the hyperspace R 4 The vectors: u =, u =, u 3 = are orthonormal. (Why?). Is Span{ u, u, u 3 } = R 4? No. (Why?). Are there unit vectors u 4 in R 4 that are orthogonal to u, u, and u 3? Yes, there are exactly two such unit vectors u 4. (Why?) 3. For such a vector R 4 as in part, if it exists, is it the case that Span{ u, u, u 3,, u 4 } = R 4? Yes. (Why?) 4

5 Some intuition from R and R 3, helpful in understanding the dot product (see pp in [Lay]): If u and v are non-zero vectors in R or R 3, then there is a nice connection between the dot product u v and the angle θ between the two vectors. As usual, we assume one common endpoint of u and v is the origin, and their respective second endpoint is determined by two coordinates (in R ) or three coordinates (in R 3 ). We verify this formula for: u v = u v cos θ θ = 0, in which case cos θ = and u v = u v. θ = 90, in which case cos θ = 0 and u v = 0. θ in general, see pages in [Lay]. This verification is not possible in hyperspace R n, with n 4, but is still helpful in understanding the next theorem. 5

6 Theorem 5 (Formula for orthogonal projection) [Theorem 8, page 348, and Theorem 0, page 35, in [Lay]]. Let V be a subspace of R n with orthogonal basis u,..., u k R n. Then, for every x R n, we have proj V ( x) = x = u x u u u + + u k x u k u k u k. Let V be a subspace of R n with orthonormal basis u,..., u k R n. Then, for every x R n, we have proj V ( x) = x = ( u x) u + + ( u k x) u k Corollary 6 Let V = R n with orthonormal basis u,..., u n R n. Then, for every x R n, we have x = ( u x) u + + ( u n x) u n 6

7 The following is a simpler re-statement of the preceding corollary: Corollary 7 If u,..., u n R n is an orthonormal basis for R n, then any vector x R n can be uniquely expressed as a linear combination of the basis: x = c u + + c n u n where c i = u i x for every i n. 7

8 Definition 8 (Orthogonal complement) Let V be a subspace of R n. The orthogonal complement V of V is the set of all vectors x in R n that are orthogonal to all vectors in V : V = { x R n v x = 0 for every v V } Remark 9 If V is a subspace of R n, then the orthogonal projection proj V ( ) on V is a linear transformation from R n to V. (You believe this?) Then the kernel of the orthogonal projection, i.e., ker(proj V ( )), is precisely the orthogonal complement V. Theorem 0 (Properties of the orthogonal complement) Let V be a subspace of R n.. The orthogonal complement V if V is a subspace of R n.. The intersection of V and V consists of the zero vector: V V = O. 3. dim(v ) + dim(v ) = n. 8

9 Exercise : Let V be a subspace of R n with orthonormal basis u,..., u k R n. Show that the orthogonal projection proj V ( ) on V is a linear transformation from R n to V. Hint: Show that the two defining properties of a linear transformation (i) and (ii) on page 65 of [Lay] are satisfied by proj V ( ). Exercise : Let V be the subspace V of R 4 spanned by two vectors u = (,,, ) and u = (,,, ).. Find an orthonormal basis for V.. Find proj V ( x) where x = (, 3,, 7) and verify that ( x proj V ( x)) is perpendicular to both u and u. 3. By the preceding exercise, proj V ( ) is a linear transformation. Find a 4 4 matrix A that represents the linear transformation proj V ( ). Hint for part : This is not trivial. You may get some help by searching the Web with the key Find the matrix of a linear transformation or some such. It is helpful to keep in mind that the particular V in this exercise has dimension as a subspace of R 4, so that rank(a) = dim(im(a)) = dim(v ) = and dim(ker(a)) = dim(v ) =. 9

10 Theorem 3 (Pythagorean theorem) [Theorem, page 334, [Lay] ] Let x, y R n. The equation x + y = x + y holds if and only if x and y are orthogonal. Theorem 4 (Inequality for the magnitude of proj V ( x)) Let V be a subspace of R n and x R n. Then proj V ( x) x The statement is an equality iff x V. 0