Orthogonal Diagonalization of Symmetric Matrices

Transcription

1 MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding a new, orthogonal, basis v 1,..., v k by setting v 1 = u 1. Then we seek v 2 = u 2 + c 1 u 1 with unknown coeff. c 1 ; we need v 2 v 1 = 0, that is, u 2 u 1 + c 1 ( u 1 u 1 ) = 0, whence c 1 = u 2 u 1 u 1 u 1 (note that u 1 u 1 0 as u 1 0 being a vector in a basis). Next we seek v 3 = u 3 + d 1 v 1 + d 2 v 2 with unknown coeff. d 1, d 2 ; we need v 3 v 1 = 0, that is, u 3 v 1 + c 1 ( v 1 v 1 ) = 0 (the term with v 2 v 1 = 0 vanishes), whence d 1 = u 3 v 1 v 1 v 1 ; also we need v 3 v 2 = 0, that is, u 3 v 2 + c 1 ( v 2 v 2 ) = 0 (the term with v 1 v 2 = 0 vanishes), whence d 2 = u 3 v 2 v 2 v 2....And so on, at each step we seek v j = u j + linear comb. of preceding v 1,..., v j 1 with unknown coeff., which are easily found. Orthogonal Diagonalization of Symmetric Matrices Definition. A square matrix A is orthogonally diagonalizable if there exists an orthogonal matrix Q such that Q T AQ = D is a diagonal matrix. Remarks. Since Q T = Q 1 for orthogonal Q, the equality Q T AQ = D is the same as Q 1 AQ = D, so A D, so this a special case of diagonalization: the diagonal entries of D are eigenvalues of A, and the columns of Q are corresponding eigenvectors. The only difference is the additional requirement that Q be orthogonal, which is equivalent to the fact that those eigenvectors columns of Q form an orthonormal basis of R n. If a matrix is orthogonally diagonalizable, then it is sym- Theorem metric. Proof. We have Q T AQ = D; times Q on the left, and Q T on the right gives A = QDQ T (since Q T = Q 1 ). Then A T = (QDQ T ) T = (Q T ) T D T Q T = QDQ T = A, so A is symmetric. Theorem All eigenvalues (all roots of the characteristic polynomial) of a symmetric matrix are real. Theorem Eigenvectors of a symmetric matrix corresponding to different eigenvalues are orthogonal. Proof. Let A T = A have eigenvectors v 1 and v 2 for eigenvalues λ 1 λ 2. We compute the dot product (A v 1 ) v 2 = (λ 1 v 1 ) v 2 = λ 1 ( v 1 v 2 ). On the other hand, the left-hand side can be written as a matrix product: (A v 1 ) v 2 =

2 MATH10212 Linear Algebra Brief lecture notes 58 (A v 1 ) T v 2 = v T 1 A T v 2 = v T 1 (A v 2 ) = v 1 (λ 2 v 2 ) = λ 2 ( v 1 v 2 ). Thus, λ 1 ( v 1 v 2 ) = λ 2 ( v 1 v 2 ). Since λ 1 λ 2 by hypothesis, we must have v 1 v 2 = 0. Theorem Every symmetric matrix is orthogonally diagonalizable. Method for orthogonal diagonalization of a symmetric matrix. Find eigenvalues of A. Find the eigenspace for each eigenvalue. For repeated eigenvalues (when the dimension of the eigenspace is greater than 1) apply Gram Schmidt orthogonalization to find an orthogonal basis. Together, these orthogonal bases of eigenspaces form an orthogonal basis of R n. Normalize, dividing each vector of the basis by its length. The resulting orthonormal basis can be taken as columns of Q such that Q T AQ = Q 1 AQ = D, where D is diagonal with the eigenvalues of A on the diagonal, in the order corresponding to the order of their eigenvectors as columns of Q. Example. For A = the characteristic polynomial is λ 2 2 det(a λi) = 2 1 λ λ = (1 λ) (1 λ) 4(1 λ) 4(1 λ) = = (λ 5)(λ + 1) 2. Thus, eigenvalues are 5 and 1. Eigenspace E 1 : (A ( 1)I) x = 0; x x 2 = 0 0 ; x 1 = x 2 x 3, x 3 0 s t where x 2, x 3 are free var.; E 1 = s s, t R ; t a basis of E 1 : u 1 = 1 1, u 2 = Apply Gram Schmidt to orthogonalize: v 1 = u 1 ; seek v 2 = u 2 + c v 1 ; for v 2 v 1 = 0 obtain c = u2 v1 v 1 v 1 = 1 2, thus, v 2 = u 2 (1/2) v 1 = 1/2 1/2. 1 Eigenspace E 5 : (A 5I) x = 0; x x 2 = 0 0 ; solve this x 3 0 system...: x 1 = x 2 = x 3, where x 3 is a free var.; E 5 = t t t R ; t a basis of E 5 : (Note, E 5 is automatically orthogonal to E 1.)

3 MATH10212 Linear Algebra Brief lecture notes 59 Together, we have an orthogonal basis of R 3 consisting of eigenvectors: 1 1, 1/2, Normalize: 1/ 2 1/ 1/ 6 2, 1/ 1/ 3 6, 1/ 3 0 2/3 1/. 3 1/ 2 1/ 6 1/ 3 Let Q = 1/ 2 1/ 6 1/ 3, which is an orthogonal matrix; 0 2/3 1/ 3 then Q T AQ = Orthogonal Complements and Orthogonal Projections Definition Let W be a subspace of R n. We say that a vector v in R n is orthogonal to W if v is orthogonal to every vector in W. The set of all vectors that are orthogonal to W is called the orthogonal complement of W, denoted W. That is, W = { v in R n v w = 0 for all w in W } Theorem 5.9 Let W be a subspace of R n. a. W is a subspace of R n. b. (W ) = W. c. W W = { 0}. d. If W = span( w 1,..., w k ), then v is in W if and only if v w i = 0 for all i = 1,..., k. Theorem 5.10 Let A be an m n matrix. Then the orthogonal complement of the row space of A is the null space of A, and the orthogonal complement of the column space of A is the null space of A T : (row(a)) =null(a) and (col(a)) =null(a T )

4 MATH10212 Linear Algebra Brief lecture notes 60 Orthogonal Projections Theorem 5.11 The Orthogonal Decomposition Theorem Let V be a subspace of R n and let u be a vector in R n. Then there are unique vectors v V and w V such that u = v + w. In fact: if v 1,..., v r is an orthogonal basis of V, then an orthogonal basis v r+1,..., v n of V must have n r vectors, and together these n vectors form an orthogonal basis v 1,..., v r, v r+1,..., v n of R n Definition In Theorem 5.11, the vector v is called the orthogonal projection of u onto V, and the length of w is called the distance from u to V. Method for finding the orthogonal projection and the distance a subspace V of R n and some vector u R n ). (given Choose an orthogonal basis v 1,..., v r of V (if V is given as a span of some non-orthogonal vectors, apply Gram Schmidt first to obtain an orthogonal basis of V ); we know that there is an orthogonal basis v r+1,..., v n of V such that v 1,..., v r, v r+1,..., v n is an orthogonal basis of R n (but we do not really need these v r+1,..., v n!). Then u = r a i v i + i=1 n j=r+1 b j v j. We now find the coefficients a i. For that, take dot product with v i0 for each i 0 = 1,..., r: on the right only one term does not vanish, since the v k are orthogonal to each other: u v i0 = a i0 ( v i0 v i0 ), whence a i0 = u v i 0 v i0 v i0. Having found these a i0 for each i 0 = 1,..., r, we now have the orthogonal projection of u onto V : r v = a i v i. i=1 The orthogonal component is found by subtracting: w = u v V, and the distance from u to V is w.

5 MATH10212 Linear Algebra Brief lecture notes 61 Corollary 5.12 If W is a subspace of R n, then (W ) = W Theorem 5.13 If W is a subspace of R n, then dim W + dim W = n Corollary 5.14 The Rank Theorem If A is an m n matrix, then rank (A) + nullity (A) = n Vector Spaces and Subspaces Definition Let V be a set on which two operations, called addition and scalar multiplication, have been defined. If u and v are defined in V, the sum of u and v is denoted by u + v, and if c is a scalar, the scalar multiple of u is denoted by c u. If the following axioms hold for all u, v and w in V and for all scalars c and d, then V is called a vector space and its elements are called vectors. 1. u + v is in V. Closure under addition 2. u + v = v + u Commutativity 3. ( u + v) + w = u + ( v + w) Associativity 4. There exists an element 0 in V, called a zero vector such that u+ 0 = u. 5. For each u in V, there is an element u in V such that u + ( u) = c u is in V. Closure under scalar multiplication 7. c( u + v) = c u + c v Distributivity 8. (c + d) u = c u + d u Distributivity 9. c(d u) = (cd) u u = u

6 MATH10212 Linear Algebra Brief lecture notes 62 Theorem 6.1 Let V be a vector space, u a vector in V, and c a scalar. a. 0 u = 0 b. c 0 = 0 c. ( 1) u = u d. If c u = 0, then c = 0 or u = 0. Subspaces Definition A subset W of a vector space V is called a subspace of V if W is itself a vector space with the same scalars, addition and scalar multiplication as V. Theorem 6.2 Let V be a vector space and let W be a nonempty subset of V. Then W is a subspace of V if and only if the following conditions hold: a. If u and v are in W, then u + v is in W. b. If u is in W and c is a scalar, then c u is in W. Example 6.14 If V is a vector space, then V is clearly a subspace of itself. The set { 0}, consisting of only the zero vector, is also a subspace of V, called the zero subspace. To show this, we simply note that the two closure conditions of Theorem 6.2 are satisfied: = 0 and c 0 = 0 for any scalar c The subspaces { 0} and V are called the trivial subspaces of V. Spanning Sets Definition If S = { v 1, v 2,..., v k } is a set of vectors in a vector space V, then the set of all linear combinations of v 1, v 2,..., v k is called the span of and is denoted by v 1, v 2,..., v k span( v 1, v 2,..., v k ) or span(s). If V =span(s), then S is called a spanning set for V and V is said to be spanned by S.

7 MATH10212 Linear Algebra Brief lecture notes 63 Theorem 6.3 Let v 1, v 2,..., v k be vectors in a vector space V. a. span( v 1, v 2,..., v k ) is a subspace of V. b. span( v 1, v 2,..., v k ) is a smallest subspace of V that contains v 1, v 2,..., v k. Linear Independence, Basis and Dimension Linear Independence Definition A set of vectors S = { v 1, v 2,..., v k } in a vector space V is linearly dependent if there are scalars c 1, c 2,..., c k at least one of which is not zero, such that c 1 v 1 + c 2 v c k v k = 0 A set of vectors that is not linearly dependent is said to be linearly independent. Theorem 6.4 A set of vectors S = { v 1, v 2,..., v k } in a vector space V is linearly dependent if and only if at least one of the vectors can be expressed as a linear combination of the others. Bases Definition A subset B of a vector space V is a basis for V if 1. B spans V and 2. B is linearly independent.