Eigenvalues and eigenvectors Tom Lyche University of Oslo Norway Eigenvalues and eigenvectors p. 1/2
eigenpair Suppose A C n,n is a square matrix, λ C and x C n. We say that (λ, x) is an eigenpair for A if Ax = λx and x is nonzero. The scalar λ is called an eigenvalue and x is said to be an eigenvector. If (λ, x) is an eigenpair for A then (λ, αx) is an eigenpair for any α C with α 0. An eigenvector is a special vector that is mapped by A into a vector parallel to itself. The length is increased if λ > 1 and decreased if λ < 1. The set of distinct eigenvalues is called the spectrum of A and is denoted by σ(a). Eigenvalues and eigenvectors p. 2/2
Characteristic polynomial For any A C n,n we have λ σ(a) π A (λ) := det(a λi) = 0. The function π A : C C given by π A (λ) = det(a λi) is called the characteristic polynomial of A. It is a polynomial of exact degree n. The eigenvalues are the roots of this polynomial. By the fundamental theorem of algebra an n n matrix has precisely n eigenvalues λ 1,...,λ n. Some of the eigenvalues can be equal (multiple). If the matrix is real then the coefficients of the characteristic polynomial are real A real matrix can have complex eigenvalues, but the complex eigenvalues occur in complex conjugate pairs. Eigenvalues and eigenvectors p. 3/2
Sums, products and trace In general det(λi A) = (λ a 11 )(λ a 22 ) (λ a nn ) + r(λ) = λ n trace(a)λ n 1 + + ( 1) n det(a)), where trace(a) := a 11 + a 22 + + a nn. Theorem 1. For any A C n,n trace(a) = λ 1 + λ 2 + + λ n, det(a) = λ 1 λ 2 λ n, (1) Eigenvalues and eigenvectors p. 4/2
Zero eigenvalue = Singular In terms of eigenvalues we have an additional characterization of a singular matrix. Theorem 2. The matrix A C n,n is singular if and only if zero is an eigenvalue. Proof. Zero is an eigenvalue if and only if π A (0) = det(a) = 0 which happens if and only if A is singular. Eigenvalues and eigenvectors p. 5/2
Some special cases The eigenvalues of a triangular matrix are given by its diagonal entries. If A = [ B C 0 D ] is block triangular then πa = π B π D. For any A C m,n and B C n,m the matrices AB and BA have the same spectrum. More precisely λ n π AB (λ) = λ m π BA (λ), λ C. Eigenvalues and eigenvectors p. 6/2
More on eigenvalues Theorem 3. Suppose (λ,x) is an eigenpair for A C n,n. Then 1. If A is nonsingular then (λ 1,x) is an eigenpair for A 1. 2. (λ k,x) is an eigenpair for A k for k N. 3. If p(t) = a 0 + a 1 t + a 2 t 2 + + a k t k is a polynomial then (p(λ),x) is an eigenpair for the matrix p(a) := a 0 I + a 1 A + a 2 A 2 + + a k A k. 4. λ is an eigenvalue for A T. 5. λ is an eigenvalue for A H. Eigenvalues and eigenvectors p. 7/2
Similarity transformation Definition 4. Two matrices A,B C n,n are said to be similar if there is a nonsingular matrix S C n,n such that B = S 1 AS. The transformation A B is called a similarity transformation Theorem 5. Similar matrices have the same eigenvalues with the same algebraic and geometric multiplicities. (λ, x) is an eigenpair for B = S 1 AS if and only if (λ,sx) is an eigenpair for A. Eigenvalues and eigenvectors p. 8/2
Eigenvectors Consider I = 1 0 0 0 1 0, J 3 := 0 0 1 1 1 0 0 1 1 0 0 1 σ(i) = σ(a) = {1}. Any vector in R 3 is an eigenvector of I. Any eigenvector of J 3 must be a multiple of e 1. Eigenvalues and eigenvectors p. 9/2
Basis of eigenvectors Theorem 6. Eigenvectors corresponding to distinct eigenvalues are linearly independent. Corollary 7. If A C n,n has distinct eigenvalues then the corresponding eigenvectors form a basis for C n. For multiple eigenvalues the situation is more complicated. Eigenvalues and eigenvectors p. 10/2
Algebraic and geometric multiplicity Definition 8. We say that an eigenvalue λ of A has algebraic multiplicity a = a(λ) = a A (λ) N, if π A (z) = (z λ) a p(z), where p(λ) 0. Definition 9. The geometric multiplicity g = g(λ) = g A (λ) of an eigenvalue λ of A is the dimension of the nullspace ker(a λi). Example: 1 0 0 1 1 0 I = 0 1 0, J 3 := 0 1 1 0 0 1 0 0 1 a I (1) = 3, g I (1) = 3, a J3 (1) = 3, g J3 (1) = 1. Eigenvalues and eigenvectors p. 11/2
g a g A (λ) a A (λ) for any λ σ(a). An eigenvalue where g A (λ) < a A (λ) is said to be defective A matrix is defective if at least one of its eigenvalues is defective. A matrix A C n,n has n linearly independent eigenvectors if and only if the algebraic and geometric multiplicity of all eigenvalues are the same. Eigenvalues and eigenvectors p. 12/2
Unitary similarity transformations If B = S 1 AS and S = U C n,n is unitary then S 1 = U H and B = U H AU. We say that B is unitary similar to A. If B is diagonal then the orthonormal columns of U are eigenvectors of A. It follows that if A is unitary similar to a diagonal matrix then A has a set of orthonormal eigenvectors. Unitary transformations are particularly important in numerical algorithms since they are insensitive to noise in the entries of the matrix. Eigenvalues and eigenvectors p. 13/2
Hermitian matrices A matrix A C n,n is Hermitian ifa H = A. A matrix A C n,n is symmetric ifa T = A. Hermitian=Symmetric for real matrices A Hermitian matrix has real eigenvalues (even if the entries in A are complex numbers). For if Ax = λx then we can multiply both sides by x H and divide by x H x to obtain λ = xh Ax x H x. But since A H = A we obtain λ = xh A H x x H x and λ is real. = xh Ax x H x = λ, Eigenvalues and eigenvectors p. 14/2
Schur Triangularization Theorem 10. For each A C n,n there exists a unitary matrix U C n,n such that U H AU is upper triangular. Eigenvalues and eigenvectors p. 15/2
Normal matrices Definition 11 (Normal Matrix). A matrix A C n,n is said to be normal if AA H = A H A. Examples of normal matrices are 1. A H = A, (Hermitian) 2. A H = A, (Skew-Hermitian) 3. A H = A 1 (Unitary) The following theorem says that a matrix has orthonormal eigenvectors if and only if it is normal. Theorem 12. A matrix A C n,n is unitary similar with a diagonal matrix if and only if it is normal. Eigenvalues and eigenvectors p. 16/2
The Spectral Theorem Theorem 13. Suppose A R n,n, A T = A. Then A has real eigenvalues λ 1,λ 2,...,λ n. Moreover, there is an orthogonal matrix U R n,n such that U T AU = diag(λ 1,λ 2,...,λ n ). The columns of U are n orthonormal eigenvectors of A. Eigenvalues and eigenvectors p. 17/2
The Real Schur Form A real matrix with complex eigenvalues cannot be reduced to triangular form by an orthogonal similarity transformation. [ ] µ υ M = (2) υ µ has complex eigenvalues λ and λ. We say that a matrix is quasi-triangular if it is block triangular with only 1 1 and 2 2 blocks on the diagonal. Moreover, no 2 2 block should have real eigenvalues. Eigenvalues and eigenvectors p. 18/2
As an example consider D = 2 1 3 4 5 1 2 4 3 2 0 0 1 2 3 0 0 0 3 2 0 0 0 1 1 D 1 = 2 1 1 2, D 2 = [ 1 ], D 3 = 3 2 1 1. The eigenvalues of D are the union of the eigenvalues of D 1,D 2,D 3. The eigenvalues of D 1 are 2+i and 2 i, while D 2 has eigenvalue 1, D 3 has the same eigenvalues as D 1, 2+i and 2 i. Thus D has one real eigenvalue 1 corresponding to the 1 1 block and complex eigenvalues 2+i, 2 i with multiplicity 2 corresponding to the two 2 2 blocks. Eigenvalues and eigenvectors p. 19/2
The Real Schur Form Theorem 14. Suppose A R n,n. Then we can find U R n,n with U T U = I such that U T AU is quasi-triangular. Eigenvalues and eigenvectors p. 20/2
The Jordan form We have seen any square matrix can be triangularized by a unitary similarity transformation. any nondefective matrix can be diagonalized how close to a diagonal matrix can we reduce a defective matrix by a similarity transformation? Definition 15. A Jordan block, denoted J m (λ) is an m m matrix of the form J m (λ) := λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0.. 0 0 0 λ 1 0 0 0 0 λ Eigenvalues and eigenvectors p. 21/2
Eigenpairs of a Jordan block A 3 3 Jordan block: J 3 (λ) = [ λ 1 0 ] 0 λ 1 0 0 λ λ is an eigenvalue of J m (λ) of algebraic multiplicity m and geometric multiplicity one. Any eigenvector must be a multiple of e 1. The Jordan canonical form is a similarity decomposition of a matrix into Jordan blocks. Eigenvalues and eigenvectors p. 22/2
The Jordan Form Theorem 16. Suppose A C n,n has k distinct eigenvalues λ 1,...,λ k of algebraic multiplicities a 1,...,a k and geometric multiplicities g 1,...,g k. There is a nonsingular matrix S C n,n such that S 1 AS = diag(u 1,...,U k ), with J i C a i,a i, (3) where U i = diag(j mi,1(λ i ),...,J mi,g i (λ i ). (4) Here m i,1,...,m i,gi are unique integers so that a i = g i j=1 m i,j for all i. Each U i contains g i Jordan blocks. The columns of S are called principal vectors. Eigenvalues and eigenvectors p. 23/2
Jordan example A = SJS 1. J := diag(j 1,J 2 ) = 2 1 0 2 2 2 1 0 0 2 1 0 0 2 3 1 0 3 R8,8. Here J 1 = diag(j 2 (2),J 1 (2),J 3 (2)) and J 2 = J 2 (3). 2 is an eigenvalue of algebraic multiplicity 6 and geometric multiplicity 3. 3 is an eigenvalue of algebraic multiplicity 2 and geometric multiplicity 1. Eigenvalues and eigenvectors p. 24/2
Principal vectors Let A = SJS 1. Then AS = SJ and with S = [s 1,...,s 8 ] we find As 1 = 2s 1, As 2 = 2s 2 + s 1, As 3 = 2s 3, As 4 = 2s 4, As 5 = 2s 5 + s 4, As 6 = 2s 6 + s 5, As 7 = 3s 7, As 8 = 3s 8 + s 7, The first principal vector in each Jordan block is an eigenvector of A. The others are not. Eigenvalues and eigenvectors p. 25/2
Powers of A = SJS 1 A 2 = SJS 1 SJS 1 = SJ 2 S 1 A r = SJ r S 1 J r = diag(u r 1,...,U r k) U r i = diag(j mi,1(λ i ) r,...,j mi,g i (λ i ) r ) J 4 (λ) r = (E 4 + λi 4 ) r, where E 4 := E 2 4 = [ 0 0 1 0 ] 0 0 0 1 0 0 0 0, E 3 4 = 0 0 0 0 E r 4 = 0 for r 4. [ 0 0 0 1 ] 0 0 0 0 0 0 0 0 0 0 0 0 J m (λ) r = (E m + λi) r = m 1 k=0 [ 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 ] ( r ) k λ r k E k m for r m 1. Eigenvalues and eigenvectors p. 26/2