Eigenvalues and Eigenvectors

Transcription

1 Math 20F Linear Algebra Lecture 2 Eigenvalues and Eigenvectors Slide Review: Formula for the inverse matrix. Cramer s rule. Determinants, areas and volumes. Definition of eigenvalues and eigenvectors. Review Theorem (Formula for the inverse matrix) If A be an n n matrix with det(a) = 0, then ( A ) ij = C ji. Slide 2 where C ij = ( ) i+j det(a ij ). Theorem 2 (Cramer s rule) If the matrix A = a,, a n is invertible, then the linear system Ax = b has a unique solution for every vector b, given by x i = det(a i(b)). where x i is the i component of x, and A i (b) = a,, b,, a n, with b in the i column.

2 Math 20F Linear Algebra Lecture 2 2 Determinant, areas and volumes Slide 3 Theorem 3 Let A = a,, a n be an n n matrix. If n = 2, then det(a) is the area of the parallelogram determined by a, a 2. If n = 3, then det(a) is the volume of the parallelepiped determined by a, a 2, and a 3. Determinant, areas and volumes Sketch of the proof for n = 2. The elementary row operations add to one row a multiple of another row; switch two rows; Slide 4 leave the absolute value of the determinant unchanged, and they also leave the area of the parallelogram unchanged. These operations transform any parallelogram into a rectangle. In the case of a rectangle, the determinant of the matrix constructed with the vectors that form the rectangle is the area of the rectangle. Therefore, the theorem follows in the case n = 2. Same argument holds for n = 3.

3 Math 20F Linear Algebra Lecture 2 3 Eigenvalues and eigenvectors Definition (Eigenvalues and eigenvectors) Let A be an n n matrix. A number λ is an eigenvalue of A if there exists a nonzero vector x IR n such that Slide 5 Ax = λx. The vector x is called an eigenvalue of A corresponding to λ. Notice: If x is an eigenvector, then tx with t 0 is also an eigenvector. Definition 2 (Eigenspace) Let λ be an eigenvalue of A. The set of all vectors x solutions of Ax = λx is called the eigenspace E(λ). That is, E(λ) = { all eigenvectors with eigenvalue λ, and 0}. Examples Consider the matrix A = 3 3. Slide 6 Show that the vectors v =, T and v 2 =, T are eigenvectors of A and find the associated eigenvalues. Av 2 = Av = Then, λ = 4 and λ 2 = 2. = = = 4 = 2 = 4v. = 2v 2.

4 Math 20F Linear Algebra Lecture 2 4 Examples Slide 7 Is v =, 2 T an eigenvector of matrix A given above? The answer is no, because of the following calculation. 3 7 Av = = λ Examples Slide 8 Consider the matrix A = Show that the vectors v = 2, T is an eigenvector of A, and find the associated eigenvalue. Av = = 0 0. = 0 2. Therefore, λ = 0.

5 Math 20F Linear Algebra Lecture 22 5 Examples Is there any other eigenvalue of the matrix A above? One has to find the solutions of Ax = λx. 2 x λx Ax = = 3 6 λx 2 x 2 Slide 9 x + 2x 2 = λx 3x + 6x 2 = λx 2 x + 2x 2 = λ 3 x2. Therefore λx = λx 2/3, that is λ ( x 3 x2 ) = 0. This implies that λ = or 3x = x + 2. The first case corresponds to the eigenvalue zero, already studied above, which has the eigenvector v = 2, T. The other case gives an eigenvector satisfying 3x = x 2, so one possible solution is v 2 =. 3 Examples Slide 0 We compute the eigenvalue associated to v 2 =, 3 T 2 7 Ax = = = Therefore, the eigenvalue is λ 2 = 7. This calculation seems complicated because one computes eigenvalues and eigenvectors at the same time. Later on we split the calculation, computing eigenvalues alone, and then eigenvectors.

6 Math 20F Linear Algebra Lecture 22 6 Eigenvalues and Eigenvectors Slide Definition of eigenvalues and eigenvectors. Eigenspace. Geometrical interpretation of eigenvectors. Characteristic equation. Eigenvalues and eigenvectors Definition 3 (Eigenvalues and eigenvectors) Let A be an n n matrix. A number λ is an eigenvalue of A if there exists a nonzero vector x IR n such that Slide 2 Ax = λx. The vector x is called an eigenvalue of A corresponding to λ. Notice: If x is an eigenvector, then tx with t 0 is also an eigenvector. Definition 4 (Eigenspace) Let λ be an eigenvalue of A. The set of all vectors x solutions of Ax = λx is called the eigenspace E(λ). That is, E(λ) = { all eigenvectors with eigenvalue λ, and 0}.

7 Math 20F Linear Algebra Lecture 22 7 Examples of eigenspaces The matrix A = 3 has eigenvectors λ = 4 and 3 λ 2 = 2. The corresponding eigenspaces are E(4) = {x = t, T, t IR}. Slide 3 E( 2) = {x = t, T, t IR}. The matrix A = 2 has eigenvectors λ = 0 and λ 2 = The corresponding eigenspaces are E(0) = {x = t 2, T, E( 2) = {x = t, 3 T, t IR}. t IR}. Notice that not every eigenspace is one-dimensional. Eigenspace Theorem 4 Let λ be an eigenvector of A, an n n matrix. Then, the set E(λ) IR n is a subspace. Slide 4 Proof: Let x, x 2 E(λ), that is, Ax = λx, Ax 2 = λx 2. Now compute A(ax + bx 2 ) = aax + bax 2 = aλx + bλx 2 = λ(ax + bx 2 ). Therefore, ax + bx 2 E(λ).

8 Math 20F Linear Algebra Lecture 22 8 Geometrical interpretation of eigenvectors Think the n n matrix A as a linear transformation A : IR n R n. Slide 5 An eigenvector x of A determines a direction in IR n where the action of A is simple: It is a stretching or a compression, depending on whether λ or λ. Theorem 5 The eigenvalue of a diagonal n n matrix are the elements of its diagonal, and its eigenvectors are the standard basis vectors e i, with i =,, n. Theorem 6 Let {v,, v r } be eigenvectors of A with eigenvalues {λ,, λ r }, respectively. If the {λ,, λ r } are all different, then the {v,, v r } are l.i.. Proof: By induction in r. For r = the theorem is true. Slide 6 Consider the case r = 2 as an intermediate step to understand the idea behind the proof. In this case we have two eigenvectors {v, v 2} corresponding to different eigenvalues λ, λ 2, that is λ λ 2. We have to show that {v, v 2} are l.i.. By contradiction, assume that they are l.d., that is, there exists a nonzero a IR such that v 2 = av. () Apply the matrix A on both sides of the equation (), then one gets λ 2v 2 = aλ v, because both vectors are eigenvectors of A. Now multiply equation () by λ 2. One gets, λ 2v 2 = aλ 2v.

9 Math 20F Linear Algebra Lecture 22 9 From these two equations one gets 0 = a(λ 2 λ )v. Slide 7 Because a 0, and the eigenvalues are different, one gets v = 0, which is a contradiction to the hypothesis that v is an eigenvector. Therefore, {v, v 2} are l.i.. This is the idea of the proof, and we now repeat it in the case of r eigenvalues. Assume that the theorem holds for r eigenvectors {v,, v r }, and then show that it also holds for r eigenvectors vectors {v,, v r}. So assume that {v,, v r } are l.i., and that, by contradiction, suppose that the {v,, v r, v r} are l.d.. Then, v r = a v + + a r v r, with some a i 0. Apply the matrix A on both ides of equation above, then λ rv r = a λ v + + a r λ r v r. Now multiply the first equation by λ r, Slide 8 λ rv r = a λ rv + + a r λ rv r. Subtract these two equations, and then one gets 0 = a (λ r λ )v + + a r (λ r λ r )v r. Because all the λ i are different, then the linear combination above says that the {v,, v r } are l.d.. But this contradicts the hypothesis. Therefore, the {v, v r} are l.i., and the theorem follows.

10 Math 20F Linear Algebra Lecture 22 0 The characteristic equation To compute the eigenvalues and eigenvectors one has to solve the equation Ax = λx for both, λ and x. This equation is equivalent to (A λi)x = 0. Slide 9 This homogeneous system has a non-zero solutions if and only if det(a λi) = 0. Notice that this last equation is an equation only for the eigenvalues! There is no x in this equation, so one can solve only for λ and then solve for x. Definition 5 (Characteristic function) Let A be an n n matrix and I the n n identity matrix. Then, the scalar function f(λ) = det(a λi) is called the characteristic function of A. The characteristic equation Theorem 7 (Characteristic polynomial) The characteristic function f(λ) of an n n matrix A is a polynomial in λ of degree n. Furthermore, the polynomial has the form f(λ) = ( ) n λ n + + det(a). Slide 20 Example: f(λ) = a c b d λ 0 0 λ = a λ c b d λ f(λ) = (a λ)(d λ) bc, = λ 2 (a + d)λ + (ad bc).

11 Math 20F Linear Algebra Lecture 22 The characteristic equation Theorem 8 (Characteristic equation) The number λ is an eigenvalue of A if and only if det(a λi) = 0. (2) Slide 2 Equation (2) is called characteristic equation. Proof: Ax = λx (A λi)x = 0 N(A λi) {0} (A λi) is not invertible det(a λi) = 0. Find the eigenvalues of A = any a, b IR with b 0. Examples 3 3 and B = a b b a for Slide 22 Let start with matrix A, λ 3 0 = 3 λ = ( λ)2 9, λ = ± 3, that is, λ = 4, and λ 2 = 2. Now, in the case of matrix B one has a λ b b a λ = (a λ)2 + b 2 0, therefore, B has none eigenvalues at all.

12 Math 20F Linear Algebra Lecture 22 2 Slide 23 Examples Find the eigenvalues of A = 2 3. The answer is: λ 3 0 = 0 2 λ = (2 λ)2, (λ 2) 2 = 0, that is, λ = 2. This eigenvalue has multiplicity 2, according to the following definition. Definition 6 (Multiplicity of eigenvalues) Let f(λ) be the characteristic polynomial of an n n matrix. The eigenvalue λ 0 has algebraic multiplicity r > 0 if and only if f(λ) = (λ λ 0 ) r g(λ), with g(λ 0 ) 0. Slide 24 Examples Find the eigenvalues and eigenspaces of the following two matrices: A = 0 3 2, B = The both matrices have the same eigenvalues, because, so the eigenvalues are: λ = 3 with multiplicity 2; λ = with multiplicity. f A(λ) = f B(λ) = (λ 3) 2 ( λ)

13 Math 20F Linear Algebra Lecture 23 3 Examples Slide 25 One can check that the eigenspaces are the following: 0 E A(3) = 0, E A() =, 0 0 E B(3) = 0,, E B() = Notice: dim E(λ) multipl.(λ). In the case of B, where dim E B(λ) = multipl.(λ) for every eigenvalue of B, the set of all eigenvectors of B is a basis of IR 3. In the case of A, where for λ = 3 holds that dim E A(3) < multipl.(3), the set of eigenvectors of A is not a basis of IR 3. Diagonalization Slide 26 Diagonalization and eigenvectors. Application: Computing powers of a matrix. Examples.

14 Math 20F Linear Algebra Lecture 23 4 Diagonalizable Definition 7 (Diagonalizable matrices) An n n matrix A is diagonalizable if there exists a diagonal matrix D and an invertible matrix P, with inverse P, such that Slide 27 A = P DP. (3) Notice that D is a diagonal matrix if d d 2 0 D = = diagd,, d n d n Diagonalization and eigenvectors Notice that if B = b,, b n, then BD = d b,, d n b n. Also notice that for a general B holds BD DB. Finally recall that AB = Ab,, Ab n. Now, the main result: Slide 28 Theorem 9 (Diagonalization and eigenvectors) An n n matrix A is diagonalizable if and only if A has n l.i. eigenvectors. Furthermore, if we write A = P DP, with D diagonal, then P = p,, p n, and D = diagλ,, λ n, where Ap i = λ i p i, i =,, n, that is, {p,, p n } are the eigenvectors with eigenvectors λ,, λ n, respectively.

15 Math 20F Linear Algebra Lecture 23 5 Proof: ( ) Let P = p,, p n be a matrix formed with the n eigenvectors of A, and denote by λ i the corresponding eigenvalues, that is, Ap i = λ ip i. Because {p,, p n} are l.i. then P is invertible. Introduce the diagonal matrix D = diagλ,, λ n. Then, P D = λ p,, λ np n = Ap,, Ap n = AP. Slide 29 Therefore, A = P DP. ( ) Given the invertible matrix P, introduce its column vectors P = p,, p n. Denote the diagonal matrix D = diagd,, d n. Now, the equation A = P DP implies AP = AD, that is, Ap,, Ap n = d p,, d np n, which says that Ap i = d ip i for every i =,, n. So the p i are eigenvectors of A with eigenvalue d i. And these vectors are l.i. because P is invertible. Example Recall the matrix A given by A = 3 3, Slide 30 which has eigenvectors and eigenvalues given by v =, λ = 4, v 2 =, λ 2 = 2. Then, P =, P = 2, D =

16 Math 20F Linear Algebra Lecture 23 6 Slide 3 Example Then, it is easy to check that P DP = 4 0 ( ) = 2 2, = 3, 3 = A., Applications Theorem 0 (Powers of matrices) Let A be an n n matrix. If A is diagonalizable, then A k = P (D k )P. Slide 32 Proof: A k = (P DP )(P DP ) (P DP ), = P D(P P )D(P P ) (P P )DP, = P (D k )P.

17 Math 20F Linear Algebra Lecture 24 7 Example Compute A 4 where Slide 33 We know that the eigenvectors and eigenvalues of A are given by 2 v =, λ = 0, v 2 =, λ 2 = 7. 3 Then, P = 2 3, P = 7 3 2, D = Slide 34 Example A ( = ) =, =, (7 3 ) 7 3 2(7 3 ) =, 3(7 3 ) 6(7 3 ) = 7 3 2, 3 6, = 7 3 A.

18 Math 20F Linear Algebra Lecture 24 8 Definition of inner product. Inner product Slide 35 Examples. Norm, distance. Orthogonal vectors. Orthogonal complement. Definition of inner product Slide 36 Definition 8 (Inner product) Let V be a vector space over IR. An inner product (, ) is a function V V IR with the following properties. u V, (u, u) 0, and (u, u) = 0 u = 0; 2. u, v V, holds (u, v) = (v, u); 3. u, v, w V, and a, b IR holds (au + bv, w) = a(u, w) + b(v, w). Notation: V together with (, ) is called an inner product space.

19 Math 20F Linear Algebra Lecture 24 9 Examples Slide 37 Let V = IR n, and {e i } n i= be the standard basis. Given two arbitrary vectors x = n i= x ie i and y = n i= y ie i, then (x, y) = n x i y i. This product is also denoted as n i= x iy i = x y. It is called Euclidean inner product. i= Let V = IR 2, and {e i } 2 i= be the standard basis. Given two arbitrary vectors x = 2 i= x ie i and y = 2 i= y ie i, then (x, y) = 2x y + 3x 2 y 2. Examples Slide 38 Let V = C(0, ). Given two arbitrary vectors f(x) and g(x), then (f, g) = 0 f(x)g(x) dx. Let V = C(0, ). Given two arbitrary vectors f(x) and g(x), then (f, g) = 0 e x f(x)g(x) dx.

20 Math 20F Linear Algebra Lecture Norm An inner product space induces a norm, that is, a notion of length of a vector. Slide 39 Definition 9 (Norm) Let V, (, ) be a inner product space. The norm function, or length, is a function V IR denoted as, and defined as u = (u, u). Example: The Euclidean norm is u = u u = (x ) (x n ) 2. Distance A norm in a vector space, in turns, induces a notion of distance between two vectors, defined as the length of their difference.. Slide 40 Definition 0 (Distance) Let V, (, ) be a inner product space, and be its associated norm. The distance between u and v V is given by dist(u, v) = u v. Example: The Euclidean distance between to points x and y IR 3 is x y = (x y ) 2 + (x 2 y 2 ) 2 + (x 3 y 3 ) 2.

21 Math 20F Linear Algebra Lecture 24 2 Angle between vectors Slide 4 Definition (Angle) Let V, (, ) be a inner product space, and be its associated norm. The angle θ between two vectors u, v V is defined as cos(θ) = (u, v) u v. Examples: The angle θ between two vectors x, y IR 2 with respect to the Euclidean inner product is given by x y = x y cos(θ). Angle between vectors The notion of angle can also be introduced in the inner space of continuous functions C(0, ). The angle between f(x) = x and g(x) = x 2 is approximately 4.5, because Slide 42 (f, g) = 0 x 3 dx = 4, then, f = 0 cos(θ) = x 2 dx =, g = x 4 dx = (f, g) 5 f g = 4 θ 4.5.

22 Math 20F Linear Algebra Lecture Orthogonal vectors Definition 2 (Orthogonal vectors) Let V, (, ) be an inner product space. Two vectors u, v V are orthogonal, or perpendicular, if and only if Slide 43 (u, v) = 0. We call them orthogonal, because cos(θ) = 0, which implies θ = π/2. Example: The vectors cos(x), sin(x) C(0, 2π) are orthogonal, because (cos(x), sin(x)) = = 2 2π = 4 = π 0 sin(x) cos(x) dx, sin(2x) dx, ( cos(2x) 2π 0 ),