Lecture 13 Review of Matrix heory I Dr. Radhakant Padhi sst. Professor Dept. of erospace Engineering Indian Institute of Science - Bangalore
Definition and Examples Definition: Matrix is a collection of elements (numbers) arranged in rows and columns. Examples: 1 1 3 13 = [ 1 3 ], B31 =, C3 =, 4 5 6 3 1 1 4 6 D3 = 3 4, E 3 3 4 5 = 5 6 6 5 3
Definitions Symmetric matrix: = Singular matrix: = 0 Inverse of a matrix: Orthogonal matrix: B is inverse of iff B= B= I ( ) 1 = adj / = = I cosθ sinθ Example: ( θ ) = sinθ cosθ Result: Columns of an orthogonal matrix are orthonormal. 3
Eigenvalues and Eigenvectors Matrices also act as linear operators with stretching and rotation operations. y1 x1 y1 Y = X = Q = y x y X P x 1 = x 4
Eigenvalues and Eigenvectors Question: Can we find a direction (vector), along which the matrix will act only as a stretching operator? nswer: If such a solution exists, then X = λx λi X = ( ) 0 For nontrivial solution, λi = 0 Utility: Stability and control, Model reduction, Principal component analysis etc. 5
erminology Definition Properties of Eigenvalues Positive definite > 0 X X > 0 X 0 λ > i 0, i Positive semi definite 0 X X 0 X 0 λ i 0, i Negative definite < 0 X X < 0 X 0 λ < i 0, i Negative semi definite 0 X X 0 X 0 λ i 0, i 6
Eigenvalues and Eigenvectors: Some useful properties If λ1, λ,, λ are eigenvalues of n n n then for any positive integer m, λ m 1, λ m,, λ m n m are eigenvalues of If is a nonsingular matrix with eigenvalues 1 1 1 λ, then 1,... 1, λ,, λn λ λ λn are 1 eigenvalues of For triangular matrix, the eigenvalues are the diagonal elements 7
Eigenvalues and Eigenvectors: Some useful properties If a n n matrix is symmetric, its eigenvalues are all REL. Moreover, it has n linearly-independent eigenvectors. If n n has n real eigenvalues and n real orthogonal eigenvectors, then the matrix is symmetric and are always positive semi definite. If is a positive definite symmetric matrix, then every principal sub-matrix of is also symmetric and positive definite. In particular, the diagonal elements of are positive. 8
Generalized Eigenvectors If an eigenvalue is repeated p times, there may or may not be p linearly independent eigenvectors corresponding to it. In case linearly independent eigenvalues cannot be found, generalized eigenvectors are the next option. Example: Suppose 3 3has eigenvalues λ1, λ, λ. hen eigenvectors V1, V and generalized eigenvector V can be found as follows: ( λ ) ( λ ) ( λ ) (a) I V = 0 3 1 1 (b) I V = 0 (c) I V = V 3 3 9
Vector Norm Vector norm is a real valued function with the following properties: (a) X > 0 and X = 0 only if X = 0 (b) α X = α X (c) X + Y X + Y X, Y 10
Vector Norm X = x + x + + x ( l norm) 1 3 1 n 1 ( ) n X = x + x + + x ( l norm) ( 3 3 3 ) n ( p p p ) 1 n 1/ 1 X = x + x + + x ( l norm) 1/3 1 3 ( ) 1 n 1/ p X = x + x + + x ( l norm) p 1/ X = x + x + + x = max x ( norm) i i p l 11
Matrix/Operator/Induced Norm Definition: Properties: X = max = max X 0 X X = 1 (a) > 0 and = 0 only if = 0 (b) α = α (c) + B + B (d) B B ( X ) 1
Matrix/Operator/Induced Norm 1-Norm 1 1 j n -Norm -Norm n = max aij : Largest of the absolute column sums 1 i n i = 1 ( ) = σ max : Largest Singular Value n = max aij : Largest of the absolute row sums j= 1 13
Matrix/Operator/Induced Norm Frobenius Norm Holds good for non-square matrices as well Used frequently in neural network and adaptive control literature m n F 1/ m n = aij i= 1 j= 1 14
Spectral Radius For with eigenvalues the spectral radius ρ is defined as Result: n n ( ) ρ = ( ) 1 ( ) max 1/ i n = ρ If is symmetric, then λ i λ1, λ,, λn 1/ 1/ 1/ ( ) ( ) ( ( )) ( ) = ρ = ρ = ρ = ρ 15
Least Square Solutions System: where X = b R, X R, b R m n n m Case 1: ( m= n and 0) (No. of equations = No. of variables) Unique solution: X = 1 b 16
Least square solutions ( m < n) Case : (under constrained problem) (No. of equations < No. of variables) In this case, there are infinitely many solutions. One way to get a meaningful solution is to formulate the following optimization problem: Minimize Solution J = X, Subject to X = b ( ) 1 ( ) + + X = b, where = right pseudo inverse his solution WILL satisfy the equation X = b exactly. 17
Least square solutions Case : ( m> n) (over constrained problem) (No. of equations > No. of variables) In this case, there is no solution. However, one way to get a meaningful (error minimizing) solution is to formulate the following optimization problem: Minimize: J = X b Solution: ( ) 1 ( ) + + X = b, where = left pseudo inverse his solution need not satisfy the equation X = b exactly. 18
Generalized/Pseudo Inverse Left pseudo inverse: Right pseudo inverse: Properties: ( ) ( ) + ( a) = ( b) = () c = + + ( d) = + + + + + + + ( ) 1 = = ( ) 1 + 1 ( e), if is square and 0 = 19
0
1