# BASIC THEORY AND APPLICATIONS OF THE JORDAN CANONICAL FORM

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 BASIC THEORY AND APPLICATIONS OF THE JORDAN CANONICAL FORM JORDAN BELL Abstract. This paper gives a basic introduction to the Jordan canonical form and its applications. It looks at the Jordan canonical form from linear algebra, as an almost diagonal matrix, and compares it to other matrix factorizations. This article also looks at it from the viewpoint of abstract algebra, as a way to partition the set of square matrices with complex entries. It presents a proof due to A.F. Filippov that any square matrix is similar to a matrix in Jordan canonical form. Some background on the Jordan canonical form is given. 1. Introduction A good use for diagonalizing a matrix is to view the linear transformation as the action of a matrix of only diagonal entries. It s often easier to understand what a diagonal matrix representation is doing, and for many computations a diagonal matrix is easier to work with than a dense matrix with many values off the diagonal. A good example of where a diagonal matrix is useful is for finding large powers of a matrix, which a diagonal matrix makes amazingly easy; e.g. finding, say, the 50-th power of a matrix made only of 2 s on the diagonal. For a fairly easy to understand introduction to matrices and diagonalization, see D.C. Lay in [8]. However, even working with the complex numbers C as our field, not all matrices can be diagonalized. For example, [ ] 0 3, 0 0 is not diagaonalizable over C. The only eigenvalue of this matrix is 0 which has multiplicity 2 as a solution to the characteristic equation (that is, the algebraic multiplicity is 2), but this eigenvalue only yields 1 eigenvector (so the geometric Date: August 31, Mathematics Subject Classification. 15A21. Key words and phrases. Jordan canonical form, matrix factorization, partitions. This paper was written during an NSERC USRA supervised by Dr. B. Stevens. The idea for this paper is from Dr. J. Poland. 1

2 2 JORDAN BELL multiplicity is 1). Since the geometric multiplicity is less than the algebraic multiplicity, this matrix is not diagonalizable. The Jordan canonical form can let us almost diagonalize this matrix (we won t form a matrix with only diagonal entries, but, as we will see, it will be fairly close). 2. Jordan blocks, Jordan basis and Jordan canonical form A square matrix of the form λ λ λ is called a Jordan block. Note that an n n matrix is a Jordan block if it is upper triangular, has all diagonal entries equal, and has a 1 directly above each diagonal entry except the first. A square matrix J, of the form J J J s where J 1,..., J s are all Jordan blocks, is said to be in Jordan canonical form. The canonical refers to its (possibly arbitrary) choice as a representative; it is representative of all matrices that are similar to it (we discuss this more in the abstract algebra section). Definition 1 (Matrix direct sum). The direct sum of two matrices A k l and B m n is an (k + m) (l + n) matrix C with A in the top left corner and B in the bottom right corner; so A is in c 1,1 to c k,l and B is in c k+1,l+1 to c k+m,l+n. We can express a matrix in Jordan canonical form as the direct sum of Jordan blocks. So if we have s Jordan blocks then our Jordan canonical form is given by J = J 1... J s and more compactly as J = s i=1j i. Definition 2 (Jordan basis). Let A be an n n matrix. An ordered set B = {b 1,..., b n }, that is a basis for C n is a Jordan basis for A if it has the Jordan basis property, that for each 1 i n, we have either Ab i = λ i b i or Ab i = λ i b i + b i 1, where λ i is also the eigenvalue for b i 1.,

3 JORDAN CANONICAL FORM 3 Our proof of Theorem 3 follows the explanations of G. Strang in [10] and Fraleigh and Beauregard in [5]. The original proof by A.F. Fillipov is available in Russian in [3]. We remember that v is an eigenvector of A if and only if Av = λv. We can also have almost eigenvectors (we call these generalized eigenvectors), that after multiple mappings by A yield (along with other generalized eigenvectors) eigenvectors of A. Let us have Ax 1 = 8x 1 and Ax 2 = 8x 2 + x 1, and Ax 3 = 0x 3 and Ax 4 = 0x 4 + x 3. Then x 2 and x 4 are the generalized eigenvectors. We note that the set of all eigenvalues of A is known as the spectrum of A, that is Sp(A). Theorem 3 (Filippov s construction). Any n n matrix A is similar to a matrix in Jordan canonical form. Proof. In the following we show that we can construct a Jordan basis with n entries for any n n matrix, and that the entries of this basis can be used as columns for a matrix P such that A = P 1 JP, from which the Jordan canonical form J easily follows. We also note that we only need to show that matrices with 0 in their spectrum can have a Jordan basis constructed for them, because of the following, where we assume that any matrix with 0 in its spectrum is similar to a matrix in Jordan canonical form: λ Sp(A) 0 Sp(A λi) A λi = P 1 JP P (A λi)p 1 = J P 1 AP λi = J P AP 1 = J + λi A = P 1 (J + λi)p. where J + λi is clearly in Jordan canonical form. We procede by induction on the dimension r of the range of a linear transformation T : C n C n, constructing a Jordan basis with r members for the matrix representation of T. For n = 1, A = [λ] and clearly A is already in Jordan canonical form, with a Jordan basis of, say, {1}. We make as our induction hypothesis that for all linear transformations with a range of dimension r < n, there exists a Jordan basis for their matrix representation. Let us have an n n matrix A as the matrix representation of a linear transformation with range of dimension r, so the rank of A is r. Thus by assumption there exists a Jordan basis for ColA, which will be F = {f 1,..., f r }. We introduce some notation now for use with eigenvectors and their generalized eigenvectors. We know that vectors in a Jordan basis are either eigenvectors or generalized eigenvectors, and for the i-th progression of generalized eigenvectors and one eigenvector, which we will

4 4 JORDAN BELL think of as going from the generalized eigenvectors at the right and ending at the eigenvector in the left, let us represent these vectors as v i1, v i2,..., v il(i), where l(i) denotes the length of the i-th progression. There are s 0 progressions of generalized eigenvectors and eigenvectors in F that end at an eigenvector with eigenvalue 0, and clearly for the i-th progression the first vector f i1 in it is in NulA. As well, the last vector in the i-th progression f il(i) is clearly in ColA, so it is of the form Ag i = f il(i) ; let us have G = {g 1,..., g s }. We recall that there are s progressions with an eigenvector with eigenvalue 0 at the left end, and thus by elementary linear algebra we can extend these s vectors with n r s vectors to form a basis for NulA. We express these as H = {h 1,..., h n r s }. Let us have B = F G H. We recall that there are s progressions in F associated with eigenvalue 0, each with a distinct vector in G at their right end; for convenience, we form a new set E that contains exactly the vectors in F associated with eigenvalue 0, and we remove these vectors from F. Thus now the vectors in F are precisely those vectors that form progressions of generalized eigenvectors leading to eigenvectors associated with nonzero eigenvalues. We will say there are q such progressions, each made by some number of generalized eigenvectors and precisely one eigenvector per progression. Finally, each vector in H constitutes its own progression, as each is an eigenvector with eigenvalue 0. Thus we compose these progressions in B such that B = {b 11,..., b il(i) }, and we can clearly compose the progressions such that B has the Jordan basis property. Now by Definition 2 we know that B is a Jordan basis for A if it is a basis for C n. It is clear that the following equation has only the trivial solution if and only if the vectors in it are linearly independent: (1) ɛ 11 e ɛ sl(s) e sl(s) + ζ 11 f ζ ql(q) f ql(q) + η 1 g η s g s + θ 1 h θ n r s h n r s = 0. We recall that each h i is in NulA, and thus by premultiplying (1) by A we have: (2) ɛ 12 e ɛ sl(s) e s,l(s) 1 + (λ 1 ζ 11 + ζ 12 )f λ q ζ ql(q) f ql(q) + η 1 f 1l(1) η s f sl(s) = 0. which is a linear combination of r vectors from E F, which we know is a basis for C r. Thus only zero weights will let (2) hold, so we can easily conclude the following: ɛ 12 =... = ɛ s2 = ζ 1l(1) =... = ζ ql(q) = η 1 =... = η s = 0.

5 JORDAN CANONICAL FORM 5 By substituting in these weights, we can make (1) into the following: (3) ɛ 11 e ɛ s1 e s1 + ζ 11 f ζ 1,l(1) 1 f 1,l(1) 1 + ζ 21 f ζ q,l(q) 1 f q,l(q) 1 + θ 1 h θ s h s = 0. Clearly the entries in H are independent from the entries in F, and by assumption the entries in H are independent from the entries in E, as we extended E with H to be a basis for NulA. Thus for (3) to hold, we must have the following: ɛ 11 =... = ɛ s1 = ζ 11 =... = ζ 1,l(1) 1 = ζ 21 =... = ζ q,l(q) 1 = θ 1 =... = θ s = 0. Hence all weights in (1) are equal to 0, so the vectors in (1) are linearly independent, so B is a basis for C n. Thus B is a Jordan basis for A. There are other proofs that any square matrix with complex entries is similar to a matrix in Jordan canonical form, aside from Filippov s proof. For example, Fletcher and Sorensen in [4] have a different proof that extends the Schur factorization (by using their results that every matrix in Schur form is similar to a block upper triangular of a certain form, and that each triangular block is similar to a matrix of a particular form) to achieve Jordan canonical form. The Jordan canonical form was defined by the French mathematician Camille Jordan in 1870, in [7]. However, [2] notes that a similar form was used earlier by the German mathematician Karl Weierstraß. 3. Other matrix factorizations When an n n matrix A is similar to a matrix J in Jordan canonical form, we can express this as A = P 1 JP, where P has as columns n linearly independent vectors of a Jordan basis for A. This is a matrix factorization of A, as we are expressing A as a product of three matrices. There are several other useful factorizations of matrices. These factorizations are usually chosen for similar reasons why we factor a number into its prime factors, that factoring can make them easier to analyze. We compare the Jordan factorization of an n n matrix, to the LU factorization, QR factorization and Schur factorization. We explain these factorizations and give an elementary look at their computational uses, but our discussion is mostly about the usefulness and interest of working with these factorizations in pure math LU factorization. The LU factorization of an m n matrix A is given by A = LU, with L m m and U m n. L is permuted unit lower triangular (unit because all entries on the diagonal are 1 s), and

6 6 JORDAN BELL U is in reduced echelon form. It is called LU factorization because L is permuted lower triangular, and the U matrix is a rectangular upper diagonal matrix. The LU factorization is used to solve linear equations of the form Ax = b for x. If there is an LU solution, there is a solution just by row reducing [ A b ]. However, finding a solution with LU factorization often takes less operations, so it is computationally better. After we ve found L and U, we first solve Ly = b and then solve Ux = y. Each of these linear equations are fairly easy to solve, because both L and U are (possibly permuted) triangular. For more discussion about the uses and efficiency of LU factorization, see section 3.2 of [6] by Golub and Van Loan; [8] also suggests looking at section 3.8 of [9] by Noble and Daniel. We now give an algorithm to find L m m and U m n for A m n. By elementary linear algebra, A can be reduced to reduced echelon form U by left multiplication by elementary matrices. It will take some finite number of operations p to reduce A to U, hence we have E p... E 1 A = U A = (E p... E 1 ) 1 U A = LU for L = (E p... E 1 ) 1 L = E Ep 1. This gives us an L that is not in general lower triangular. However, it is permuted lower triangular, so some rearrangement of its rows will give us a lower triangular matrix. Thus when we are solving Lx = y by reducing [ L y ] to [ I x ], instead of following the normal algorithm for reducing a matrix to reduced echelon form of adding multiples of a row to the rows below it, and then adding multiples of the rows above it, we add multiples of the row with the pivot of the first column to the other rows, and then add multiples of the row with the pivot of the second column to the other rows, etc. For example, if the pivot of column 1 was in row 3, and the pivot of column 2 was in row 4, and the pivot of column 3 was in row 10, etc., we would add multiples of row 3 to the other rows to eliminate all other entries of column 1, then add multiples of row 4 to the other rows to get rid of all the other entries in column 2, then add multiples of row 10 to the other rows to get rid of all the other entries in column 3, etc. At the end of these row operations, we will be done; that is, we will not have to use pivots to eliminate entries off the diagonal. Finding the Jordan canonical form of a matrix requires quite a lot of operations. There are several methods of finding the Jordan canonical form of a matrix, but they all require finding things like null space, column space or eigenspaces. The LU form, on the other hand, only

7 JORDAN CANONICAL FORM 7 requires getting a matrix in echelon form, and then perhaps rearranging the rows of a permuted lower triagular L. So the LU factorization is computationally better, in the sense of being easier to find, than the Jordan factorization. The Jordan canonical form is used for mostly different reasons than the LU factorization, however, the LU factorization does not have any clear uses for similarity. We only give the LU factorization to compare the matrix factorization it uses to the Jordan canonical form s matrix factorization QR factorization. We first give the definition of orthonormal that we use in QR factorization. Definition 4 (Orthonormal). Let us have n vectors v 1,..., v n. We say that these vectors are orthonormal if the inner product of any choice of two of them is equal to 0, so v i, v j =0 for i j, and if the vector length of each is equal to 1, so v i = 1. If we have an injective m n matrix A (that is, A has n linearly independent column vectors), we can use QR factorization to give us A = QR, with Q m n with columns an orthonormal basis for ColA and R n n an upper triangular invertible matrix with positive entries on the main diagonal. If we have a basis B = {b 1,..., b n } for a vector space W, the Gram-Schmidt process allows us to form an orthonormal basis C = {c 1,..., c n }. For our matrix A, we can have A = [ b 1... b n ]. As well, the Gram-Schmit process has, for a i R: (4) db p = a 1 c a p c p + 0c p b n. Hence we can form an orthonormal basis C = {c 1,..., c n } from a basis for ColA. Let the ordered vectors in C be the ordered columns of Q, so Q = [ ] c 1... c n. Since we have Equation 4, we can have Qr i = b i, with r i a vector of weights for the column vectors of Q; these weights will all be 0 for the columns after i. So we can have: ρ 1. ρ r i = i By the definition of the Gram-Schmidt process, b i = Qr i. Thus A = [ Qr 1... Qr n ], as bi s are the columns of A. Hence for R =

8 8 JORDAN BELL [ r1... r n ], we have A = QR. We know that Q has as column vectors an orthonormal basis for ColA, and we know that R is upper triangular because of the stairsteps implied by Equation 4. To have R with only positive entries on the main diagonal, if for r i we have ρ i < 0, we multiply c i by -1. This is the same as multiplying a p by -1 and c p by -1 in Equation 4 to keep b p the same; multiplying c p by -1 does not change the vector space W it spans, and C is still an orthonormal basis for W. The QR factorization is very useful in least squares calculations, for finding the best approximation inside a vector space to something outside the vector space. For example, if a matrix A has linearly independent columns, we can find a unique least squares solution for Ax = b, with ˆx = R 1 Q T b. The QR method does not have immediate applications to similarity Schur decomposition. We should explain several notations before considering Schur decomposition. For A M n n (C), A H is the Hermitian conjugate of A, which takes the complex conjugate of every entry of A and then tranposes A. A matrix A is unitary if and only if A H = A 1 ; thus, for example, an orthogonal matrix with real entries is unitary. A nilpotent matrix A has A k = 0 for some k 1. Theorem 5 (Schur s decomposition). If A is in M n n (C) then there exists a unitary matrix Q such that Q H AQ = T = D + N, where D = λ 1... λ s, with λ i a Jordan block and N a nilpotent strictly upper triangular matrix. Q is unitary, Q H = Q 1, so for Q H AQ = T, A T. Thus both the Jordan canonical form and the Schur decomposition express similarity. The difference is that the Jordan form is more likely to give a nicer J, as it has only diagonal eigenvalue entries and sometimes 1 s above those so it is sparse, while T is only upper diagonal, so it could be denser. However, the Schur decomposition has a very nice similarity transformation matrix Q, because Q is orthogonal. 4. Applications in group theory We show that being similar to a Jordan canonical form matrix is a relation that partitions the set of square matrices with complex entries M n n (C). Of course because only one equivalence class has the identity matrix, Jordan canonical form does not partitition the ring of matrices into subrings. We study the question of forming an algebraic structure from the equivalence classes made by partitioning M n (C).

9 JORDAN CANONICAL FORM 9 We give a lemma on partitioning a set into equivalence classes. proof is in, eg, [1]. A Lemma 6. Let be an equivalence relation on a set X. Then the collection of equivalence classes C (x) forms a partition of X. So by Lemma 6, if we can show that is an equivalence relation on M n n (C), it will show that partitions that set into subsets all similar to a particular Jordan canonical form matrix (so the Jordan form of the subset of each subset represents the members of that subset). Recall from Cantor s diagonal proof that the real numbers are uncountable, so the complex numbers, which contain the real numbers, are also uncountable. But note that for an n n matrix A = n i=1λ i, with λ i C, we can have any eigenvalues for A in the complex numbers. Since we have partitioned M n n (C) based on eigenvalues, this gives us uncountably many similarity equivalence classes. Note that for similarity A B expresses A = P 1 BP, for some P. But we can have P = I, and then P 1 = I. So we find that is reflexive, because we have an invertible P such that A = P 1 AP, as A = A. If A B, then A = P 1 BP. But then B = P AP 1, and we can just have Q = P 1, which gives us B = Q 1 AQ, so B A, so is symmetric. If A B and B C, then A = P 1 BP and B = Q 1 CQ. Note that B = P AP 1, so P AP 1 = Q 1 CQ. Thus A = P 1 Q 1 CQP. Let R = QP. As P and Q are invertible, R, their product, is also invertible. So R 1 = P 1 Q 1. Thus A = R 1 CR, so is transitive. Thus is an equivalence relation. Hence, by the partitioning lemma, partitions M n n (C) into subsets of matrices similar to some matrix in Jordan canonical form (and all similar to each other). We can show that the product of two arbitrary matrices in a certain equivalence class is not in general in that equivalence class; similarly, the equivalence classes are not themselves closed under addition. As well, multiplication of elements in these classes is not well defined, as in general the product of two different choices of matrices as representatives of their equivalence class are not in the same equivalence class. For example, let us have the following: A = [ ], B = [ ], C = [ ] Clearly both A and B are similar to each other, and B is the Jordan canonical form representative of their equivalence class. We find that AB = C, and that C is not similar to B 2. Thus we cannot make any

10 10 JORDAN BELL groupoid with matrix multiplication as its operation out of similarity classes, because multiplication of similarity classes is not well defined. 5. Conclusion The Jordan canonical form has many uses in linear and abstract algebra. When it can happen, diagaonalization is quite useful, and it is good to know that we can always get something very close to diagonal form with Jordan canonical form. Of the factorizations we considered, the Jordan canonical form seems most comparable to the Schur decomposition; it seems that for an original matrix A equal to P 1 T P, the Jordan canonical form gives the best T and the Schur decomposition gives the best P and P 1. We also found that we can partition the set of square matrices M n n (C) into equivalence classes of different Jordan canonical forms, but that there does not seem to be any natural algebraic structure that can be formed by similarity classes. References 1. Norman J. Bloch, Abstract algebra with applications, Prentice-Hall, Englewood Cliffs, New Jersey, A. J. Coleman, The greatest mathematical paper of all time, Math. Intelligencer 11 (1989), no. 3, MR 90f: A. F. Filippov, A short proof of the theorem on reduction of a matrix to Jordan form., Vestnik Moskov. Univ. Ser. I Mat. Meh. 26 (1971), no. 2, MR 43 # R. Fletcher and D. C. Sorensen, An algorithmic derivation of the Jordan canonical form, Amer. Math. Monthly 90 (1983), no. 1, MR 84d: John B. Fraleigh and Raymond A. Beauregard, Linear algebra, third ed., Addison-Wesley, Reading, Mass., Gene H. Golub and Charles F. Van Loan, Matrix computations, third ed., Johns Hopkins Studies in the Mathematical Sciences, Johns Hopkins University Press, Baltimore, MD, MR 97g: Camille Jordan, Traité des substitutions et des équations algébriques, Les Grands Classiques Gauthier-Villars. [Gauthier-Villars Great Classics], Éditions Jacques Gabay, Sceaux, 1989, Reprint of the 1870 original. MR 94c: David C. Lay, Linear algebra and its applications, third ed., Addison-Wesley, Reading, Mass., Ben Noble and James W. Daniel, Applied linear algebra, third ed., Prentice- Hall, Englewood Cliffs, NJ, Gilbert Strang, Linear algebra and its applications, second ed., Academic Press [Harcourt Brace Jovanovich Publishers], New York, MR 81f: address: School of Mathematics and Statistics, Carleton University, 1125 Colonel By Drive, Ottawa, K1S 5B6, Canada.

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### 1 Eigenvalues and Eigenvectors

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Lecture 1: Schur s Unitary Triangularization Theorem

Lecture 1: Schur s Unitary Triangularization Theorem This lecture introduces the notion of unitary equivalence and presents Schur s theorem and some of its consequences It roughly corresponds to Sections

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Section 2.1. Section 2.2. Exercise 6: We have to compute the product AB in two ways, where , B =. 2 1 3 5 A =

Section 2.1 Exercise 6: We have to compute the product AB in two ways, where 4 2 A = 3 0 1 3, B =. 2 1 3 5 Solution 1. Let b 1 = (1, 2) and b 2 = (3, 1) be the columns of B. Then Ab 1 = (0, 3, 13) and

### University of Lille I PC first year list of exercises n 7. Review

University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients

### Diagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions

Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### MAT 242 Test 2 SOLUTIONS, FORM T

MAT 242 Test 2 SOLUTIONS, FORM T 5 3 5 3 3 3 3. Let v =, v 5 2 =, v 3 =, and v 5 4 =. 3 3 7 3 a. [ points] The set { v, v 2, v 3, v 4 } is linearly dependent. Find a nontrivial linear combination of these

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### Linear Least Squares

Linear Least Squares Suppose we are given a set of data points {(x i,f i )}, i = 1,...,n. These could be measurements from an experiment or obtained simply by evaluating a function at some points. One

The Hadamard Product Elizabeth Million April 12, 2007 1 Introduction and Basic Results As inexperienced mathematicians we may have once thought that the natural definition for matrix multiplication would

### Matrix Inverse and Determinants

DM554 Linear and Integer Programming Lecture 5 and Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1 2 3 4 and Cramer s rule 2 Outline 1 2 3 4 and

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### Applied Linear Algebra I Review page 1

Applied Linear Algebra Review 1 I. Determinants A. Definition of a determinant 1. Using sum a. Permutations i. Sign of a permutation ii. Cycle 2. Uniqueness of the determinant function in terms of properties

### [1] Diagonal factorization

8.03 LA.6: Diagonalization and Orthogonal Matrices [ Diagonal factorization [2 Solving systems of first order differential equations [3 Symmetric and Orthonormal Matrices [ Diagonal factorization Recall:

### MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 6. Eigenvalues and Singular Values In this section, we collect together the basic facts about eigenvalues and eigenvectors. From a geometrical viewpoint,

### MATH 551 - APPLIED MATRIX THEORY

MATH 55 - APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### Additional Topics in Linear Algebra Supplementary Material for Math 540. Joseph H. Silverman

Additional Topics in Linear Algebra Supplementary Material for Math 540 Joseph H Silverman E-mail address: jhs@mathbrownedu Mathematics Department, Box 1917 Brown University, Providence, RI 02912 USA Contents

### Similar matrices and Jordan form

Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

### (January 14, 2009) End k (V ) End k (V/W )

(January 14, 29) [16.1] Let p be the smallest prime dividing the order of a finite group G. Show that a subgroup H of G of index p is necessarily normal. Let G act on cosets gh of H by left multiplication.

### Lecture Note on Linear Algebra 15. Dimension and Rank

Lecture Note on Linear Algebra 15. Dimension and Rank Wei-Shi Zheng, wszheng@ieee.org, 211 November 1, 211 1 What Do You Learn from This Note We still observe the unit vectors we have introduced in Chapter

### Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Pearson Education, Inc.

2 Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Theorem 8: Let A be a square matrix. Then the following statements are equivalent. That is, for a given A, the statements are either all true

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### 4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION

4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION STEVEN HEILMAN Contents 1. Review 1 2. Diagonal Matrices 1 3. Eigenvectors and Eigenvalues 2 4. Characteristic Polynomial 4 5. Diagonalizability 6 6. Appendix:

### Practice Math 110 Final. Instructions: Work all of problems 1 through 5, and work any 5 of problems 10 through 16.

Practice Math 110 Final Instructions: Work all of problems 1 through 5, and work any 5 of problems 10 through 16. 1. Let A = 3 1 1 3 3 2. 6 6 5 a. Use Gauss elimination to reduce A to an upper triangular

### Notes on Orthogonal and Symmetric Matrices MENU, Winter 2013

Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,

### Sergei Silvestrov, Christopher Engström, Karl Lundengård, Johan Richter, Jonas Österberg. November 13, 2014

Sergei Silvestrov,, Karl Lundengård, Johan Richter, Jonas Österberg November 13, 2014 Analysis Todays lecture: Course overview. Repetition of matrices elementary operations. Repetition of solvability of

### MATH36001 Background Material 2015

MATH3600 Background Material 205 Matrix Algebra Matrices and Vectors An ordered array of mn elements a ij (i =,, m; j =,, n) written in the form a a 2 a n A = a 2 a 22 a 2n a m a m2 a mn is said to be

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS

NUMERICALLY EFFICIENT METHODS FOR SOLVING LEAST SQUARES PROBLEMS DO Q LEE Abstract. Computing the solution to Least Squares Problems is of great importance in a wide range of fields ranging from numerical

### Numerical Methods I Eigenvalue Problems

Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course G63.2010.001 / G22.2420-001, Fall 2010 September 30th, 2010 A. Donev (Courant Institute)

### Examination paper for TMA4115 Matematikk 3

Department of Mathematical Sciences Examination paper for TMA45 Matematikk 3 Academic contact during examination: Antoine Julien a, Alexander Schmeding b, Gereon Quick c Phone: a 73 59 77 82, b 40 53 99

### Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).

MAT 2 (Badger, Spring 202) LU Factorization Selected Notes September 2, 202 Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix

### Recall the basic property of the transpose (for any A): v A t Aw = v w, v, w R n.

ORTHOGONAL MATRICES Informally, an orthogonal n n matrix is the n-dimensional analogue of the rotation matrices R θ in R 2. When does a linear transformation of R 3 (or R n ) deserve to be called a rotation?

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each)

Math 33 AH : Solution to the Final Exam Honors Linear Algebra and Applications 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each) (1) If A is an invertible

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL

SOLUTIONS OF THEORETICAL EXERCISES selected from INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL Eighth Edition, Prentice Hall, 2005. Dr. Grigore CĂLUGĂREANU Department of Mathematics

### University of Ottawa

University of Ottawa Department of Mathematics and Statistics MAT 1302A: Mathematical Methods II Instructor: Alistair Savage Final Exam April 2013 Surname First Name Student # Seat # Instructions: (a)

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### The Moore-Penrose Inverse and Least Squares

University of Puget Sound MATH 420: Advanced Topics in Linear Algebra The Moore-Penrose Inverse and Least Squares April 16, 2014 Creative Commons License c 2014 Permission is granted to others to copy,

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### 2.1: MATRIX OPERATIONS

.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences More than you wanted to know about quadratic forms KC Border Contents 1 Quadratic forms 1 1.1 Quadratic forms on the unit

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### 13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

### Mathematics Notes for Class 12 chapter 3. Matrices

1 P a g e Mathematics Notes for Class 12 chapter 3. Matrices A matrix is a rectangular arrangement of numbers (real or complex) which may be represented as matrix is enclosed by [ ] or ( ) or Compact form

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### 1 Orthogonal projections and the approximation

Math 1512 Fall 2010 Notes on least squares approximation Given n data points (x 1, y 1 ),..., (x n, y n ), we would like to find the line L, with an equation of the form y = mx + b, which is the best fit

### Using the Singular Value Decomposition

Using the Singular Value Decomposition Emmett J. Ientilucci Chester F. Carlson Center for Imaging Science Rochester Institute of Technology emmett@cis.rit.edu May 9, 003 Abstract This report introduces

### Introduction to Matrix Algebra I

Appendix A Introduction to Matrix Algebra I Today we will begin the course with a discussion of matrix algebra. Why are we studying this? We will use matrix algebra to derive the linear regression model

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

### Matrix Differentiation

1 Introduction Matrix Differentiation ( and some other stuff ) Randal J. Barnes Department of Civil Engineering, University of Minnesota Minneapolis, Minnesota, USA Throughout this presentation I have

### Topic 1: Matrices and Systems of Linear Equations.

Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method

### Inverses and powers: Rules of Matrix Arithmetic

Contents 1 Inverses and powers: Rules of Matrix Arithmetic 1.1 What about division of matrices? 1.2 Properties of the Inverse of a Matrix 1.2.1 Theorem (Uniqueness of Inverse) 1.2.2 Inverse Test 1.2.3

### MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix.

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. Matrices Definition. An m-by-n matrix is a rectangular array of numbers that has m rows and n columns: a 11

### Inner product. Definition of inner product

Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product

### Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain

Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

Facts About Eigenvalues By Dr David Butler Definitions Suppose A is an n n matrix An eigenvalue of A is a number λ such that Av = λv for some nonzero vector v An eigenvector of A is a nonzero vector v

### Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible:

Cramer s Rule and the Adjugate Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible: Theorem [Cramer s Rule] If A is an invertible

### 2.1: Determinants by Cofactor Expansion. Math 214 Chapter 2 Notes and Homework. Evaluate a Determinant by Expanding by Cofactors

2.1: Determinants by Cofactor Expansion Math 214 Chapter 2 Notes and Homework Determinants The minor M ij of the entry a ij is the determinant of the submatrix obtained from deleting the i th row and the

### (57) (58) (59) (60) (61)

Module 4 : Solving Linear Algebraic Equations Section 5 : Iterative Solution Techniques 5 Iterative Solution Techniques By this approach, we start with some initial guess solution, say for solution and

### MAT 200, Midterm Exam Solution. a. (5 points) Compute the determinant of the matrix A =

MAT 200, Midterm Exam Solution. (0 points total) a. (5 points) Compute the determinant of the matrix 2 2 0 A = 0 3 0 3 0 Answer: det A = 3. The most efficient way is to develop the determinant along the

### Matrices, Determinants and Linear Systems

September 21, 2014 Matrices A matrix A m n is an array of numbers in rows and columns a 11 a 12 a 1n r 1 a 21 a 22 a 2n r 2....... a m1 a m2 a mn r m c 1 c 2 c n We say that the dimension of A is m n (we

### 5. Orthogonal matrices

L Vandenberghe EE133A (Spring 2016) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### MA 242 LINEAR ALGEBRA C1, Solutions to Second Midterm Exam

MA 4 LINEAR ALGEBRA C, Solutions to Second Midterm Exam Prof. Nikola Popovic, November 9, 6, 9:3am - :5am Problem (5 points). Let the matrix A be given by 5 6 5 4 5 (a) Find the inverse A of A, if it exists.

### 6. Cholesky factorization

6. Cholesky factorization EE103 (Fall 2011-12) triangular matrices forward and backward substitution the Cholesky factorization solving Ax = b with A positive definite inverse of a positive definite matrix

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Matrix Algebra and Applications

Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

### WHICH LINEAR-FRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE?

WHICH LINEAR-FRACTIONAL TRANSFORMATIONS INDUCE ROTATIONS OF THE SPHERE? JOEL H. SHAPIRO Abstract. These notes supplement the discussion of linear fractional mappings presented in a beginning graduate course

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### Sec 4.1 Vector Spaces and Subspaces

Sec 4. Vector Spaces and Subspaces Motivation Let S be the set of all solutions to the differential equation y + y =. Let T be the set of all 2 3 matrices with real entries. These two sets share many common

### Continuity of the Perron Root

Linear and Multilinear Algebra http://dx.doi.org/10.1080/03081087.2014.934233 ArXiv: 1407.7564 (http://arxiv.org/abs/1407.7564) Continuity of the Perron Root Carl D. Meyer Department of Mathematics, North

### Solution based on matrix technique Rewrite. ) = 8x 2 1 4x 1x 2 + 5x x1 2x 2 2x 1 + 5x 2

8.2 Quadratic Forms Example 1 Consider the function q(x 1, x 2 ) = 8x 2 1 4x 1x 2 + 5x 2 2 Determine whether q(0, 0) is the global minimum. Solution based on matrix technique Rewrite q( x1 x 2 = x1 ) =

### Name: Section Registered In:

Name: Section Registered In: Math 125 Exam 3 Version 1 April 24, 2006 60 total points possible 1. (5pts) Use Cramer s Rule to solve 3x + 4y = 30 x 2y = 8. Be sure to show enough detail that shows you are

### October 3rd, 2012. Linear Algebra & Properties of the Covariance Matrix

Linear Algebra & Properties of the Covariance Matrix October 3rd, 2012 Estimation of r and C Let rn 1, rn, t..., rn T be the historical return rates on the n th asset. rn 1 rṇ 2 r n =. r T n n = 1, 2,...,

### Lecture Notes 1: Matrix Algebra Part B: Determinants and Inverses

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 1 of 57 Lecture Notes 1: Matrix Algebra Part B: Determinants and Inverses Peter J. Hammond email: p.j.hammond@warwick.ac.uk Autumn 2012,