# Matrix Algebra, Class Notes (part 1) by Hrishikesh D. Vinod Copyright 1998 by Prof. H. D. Vinod, Fordham University, New York. All rights reserved.

Save this PDF as:

Size: px
Start display at page:

Download "Matrix Algebra, Class Notes (part 1) by Hrishikesh D. Vinod Copyright 1998 by Prof. H. D. Vinod, Fordham University, New York. All rights reserved."

## Transcription

1 Matrix Algebra, Class Notes (part 1) by Hrishikesh D. Vinod Copyright 1998 by Prof. H. D. Vinod, Fordham University, New York. All rights reserved. 1 Sum, Product and Transpose of Matrices. If a ij with i 1, 2,, m and j 1, 2,, n are a set of real numbers arranged in m rows and n columns we have an m by n matrix A (a ij ). For example, when m 2 and n 3 we have a 2 3 matrix A defined as: [ ] [ ] a11 a 12 a A a 21 a 22 a (1) In the above example if we had three rows instead of two, it would have been a square matrix. A square matrix has as many rows as columns, or m n. Vectors are matrices with only one column. Ordinary numbers can also be thought to be matrices of dimension 1 1, and are called scalars. Similar to most authors we use only column vectors, unless stated otherwise. Both vectors and matrices follow some simple rules of algebra. (1) Sum. If A (a ij ) and B (b ij ) then C (c ij ) is defined by c ij a ij + b ij. This is defined only when both A and B are of exactly the same dimension, and obtained by simply adding the corresponding terms. Exercise verify that A+B is a matrix of all zeros when A is given by (1) and [ ] [ ] b11 b 12 b B b 21 b 22 b (2) (2) Product. If A (a ij ) is m n and B (b ij ) has i 1, 2,, n and j 1, 2,, p then the product matrix C (c ij ) is of dimension m p, and is obtained by row-column multiplication as follows. c ij Σ n t1 a itb tj, where i 1, 2,, m and j 1, 2,, p. Note that a it represent the elements of i-th row and b tj represent elements of j-th column. It is obvious that the above product can be defined only when the range of t is the same for both matrices A and B, that is, A should have as many columns as B has rows. An easy way to remember this is that for our row-column multiplication, column-rows must 1

2 match. Exercise 1: Verify that the product AB does not exist or is not defined for A and B defined by (1) and (2). Since A has 3 columns B must have exactly 3 rows and any number of columns for the product AB to exist. Exercise 2: Verify that a product of two column vectors D and E does not exist if we write it as DE. We have to write the first vector D as a row vector D, and then write D E. Exercise 3 Let the vector x [ 11 2 ] be a column vector. Verify that the sum of squares of its elements s is a scalar, and that s x x, where x denotes row vector [11 2]. In general we do not expect AB BA. In fact BA may not even exist even when AB is well defined. In the rare case when AB BA we say that matrices A and B commute. (3) Multiplication of a matrix by a scalar element. If h is an element and A is a matrix as above ha Ah C defined by C (c ij ), where c ij h a ij. (4) Multiplication of a matrix by a vector. Let A be m n as above, and x ( x 1, x 2,, x n ) be a 1 n row vector. Now the multiplication Ax is not defined. To see this, simply treat multiplication of a matrix by a vector as a product of two matrices above. On the other hand, if x is a column vector n 1, then Ax c is an m 1 column vector. (5) Rules for Sum and Product of more than two matrices. (i) Associativity. A+B + C A+ (B+C) (A + B) +C; or ABC A(BC) (AB)C. (ii) Distributivity: A(B+C) AB + AC. (B+C)A BA + CA. (iii) Identity: If O m,n is an m n matrix of all zeroes, then A+O m,n A If I n is an n n matrix (δ ij ), where δ ij 1 when i j and δ ij 0 otherwise, it is called the identity matrix. Verify that the nonzero elements of I are only along it main diagonal, hence it is a diagonal matrix. AI n A. Note that cancellation and simplification of matrices in complicated matrix expressions is usually obtained by making some expressions equal to the identity matrix. (6) Transpose of a matrix is usually denoted by a prime, and is obtained by interchanging rows and columns. A (a ji ). The transpose of a transpose gives the original matrix. If a matrix equals its transpose, it is called a symmetric matrix. Exercise 1: Find the transpose of A in (1) and verify that the product A B where B is from (2) is well defined. The transpose A has 3 columns matching the 3 rows of B, giving the column-row match mentioned above. A useful rule for the transpose of a product of two matrices AB (not necessarily square matrices) is: (AB) B A, where the order of multiplication reverses. Exercise 2: Verify that the transpose of (A B) B A, and that when A is from (1) and B is from (2) B A is also well defined. The transpose B has 2 columns, matching the 2 rows of A, giving the necessary column-row match mentioned above. 2

3 2 Determinant of a Square Matrix and Singularity Let the matrix A be a square matrix A (a ij ) with i and j 1,2,, n in this section. The determinant is a scalar number. When n 2 we have a simple calculation of the determinant as follows: a 11 a 12 A a 21 a 22 a 11a 22 a 21 a 22, (1) where one multiplies the elements along the diagonal (going from top left corner to bottom right corner) and subtracts a similar multiplication of diagonals going from bottom to top. When n 3 we need to expand it in terms of three 2 2 determinants and the first row as follows: a 11 a 12 a 13 A a 21 a 22 a 23 a 11 A 11 a 12 A 12 + a 13 A 13, (2) a 31 a 32 a 33 where the upper case A with subscripts is a new notation. A ij denotes a submatrix formed by erasing i-th row and j-th column from the original A. For example, [ ] a21 a 22 A 13. This is also called a minor. A ij in (2) denotes the determinant of a 31 a 32 the 2 2 matrix. Exercise: Verify the expansion of A of (2) upon substituting for determinants of minors: A a 11 a 22 a 33 a 11 a 23 a 32 c+ a 12 a 23 a 31 a 12 a 21 a 33 + a 13 a 21 a 32 a 13 a 22 a 31. An easy way to compute a 3 3 determinant is to carefully write its numbers as a 3 5 grid with the two extra columns created by simply repeating the first two columns. Note that this expanded matrix has three diagonals top to bottom and three diagonals bottom to top. Now by analogy with the 2 2 matrix multiply out and add the elements along the three diagonals (going from top left corner to bottom right corner) and subtract a similar multiplication of three diagonals going from bottom to top. Your exercise is to verify that this easy method yields exactly the same six terms as the ones listed above. For example, a d g det b e h c f j 3

4 a d g a d b e h b e c f j c f gives the solution as aej + dhc + gbf ceg fha jbd. Cofactor: Signed determinants of minors A ij are called cofactors. Denote c ij ( 1) i+j A ij (3) In general, for any n one can expand the determinant of A, denoted by A or det(a) in terms of cofactors as det(a) n a ij c ij for any one row from out of i 1, 2, n. j1 n i1 a ij c ij for any one column from out of j 1, 2, n. (4) An alternative definition of a determinant is det(a) Σ σ sgn(σ)π n i1 a i,σ(i) (5) where σ runs over all permutations of integers 1,2,, n and the function sgn(σ) is sometimes called signature function. Here it is 1 or -1 depending on whether a permutation is odd or even. For example, if n 2, the even permutation of { 1,2} is { 1,2}, whereby σ(1) 1 and σ(2) 2. Now the odd permutation of { 1,2} is { 2,1} having σ(1) 2 and σ(2) 1. Exercise: Verify that the determinant of a simple 2 2 matrix A defined by (5) remains a 11 a 22 a 12 a Properties of Determinants Following properties of determinants are useful, and may be verified as an exercise: 1. det(a) det(a ), where A is the transpose of A. 2. det(ab) det(a)det(b) for the determinant of a product of two matrices. Unfortunately, for the determinant of a sum of two matrices there is no simple equality or inequality: det(a+b) det(a) + det(b). 4

5 3. If B is obtained from A by interchanging a pair of rows (or columns), then det(b) det(a) 4. If B is obtained from A by multiplying the elements of a row (or column), by constant k, then det(b) k det(a). Exercise: How is this property of determinants different from matrices? If A is a matrix ka means multiply each element of A by k, but in computing k det(a) only one row or column is multiplied by k. Show that this follows from the expansions in (4). Also, show that det(ka) k n det(a), whence det( A) ( 1) n det(a), by choosing k If B is obtained from A by multiplying the elements of i-th row (or column), of A by constant k, and adding the result of j-th row of A then det(b) det(a). 6. Determinant of a diagonal matrix is simply the product of diagonal entries. 7. Determinant is a product of eigenvalues. det(a) Π n i1 λ i(a), where the eigenvalues are defined in Section 4 below. Zero Determinant and Singularity. When is the det(a) 0? (1) If two rows of A are identical (2)If two columns of A are identical, (3)If a row or column has all zeros. (4) One eigenvalue is zero. Nonsingularity: Matrix A is nonsingular if det(a) 0. 3 The rank and trace of a matrix An T p matrix X is said to be of rank p if the dimension of the largest nonsingular square submatrix is p. (Recall that nonsingular means nonzero determinant, and X is often the matrix of regressors with T observations). The idea of the rank is related to linear independence as follow. We have rank(a) min[ Row rank(a), Column rank(a)], (1) where Row rank is the largest number of linearly independent rows, and where Column rank is the largest number of linearly independent columns. In the following example, [ ] [ ] a11 a 12 a A a 21 a 22 a (1b) Note that a 12 2a 11 and a 22 2a 21. Hence the first two columns are linearly dependent, and there are only two linearly independent columns. The column rank is only 2. Recall that a set of vectors a 1, a 2, a n is linearly dependent if a set of scalars c i exists which are 5

6 not all zero and which satisfy n i1 c i a i 0. In (1) we can let a 1 [ ] and a 2 [ ] and choose c 1 2 and c 2 1 to verify that the summation is indeed zero. Exercise: Show that for a square nonsingular matrix, the rank row rank column rank. 3.1 Properties of the Rank of a Matrix Following properties of rank are useful: 1) rank(x) min(t,p), which says that the rank of a matrix is no greater than the smaller of the two dimensions of rows (T) and columns(p). In Statistics and Econometrics texts one often encounters the expression that the matrix X of regressors is assumed to be of full (column) rank. Since the number of observations in a regression problem should exceed the number of variables, T> p should hold, which means min(t,p) p, whence rank(x) p. If rank(x) p, the rank is the largest it can be, and hence we say that X is of full rank. 2) rank(a) rank(a ). 3) rank(a+ B) rank(a) + rank(b). 4) Sylvester s Law: If A is m n and B is n q defined over a field rank(a)+rank(b) n rank(ab) min [ rank(a), rank(b)] 5) If B is nonsingular rank(ab) rank(ba) rank(a) 6) If A is a matrix of real numbers, rank(a) rank(a A) rank(a A). Assuming the matrix products AB, BC and ABC are well defined, we have rank(ab)+rank(bc) rank(b)+rank(abc). 7) The rank equals the number of nonzero eigenvalues(defined in Section 5) of a matrix. With the advent of computer programs for eigenvalue computation, it is sometimes easier to determine the rank by merely counting the number of nonzero eigenvalues. 3.2 Trace of a Matrix Trace is simply the summation of the diagonal elements. It is also the sum of eigenvalues. Tr(A) n a ii n λ i (A) i1 i1 Also note that Tr(A+B) T r(a) + Tr(B), and Tr(AB) T r(ba). It is customary to use the latter result to simplify matrix expressions. 4 Matrix Inverse, Partitioned Matrices and their Inverse Let A ij denote an (n 1) (n-1) submatrix (called a minor) obtained by deleting i-th row and j-th column of an n n matrix A (a ij ). The cofactor of A is defined in equation (3) of Section 2 above as C (c ij ), where i,j the element is c ij ( 1) i+j det(a ij ). The adjoint of 6

7 A is defined as Adj(A) C (c ji ), which involves interchanging the rows and columns of th matrix of cofactors indicated by the subscript ji instead of the usual ij. The operation of interchange is the transpose. (See item 5 of Section 1). 4.1 Matrix Inverse Matrix inverse is defined for only square matrices and denoted by a superscript -1. If A (a ij ) with i,j 1, 2,, n then its inverse A 1 (a ij ) Adj(A)/det(A) ( 1) i+j det(a ji )/det(a) (1) where the element at the i,j location of A 1 is denoted by the superscript a ij, and where A ji is the submatrix of A obtained by eliminating j-th row and i-th column. Since the denominator of the inverse has its determinant, the inverse of a matrix is not defined unless the matrix is nonsingular, i.e., has nonzero determinant. Otherwise, one has the problem of dividing by a zero. Inverse of a product of two matrices: (AB) 1 B 1 A 1, where the order is reversed, which is similar to the transpose of a product of two or more matrices. However, do not apply a similar rule to the inverse of a sum. In general, (A+B) 1 A 1 + B 1. In fact even if A 1 and B 1 exist their sum may not even have an inverse. For example, if B A the sum A+B is the null matrix, which does not have an inverse, since one cannot divide by a zero. 4.2 Solution of a set of Linear Equations Sβ y Let S be a 2 2 matrix defined below and let β (β 1, β 2 ), be the vector of unknown quantities to be computed (solved for) and let y (19, 14) be the right hand known column vector of constants. Upon solving the system of two equations, verify that the following two equations have β 1 2 and β 2 3 as the solutions written as β S 1 y. 5 β β 2 19 (2) 4 β β 2 14 (3) Note that since β (2, 3) is a solution. [ ] [ ] S, Cofactor(S), det(s) [ ] [ ] [ Adj(S), S 1, S 1 y ] [ ] [ ] [ ] 7

8 4.3 Algebra of Partitioned Matrices [ ] [ ] [ ] A11 A 12 B11 B 12 C11 C 12 A, B, and C, are all m n matrices, and where A 21 A 22 B 21 B 22 C 21 C 22 the submatrices have appropriate and comparable dimensions: A ij is m i n j for i and j 1, 2. Now m m 1 + m 2, n n 1 + n 2. Such partitions of A, B and C are called conformable. Now we have the sum defined by [ ] [ ] [ ] A11 A 12 B11 B 12 A11 + B 11 A 12 + B 12 + A 21 A 22 B 21 B 22 A 21 + B 21 A 22 + B 22 (4) The product is defined by: [ ] [ ] [ ] A11 A 12 B11 B 12 A11 B 11 + A 12 B 21 A 11 B 12 + A 12 B 22. (5) A 21 A 22 B 21 B 22 A 21 B 11 + A 22 B 21 A 21 B 12 + A 22 B 22 The submatrices are called blocks and if A 12 0 (the null matrix), A is called a lower block triangular matrix. If A 21 0 it is called upper block triangular. If both A 12 0 and A 21 0 it is called a block diagonal matrix. The determinant of a partitioned matrix is not intuitive, and needs to be learned. There are two answers for the above A: det(a) det(a 11 )det(a 22 A 21 A 1 11 A 12) det(a 22 )det(a 11 A 12 A 1 22 A 21). (6) As an aid to memory, think of the submatrices as scalars, i.e., 1 1 matrices. Now the det(a) A 11 A 22 A 21 A 12 A 11 ( A 22 [A 21 A 12 ]/A 11 ), and the above formula (6) merely replaces the A 11 in the denominator with A 1 11 conformably placed in the middle to make sure that the matrix multiplication is well defined. Exercise: Verify that if A is block diagonal, det(a) det(a 11 )det(a 22 ), and furthermore that the same relation holds for upper or lower triangular A. 4.4 Computation of Elements of the Inverse of a Partitioned Matrix The inverse B of a square partitioned matrix A is also not intuitive. It can be evaluated from the basic definition of any inverse: 8

9 [ ] [ ] [ ] [ ] A11 A 12 B11 B 12 A11 B 11 + A 12 B 21 A 11 B 12 + A 12 B 22 I 0 A 21 A 22 B 21 B 22 A 21 B 11 + A 22 B 21 A 21 B 12 + A 22 B 22 0 I which assumes that the matrix B A 1. The last equation leads to four matrix equations where the first is I A 11 B 11 + A 12 B 21, based on equating the expressions in the 1,1 position of the last two matrices. Now these four equations can be solved for the four elements of B are respectively: B 11 (A 11 A 12 A 1 22 A 21) 1, B 12 A 1 11 A 12(A 22 A 21 A 1 11 A 12), B 21 A 1 22 A 21(A 11 A 12 A 1 22 A 21) 1, and B 22 (A 22 A 21 A 1 11 A 12). Thus the partitioned matrix elements A ij obtain the elements B ij of its inverse. can be used directly in these formulas to 5 Characteristic Polynomial, Eigenvalues and Eigenvectors. There are many notions in Econometrics, such as multicollinearity which are better understood with the help of characteristic polynomials, eigenvalues and eigenvectors. Given a matrix A, it is interesting to note that one can define a characteristic polynomial in λ as: f(λ) det(a λi), (1) where A λi is sometimes called the characteristic matrix of A. How does a determinant become a polynomial? The best way to see this is with an example of a symmetric 2 2 matrix [ ] [ ] [ ] λ 0.7 A λi λ λ (2) Now the determinant det(a λi) (1 λ) , which is seen to be a quadratic (polynomial) in λ. Note that the highest power of λ is 2 which is also the dimension of A. For n dimensional matrix A we would have an n-th degree polynomial in λ. 5.1 Eigenvalues The eigenvalues are also called characteristic roots, proper values, latent roots, etc. These have fundamental importance in understanding the properties of a matrix. For arbitrary square matrices, eigenvalues are complex roots of the characteristic polynomial. After all, not all polynomials have real roots. For example, the polynomial λ has roots λ 1 9

10 i and λ 2 i where i denotes 1. The eigenvalues are denoted by λ i where i 1, 2,, n are in non-increasing order of absolute values. For complex numbers, the absolute value is defined as the square root of the product of the number and its complex conjugate. The complex conjugate of i is i, their product is 1, with square root also 1. Thus i 1 and i 1, and the polynomial λ (λ λ 1 )(λ λ 2 ) holds true. As in the above example, the roots of a polynomial need not be distinct. By the fundamental theorem of algebra, an n-th degree polynomial defined over the field of complex numbers has n roots. Hence we have to count each eigenvalue with its proper multiplicity when we write λ 1 (A) λ 2 (A),, λ n 1 (A) λ n (A). If eigenvalues are all real, we distinguish between positive and negative eigenvalues when ordering them by λ 1 (A) λ 2 (A),, λ n 1 (A) λ n (A). (3) For the cases of interest in statistical applications we usually consider eigenvalues of only those matrices which can be proved to have only real eigenvalues. Exercise: Find the eigenvalues of a symmetric matrix and verify that they are real numbers. 5.2 Eigenvectors Eigenvector z of an n n matrix A is of dimension n 1. These are defined by the relationship: Az λz, which can be written as Az λiz. Now moving Iz to the left hand side, we have (A λi)z 0. In the above example where n 2 this relation is a system of two equations when the matrices are explicitly written out. For the 2 2 symmetric matrix above the two equations are: (1 λ) z z 2 0 (4) 0.7 z 1 + (1 λ)z 2 0 (5) Note that a so-called trivial solution is to choose z 0, that is both elements of z are zero. We ignore this valid solution because it is not interesting or useful. This system of n equations is called degenerate, and it has no solution unless additional restrictions are imposed. The vector z has two elements z 1 and z 2 to be solved from two equations, then why is it that it is degenerate? The problem is that both equations yield the same solution. The two equations are not linearly independent. Exercise: Impose the additional restriction that the z vector must lie on the unit circle, that is z1 2 + z Now show that there are two ways of solving the system of two equations. The choice λ and λ yields the two solutions. Associated with each of the two solutions there are two eigenvectors which satisfy the defining equations Az λ 1 Iz and Az λ 2 Iz. 10

11 The characteristic equation (1) for the above example (n 2) is det(a λi) 0 Π n i1 (λ λ i). (6) A remarkable result known as Cayley-Hamilton theorem states that the matrix A satisfies its own characteristic equation in the sense that if we replace λ by A, we still have [ ][ ] [ ] Π n i1 (A λ ii) (A 1.7 I)(A 0.3 I) (7) 6 Orthogonal Vectors and Matrices Geometrically, orthogonal means perpendicular. Two nonnull vectors a and b are perpendicular to each other if the angle θ between them is 90 degrees or π/2 radians. Geometrically, the vector a is represented as line from the origin to the point A (say) whose coordinates are a 1 to a n in an n-dimensional Euclidian space. Similarly the vector b is represented by a line from the origin to a point B with coordinates b 1 to b n. From analytic Geometry, the line joining the points A and B is represented by a b, and we know the following fact about the angle θ. cos θ a 2 + b 2 a b 2 2 a b, (1) where a is the Euclidian length of the vector a, that is a (Σ n i1 a2 i ), and similarly for b. The length of the line joining the points A and B is a b (Σ n i1 [a i b i ] 2 ) (Σ n i1 [a2 i + b2 i 2 a ib i ]). (2) The vectors are perpendicular or orthogonal if θ π/2, and cos θ 0. Thus the left side of the above equation for cos θ is zero if the numerator is zero, since the denominator is definitely nonzero. Thus we require that a 2 + b 2 a b 2, which amounts to requiring the cross-product term to be zero, or Σ n i1 2a ib i 0, that is Σ n i1 a ib i 0 a b, using the prime to denote the transpose of the column vector a. The reader should clearly understand the representation of summation Σ n i1 a ib i by the simpler a b. In particular if b a one has the squared Euclidian length a 2 a a is the sum of squares of the elements of the vector a. If the Euclidian length of both a and b is unity, and a b 0, that is the vectors are orthogonal to each other they are called orthonormal vectors. For example let the two vectors defining the usual two dimensional axes be e 1 (1, 0) and e 2 (0, 1) respectively. Now e 1 e 2 (0 + 0) 0, that is, e 1 and e 2 are orthogonal. Since e 1 2 e 1 e 1 1 and 11

12 similarly e 2 1 means they are orthonormal also. A remarkable result called Gram-Schmidt orthogonalization is a process which assures that any given m linearly independent vectors can be transformed into a set of m orthonormal vectors by a set of linear equations. 6.1 Orthogonal Matrices The distinction between orthogonal and orthonormal is usually not made for matrices. A matrix A is said to be orthogonal if its transpose equals its inverse. A A 1, which also means that A A I. The determinant of orthogonal matrix is either +1 or 1, and its eigenvalues are also +1 or 1. To make these results plausible, recall that det(ab) det(a)det(b). Hence applying this to the relation A A I for orthogonal A we have det(a)det(a ) det(i). Now det(i) 1 is obvious. 7 Idempotent Matrices In ordinary algebra the familiar numbers whose square is itself are: 1 or 0. Any integer power of 1 is 1. In matrix algebra, the Identity matrix I plays the role of the number 1, and I certainly has the property I n I. There are other nonnull matrices which are not identity and yet have this property, namely A 2 A. Hence a new name idempotent is needed to describe them. For example, the hat matrix is defined as Ĥ X(X X) 1 X. Writing Ĥ2 ĤĤ X(X X) 1 X X(X X) 1 X X(X X) 1 X Ĥ means that Ĥ matrix is idempotent. Verify that Ĥn Ĥ for any integer power n. If eigenvalues are λi(ĥ), the relation Ĥn Ĥ means that λi(ĥn ) λ i (Ĥ) [λi(ĥ) ]n. Since the only numbers whose n-th power is itself are unity and zero, it is plausible that the eigenvalues of an idempotent matrix are 1 or 0. Recall that the rank of a matrix equals the number of nonzero eigenvalues and that the trace of matrix is the sum of eigenvalues. Using these two results it is clear that for idempotent matrices G whose eigenvalues are 0 or 1 we have the interesting relation trace(g) rank(g). 8 Quadratic and Bilinear Forms If x is an n 1 vector and A is an n n matrix the expression x Ax is of dimension 1 1 or a scalar. A scalar does not mean that it is just one term in an expression, only that when 12

14 a 11 a a 11 a 12 a a 11 > 0, a 21 a 22 > 0 a 21 a 22 a 23 > 0,, (3) a 31 a 32 a 33 where we have indicated the principal determinants of orders 1, 2, 3. To see the intuitive reason for these relations consider n 2 and the quadratic form Q a 11 x a 12x 1 x 2 + a 22 x 2 2 (4) Complete the square by adding and subtracting (a 12 /a 11 ) 2 x 2 2. Now write Q a 11 [x 1 + (a 12 /a 11 )x 2 ] 2 + [a 22 (a 2 12 /a 11)]x 2 2. (5) Observe that Q> 0 requires that a 11 > 0 from the first term, and a 11 a 22 a 2 12 > 0 from the second term. The second term is positive if and only if the 2 2 determinant above is positive. 8.2 Bilinear Form In the above discussion of quadratic forms, if we define y as an n 1 vector, x Ay is called a bilinear form. This is also a scalar. Exercise: Let e k denote a column vector of all zeros except for the k-th element, which is unity. The individual elements a ij of the matrix A are obviously scalar. Show that a ij e i Ae j, a bilinear form. 9 Further Study of Eigenvalues and Eigenvectors There are two fundamental results in the analysis of real symmetric matrices. (1) The eigenvalues of a real symmetric matrix are real. This is proved by using the fact that if the eigenvalues were complex numbers, they must come in conjugate pairs. Ax λx and A x λ x, where the bar indicates the conjugate complex number. Premultiply Ax λx by the x vector to yield x Ax λ x x, (λ is a scalar). Now premultiply A x λ x by x to yield x A x λx x. Now the quadratic forms on the left sides of these relations must be equal to each other by the symmetry of A, i.e., x Ax x A x. Hence we have (λ λ)x x 0 which can only be true if λ λ, a contradiction. The complex conjugate cannot be equal to the original. (2) Eigenvectors associated with distinct eigenvalues are orthogonal to each other. 14

15 The proof involves an argument similar to the one above, except that instead of λ we use a distinct root µ say and instead of x we use the vector y. The relation (λ µ)x y 0, now implies that x y 0 since the roots λ and µ are assumed to be distinct. Hence the two eigenvectors x and y must be orthogonal. 9.1 Reduction of a Real Symmetric Matrix to the Diagonal Form Let A be a real symmetric matrix of order n n having distinct eigenvalues λ 1 λ 2 λ n ordered in a sequence from the largest λ 1 max i {λ i, i 1, 2, n} to the smallest λ n min i {λ i, i 1, 2, n}. Denote by G the n n orthogonal matrix of corresponding eigenvectors G [x.1, x.2,, x.n ], where the dot notation is used to remind us that the first subscript is suppressed and each x.i is an n 1 column vector. Orthogonality of G is verified by the property: G G GG. Recall the fundamental relation defining i-th eigenvalue and eigenvector Ax.i λ i x.i, in terms of the dot notation. Denote by Λ diag(λ 1, λ 2,, λ n ), a diagonal matrix containing the eigenvalues in the non-increasing order. Combining all eigenvalues and eigenvectors we write AG GΛ. (Exercise: verify that GΛ ΛG, and that the latter will not work on the RHS). The relation G AG G GΛ Λ is called eigenvalueeigenvector decomposition of the real symmetric matrix A. For example, let [ ] [ ] w w A, G, where w 1/ [ ] , and Λ w w (1) One can easily verify that G G [ I ] GG, and G AG Λ by direct multiplication. In the dot notation, note that x.1 w w and x.2 [ w ] w and G [x.1, x.2 ]. Let us use (1) to verify A GΛG. [ ] [ ] [ ] w w w 0.3w GΛ w w w 0.3w GΛG [ 1.7w w 2 1.7w w 2 ] 1.7w w 2 1.7w w 2 [ 2w 2 1.4w 2 ] [ ] w 2 2w 2 A (2)

16 since w 2 (1/2) by the definition of w. This completes the verification of the eigenvalue eigenvector decomposition in the simple 2 2 case. 9.2 Simultaneous Reduction of two Matrices to a Diagonal Form: Let G be an n n orthogonal matrix of eigenvectors as above, which reduces the n n symmetric matrix A to the diagonal form in the sense that G AG Λ. Now, AB BA is a necessary and sufficient condition that G also reduces another n n symmetric matrix B to a diagonal form with the eigenvalues of B denoted by φ i along the diagonal of Φ, in the sense that GBG Φ, where Φ diag(φ i, i 1, 2,, n). In other words, if and only if A and B commute, there is an orthogonal matrix G with common eigenvectors which simultaneously diagonalizes both matrices. In the above reduction we insisted on G being an orthogonal matrix, with common eigenvectors. For some purposes it is not necessary to have an orthogonal matrix, but one would like to relax the commutativity requirement. Then we have the following result: Given A which is real symmetric, nonsingular and positive definite, and B which is real symmetric, then there exists a non-singular matrix S which diagonalizes A to the identity matrix and simultaneously diagonalizes B, such that S AS I and S BS Φ, where Φ diag(φ i, i 1, 2,, n). 10 Kronecker Product of Matrices Kronecker product of matrices is mainly a notational device. We define the Kronecker product by the notation C A B, where A is T n, B is m p and C is a large matrix of dimension Tm np with elements defined by C (a ij B), where each term a ij B is actually a ij times the whole m p matrix. [ ] b11 b 12 For example, if A (a 11, a 12, a 13 ) with T 1 and n 3, and if B we have b 21 b 22 C [ a 11 [ b11 b 12 b 21 b 22 ], a 12 [ b11 b 12 b 21 b 22 ], a 13 [ b11 b 12 b 21 b 22 [ ] a11 b 11 a 11 b 12 a 12 b 11 a 12 b 12 a 13 b 11 a 13 b 12. a 11 b 21 a 11 b 22 a 12 b 21 a 12 b 22 a 13 b 21 a 13 b 22 In the following, let A, B, C and D be arbitrary matrices. The Kronecker product satisfies the following kinds of linearity and distributive laws. (i) A (ab) a(a B), a being a scalar; 16 ]]

17 (ii) (A + B) C A C + B C, A and B being of the same order. (iii) A (B+C) A B + A C, B and C being of the same order; (iv) A (B C) (A B) C; (v) (A B) A B ; (The order does not reverse here as in simple matrix multiplication) (vi) (A B)(C D) AC BD; (vii) (A B) 1 A 1 B 1. Again, the order does not reverse here as in the inverse of simple matrix multiplication. Obviously we assume that A and B are square non-singular matrices. (viii) tr(a B) (tr A)(trB), A and B being square matrices; (ix) det(a B) (det A) m (det B) n, A and B being n n and m m matrices; Eigenvalues of Kronecker Product equals the product of eigenvalues If A and B are square matrices of dimension m and n respectively, then C A B is mn dimensional matrix. If the eigenvalues of A are denoted by λ i (A) i 1, 2,, m and those of B are denoted by λ j (B) j 1, 2,, n the mn eigenvalues of C are simply a product of the eigenvalues (λ i (A) λ j (B)) with i 1, 2,, m and j 1, 2,, n. Eigenvectors of Kronecker Products contain stacked products of elements of corresponding individual eigenvectors Using the above notation, the eigenvectors of C have k i(j) 1, 2, mn elements each. Let the t-th element of i-th eigenvector of A be denoted by α ti with t 1, 2, m. Also, let the s-th element of j-th eigenvector of B be denoted by β sj with s 1, 2, n. The elements in the k-th eigenvector of C equal (α 1i β sj with s 1, 2, n; α 2i β sj with s 1, 2,, n; ; α mi β sj with s 1, 2, n). 17

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### MATH36001 Background Material 2015

MATH3600 Background Material 205 Matrix Algebra Matrices and Vectors An ordered array of mn elements a ij (i =,, m; j =,, n) written in the form a a 2 a n A = a 2 a 22 a 2n a m a m2 a mn is said to be

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### Introduction to Matrix Algebra I

Appendix A Introduction to Matrix Algebra I Today we will begin the course with a discussion of matrix algebra. Why are we studying this? We will use matrix algebra to derive the linear regression model

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### Mathematics Notes for Class 12 chapter 3. Matrices

1 P a g e Mathematics Notes for Class 12 chapter 3. Matrices A matrix is a rectangular arrangement of numbers (real or complex) which may be represented as matrix is enclosed by [ ] or ( ) or Compact form

### MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

### Determinants. Dr. Doreen De Leon Math 152, Fall 2015

Determinants Dr. Doreen De Leon Math 52, Fall 205 Determinant of a Matrix Elementary Matrices We will first discuss matrices that can be used to produce an elementary row operation on a given matrix A.

### Chapter 17. Orthogonal Matrices and Symmetries of Space

Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.

Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product

### 13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### 1 Determinants. Definition 1

Determinants The determinant of a square matrix is a value in R assigned to the matrix, it characterizes matrices which are invertible (det 0) and is related to the volume of a parallelpiped described

### 2.1: Determinants by Cofactor Expansion. Math 214 Chapter 2 Notes and Homework. Evaluate a Determinant by Expanding by Cofactors

2.1: Determinants by Cofactor Expansion Math 214 Chapter 2 Notes and Homework Determinants The minor M ij of the entry a ij is the determinant of the submatrix obtained from deleting the i th row and the

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

### Chapter 8. Matrices II: inverses. 8.1 What is an inverse?

Chapter 8 Matrices II: inverses We have learnt how to add subtract and multiply matrices but we have not defined division. The reason is that in general it cannot always be defined. In this chapter, we

### INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL

SOLUTIONS OF THEORETICAL EXERCISES selected from INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL Eighth Edition, Prentice Hall, 2005. Dr. Grigore CĂLUGĂREANU Department of Mathematics

### The Inverse of a Matrix

The Inverse of a Matrix 7.4 Introduction In number arithmetic every number a ( 0) has a reciprocal b written as a or such that a ba = ab =. Some, but not all, square matrices have inverses. If a square

### Summary of week 8 (Lectures 22, 23 and 24)

WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry

### Matrix Algebra LECTURE 1. Simultaneous Equations Consider a system of m linear equations in n unknowns: y 1 = a 11 x 1 + a 12 x 2 + +a 1n x n,

LECTURE 1 Matrix Algebra Simultaneous Equations Consider a system of m linear equations in n unknowns: y 1 a 11 x 1 + a 12 x 2 + +a 1n x n, (1) y 2 a 21 x 1 + a 22 x 2 + +a 2n x n, y m a m1 x 1 +a m2 x

### 10. Graph Matrices Incidence Matrix

10 Graph Matrices Since a graph is completely determined by specifying either its adjacency structure or its incidence structure, these specifications provide far more efficient ways of representing a

### Lecture Notes 1: Matrix Algebra Part B: Determinants and Inverses

University of Warwick, EC9A0 Maths for Economists Peter J. Hammond 1 of 57 Lecture Notes 1: Matrix Algebra Part B: Determinants and Inverses Peter J. Hammond email: p.j.hammond@warwick.ac.uk Autumn 2012,

### Helpsheet. Giblin Eunson Library MATRIX ALGEBRA. library.unimelb.edu.au/libraries/bee. Use this sheet to help you:

Helpsheet Giblin Eunson Library ATRIX ALGEBRA Use this sheet to help you: Understand the basic concepts and definitions of matrix algebra Express a set of linear equations in matrix notation Evaluate determinants

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### R mp nq. (13.1) a m1 B a mn B

This electronic version is for personal use and may not be duplicated or distributed Chapter 13 Kronecker Products 131 Definition and Examples Definition 131 Let A R m n, B R p q Then the Kronecker product

### Diagonalisation. Chapter 3. Introduction. Eigenvalues and eigenvectors. Reading. Definitions

Chapter 3 Diagonalisation Eigenvalues and eigenvectors, diagonalisation of a matrix, orthogonal diagonalisation fo symmetric matrices Reading As in the previous chapter, there is no specific essential

### Matrices, Determinants and Linear Systems

September 21, 2014 Matrices A matrix A m n is an array of numbers in rows and columns a 11 a 12 a 1n r 1 a 21 a 22 a 2n r 2....... a m1 a m2 a mn r m c 1 c 2 c n We say that the dimension of A is m n (we

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### Sergei Silvestrov, Christopher Engström, Karl Lundengård, Johan Richter, Jonas Österberg. November 13, 2014

Sergei Silvestrov,, Karl Lundengård, Johan Richter, Jonas Österberg November 13, 2014 Analysis Todays lecture: Course overview. Repetition of matrices elementary operations. Repetition of solvability of

### A matrix over a field F is a rectangular array of elements from F. The symbol

Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F) denotes the collection of all m n matrices over F Matrices will usually be denoted

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### = [a ij ] 2 3. Square matrix A square matrix is one that has equal number of rows and columns, that is n = m. Some examples of square matrices are

This document deals with the fundamentals of matrix algebra and is adapted from B.C. Kuo, Linear Networks and Systems, McGraw Hill, 1967. It is presented here for educational purposes. 1 Introduction In

### Topic 1: Matrices and Systems of Linear Equations.

Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Notes on Symmetric Matrices

CPSC 536N: Randomized Algorithms 2011-12 Term 2 Notes on Symmetric Matrices Prof. Nick Harvey University of British Columbia 1 Symmetric Matrices We review some basic results concerning symmetric matrices.

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### Linear Algebra: Determinants, Inverses, Rank

D Linear Algebra: Determinants, Inverses, Rank D 1 Appendix D: LINEAR ALGEBRA: DETERMINANTS, INVERSES, RANK TABLE OF CONTENTS Page D.1. Introduction D 3 D.2. Determinants D 3 D.2.1. Some Properties of

### Inverses and powers: Rules of Matrix Arithmetic

Contents 1 Inverses and powers: Rules of Matrix Arithmetic 1.1 What about division of matrices? 1.2 Properties of the Inverse of a Matrix 1.2.1 Theorem (Uniqueness of Inverse) 1.2.2 Inverse Test 1.2.3

### The Solution of Linear Simultaneous Equations

Appendix A The Solution of Linear Simultaneous Equations Circuit analysis frequently involves the solution of linear simultaneous equations. Our purpose here is to review the use of determinants to solve

### 4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION

4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION STEVEN HEILMAN Contents 1. Review 1 2. Diagonal Matrices 1 3. Eigenvectors and Eigenvalues 2 4. Characteristic Polynomial 4 5. Diagonalizability 6 6. Appendix:

### MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix.

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. Matrices Definition. An m-by-n matrix is a rectangular array of numbers that has m rows and n columns: a 11

### MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### 5.3 Determinants and Cramer s Rule

290 5.3 Determinants and Cramer s Rule Unique Solution of a 2 2 System The 2 2 system (1) ax + by = e, cx + dy = f, has a unique solution provided = ad bc is nonzero, in which case the solution is given

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### EC9A0: Pre-sessional Advanced Mathematics Course

University of Warwick, EC9A0: Pre-sessional Advanced Mathematics Course Peter J. Hammond & Pablo F. Beker 1 of 55 EC9A0: Pre-sessional Advanced Mathematics Course Slides 1: Matrix Algebra Peter J. Hammond

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20

Matrix algebra January 20 Introduction Basics The mathematics of multiple regression revolves around ordering and keeping track of large arrays of numbers and solving systems of equations The mathematical

### 2.5 Elementary Row Operations and the Determinant

2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)

### Matrix Inverse and Determinants

DM554 Linear and Integer Programming Lecture 5 and Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1 2 3 4 and Cramer s rule 2 Outline 1 2 3 4 and

### Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants

Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants Hartmut Führ fuehr@matha.rwth-aachen.de Lehrstuhl A für Mathematik, RWTH Aachen October 30, 2008 Overview

### The Determinant: a Means to Calculate Volume

The Determinant: a Means to Calculate Volume Bo Peng August 20, 2007 Abstract This paper gives a definition of the determinant and lists many of its well-known properties Volumes of parallelepipeds are

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### 1 Spherical Kinematics

ME 115(a): Notes on Rotations 1 Spherical Kinematics Motions of a 3-dimensional rigid body where one point of the body remains fixed are termed spherical motions. A spherical displacement is a rigid body

### 1.5 Elementary Matrices and a Method for Finding the Inverse

.5 Elementary Matrices and a Method for Finding the Inverse Definition A n n matrix is called an elementary matrix if it can be obtained from I n by performing a single elementary row operation Reminder:

### Matrix Algebra and Applications

Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

### ( % . This matrix consists of \$ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&

Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important

### APPENDIX QA MATRIX ALGEBRA. a 11 a 12 a 1K A = [a ik ] = [A] ik = 21 a 22 a 2K. a n1 a n2 a nk

Greene-2140242 book December 1, 2010 8:8 APPENDIX QA MATRIX ALGEBRA A.1 TERMINOLOGY A matrix is a rectangular array of numbers, denoted a 11 a 12 a 1K A = [a ik ] = [A] ik = a 21 a 22 a 2K. a n1 a n2 a

### Determinants. Chapter Properties of the Determinant

Chapter 4 Determinants Chapter 3 entailed a discussion of linear transformations and how to identify them with matrices. When we study a particular linear transformation we would like its matrix representation

### Chapter 1 - Matrices & Determinants

Chapter 1 - Matrices & Determinants Arthur Cayley (August 16, 1821 - January 26, 1895) was a British Mathematician and Founder of the Modern British School of Pure Mathematics. As a child, Cayley enjoyed

### THREE DIMENSIONAL GEOMETRY

Chapter 8 THREE DIMENSIONAL GEOMETRY 8.1 Introduction In this chapter we present a vector algebra approach to three dimensional geometry. The aim is to present standard properties of lines and planes,

### Economics 102C: Advanced Topics in Econometrics 2 - Tooling Up: The Basics

Economics 102C: Advanced Topics in Econometrics 2 - Tooling Up: The Basics Michael Best Spring 2015 Outline The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices The Evaluation Problem:

### Partitioned Matrices and the Schur Complement

P Partitioned Matrices and the Schur Complement P 1 Appendix P: PARTITIONED MATRICES AND THE SCHUR COMPLEMENT TABLE OF CONTENTS Page P1 Partitioned Matrix P 3 P2 Schur Complements P 3 P3 Block Diagonalization

### Recall the basic property of the transpose (for any A): v A t Aw = v w, v, w R n.

ORTHOGONAL MATRICES Informally, an orthogonal n n matrix is the n-dimensional analogue of the rotation matrices R θ in R 2. When does a linear transformation of R 3 (or R n ) deserve to be called a rotation?

### Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws.

Matrix Algebra A. Doerr Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Some Basic Matrix Laws Assume the orders of the matrices are such that

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### 9 Matrices, determinants, inverse matrix, Cramer s Rule

AAC - Business Mathematics I Lecture #9, December 15, 2007 Katarína Kálovcová 9 Matrices, determinants, inverse matrix, Cramer s Rule Basic properties of matrices: Example: Addition properties: Associative:

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### Linear Algebra I. Ronald van Luijk, 2012

Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.

### 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each)

Math 33 AH : Solution to the Final Exam Honors Linear Algebra and Applications 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each) (1) If A is an invertible

### Applied Linear Algebra I Review page 1

Applied Linear Algebra Review 1 I. Determinants A. Definition of a determinant 1. Using sum a. Permutations i. Sign of a permutation ii. Cycle 2. Uniqueness of the determinant function in terms of properties

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### Facts About Eigenvalues

Facts About Eigenvalues By Dr David Butler Definitions Suppose A is an n n matrix An eigenvalue of A is a number λ such that Av = λv for some nonzero vector v An eigenvector of A is a nonzero vector v

### GRA6035 Mathematics. Eivind Eriksen and Trond S. Gustavsen. Department of Economics

GRA635 Mathematics Eivind Eriksen and Trond S. Gustavsen Department of Economics c Eivind Eriksen, Trond S. Gustavsen. Edition. Edition Students enrolled in the course GRA635 Mathematics for the academic

### 1 Introduction. 2 Matrices: Definition. Matrix Algebra. Hervé Abdi Lynne J. Williams

In Neil Salkind (Ed.), Encyclopedia of Research Design. Thousand Oaks, CA: Sage. 00 Matrix Algebra Hervé Abdi Lynne J. Williams Introduction Sylvester developed the modern concept of matrices in the 9th

### Linear Algebra: Vectors

A Linear Algebra: Vectors A Appendix A: LINEAR ALGEBRA: VECTORS TABLE OF CONTENTS Page A Motivation A 3 A2 Vectors A 3 A2 Notational Conventions A 4 A22 Visualization A 5 A23 Special Vectors A 5 A3 Vector

### Lecture 2 Matrix Operations

Lecture 2 Matrix Operations transpose, sum & difference, scalar multiplication matrix multiplication, matrix-vector product matrix inverse 2 1 Matrix transpose transpose of m n matrix A, denoted A T or

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### Lecture 4: Partitioned Matrices and Determinants

Lecture 4: Partitioned Matrices and Determinants 1 Elementary row operations Recall the elementary operations on the rows of a matrix, equivalent to premultiplying by an elementary matrix E: (1) multiplying

### MATH 511 ADVANCED LINEAR ALGEBRA SPRING 2006

MATH 511 ADVANCED LINEAR ALGEBRA SPRING 26 Sherod Eubanks HOMEWORK 1 1.1 : 3, 5 1.2 : 4 1.3 : 4, 6, 12, 13, 16 1.4 : 1, 5, 8 Section 1.1: The Eigenvalue-Eigenvector Equation Problem 3 Let A M n (R). If

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### 2.5 Gaussian Elimination

page 150 150 CHAPTER 2 Matrices and Systems of Linear Equations 37 10 the linear algebra package of Maple, the three elementary 20 23 1 row operations are 12 1 swaprow(a,i,j): permute rows i and j 3 3

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 6. Eigenvalues and Singular Values In this section, we collect together the basic facts about eigenvalues and eigenvectors. From a geometrical viewpoint,

### University of Lille I PC first year list of exercises n 7. Review

University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients

### Using row reduction to calculate the inverse and the determinant of a square matrix

Using row reduction to calculate the inverse and the determinant of a square matrix Notes for MATH 0290 Honors by Prof. Anna Vainchtein 1 Inverse of a square matrix An n n square matrix A is called invertible

### Linear Algebra: Matrices

B Linear Algebra: Matrices B 1 Appendix B: LINEAR ALGEBRA: MATRICES TABLE OF CONTENTS Page B.1. Matrices B 3 B.1.1. Concept................... B 3 B.1.2. Real and Complex Matrices............ B 3 B.1.3.

### Solution. Area(OABC) = Area(OAB) + Area(OBC) = 1 2 det( [ 5 2 1 2. Question 2. Let A = (a) Calculate the nullspace of the matrix A.

Solutions to Math 30 Take-home prelim Question. Find the area of the quadrilateral OABC on the figure below, coordinates given in brackets. [See pp. 60 63 of the book.] y C(, 4) B(, ) A(5, ) O x Area(OABC)

### STUDY GUIDE FOR SOME BASIC INTERMEDIATE ALGEBRA SKILLS

STUDY GUIDE FOR SOME BASIC INTERMEDIATE ALGEBRA SKILLS The intermediate algebra skills illustrated here will be used extensively and regularly throughout the semester Thus, mastering these skills is an