# THE APPROXIMATION OF ONE MATRIX BY ANOTHER OF LOWER RANK. CARL ECKART AND GALE YOUNG University of Chicago, Chicago, Illinois

Save this PDF as:

Size: px
Start display at page:

Download "THE APPROXIMATION OF ONE MATRIX BY ANOTHER OF LOWER RANK. CARL ECKART AND GALE YOUNG University of Chicago, Chicago, Illinois"

## Transcription

1 PSYCHOMETRIKA--VOL. I, NO. 3 SEPTEMBER, THE APPROXIMATION OF ONE MATRIX BY ANOTHER OF LOWER RANK CARL ECKART AND GALE YOUNG University of Chicago, Chicago, Illinois The mathematical problem of approximating one matrix by another of lower rank is closely related to the fundamental postulate of factor-theory. When formulated as a least-squares problem, the normal equations cannot be immediately written do~vn, since the elements of the approximate matrix are not independent of one another. The solution of the problem is simplified by first expressing the matrices in a canonic form. It is found that the problem always has a solution which is usually unique. Several conclusions can be drawn from the form of this solution. A hypothetical interpretation of the canonic components of a score matrix is discussed. Introduction If N individuals are each subjected to n tests, it is a fundamental postulate of factor theory that the resulting n X N score matrix a can be adequately approximated by another matrix fl whose rank r is less than the smaller of n or N. Closely associated to this postulate is the purely mathematical problem of finding that matrix fl of rank r which most closely approximates a given matrix a of higher rank R. It will be shown that if the least-squares criterion of approximation be adopted, this problem has a general solution which is relatively simple in a theoretical sense, though the amount of numerical work involved in applications may be prohibitive. Certain conclusions can be drawn from the theoretical solution which may be of importance in practical work. To formulate the problem precisely, it is convenient to define the "scalar product" of two n )~ N matrices as the following numerical function of their elements: (a,,8) ~ ~ ais flis (1) i:l ]=I where.~ is the C]-element of the matrix u. This function has all the properties of the scalar product of vectors: (a, fl)~- (flea) (2)

2 212 PSYCHOMETRIKA (a _ fl, r) ~ (a, r) (fi, r) ; (3) (za, fl) ~--- z (a, if z i s a number; (4) (a,a) >0 ifa=fi0. (5) The positive number l, defined by l 2 ~ (a, a) may be called the length of the matrix a; the number l ~ was called the span (Spannung) of by Frobenius.* The length of the matrix a -- ~ may be called the distance from a to ft. The problem can now be formulated in a definite manner: to find that matrix ~ of rank r, such that there is no other matrix of rank r whose distance from a is less than the distance from ~ to a. This amounts to requiring a least-squares solution of the approximation problem, every element of the given matrix being given equal weight. Some preliminary theorems and remarks. To simplify the following discussion, let a, b, u,... denote n X n matrices, A, B, U,... denote N X N matrices, and a, fl,... denote n ~( N matrices; the special case n --- N is not excluded. The N ~( matrix which is obtained by writing the columns of a as rows is the transpose of a and will be denoted by a. The products as, aa, ab, a a,... are defined in the usual way. It is then seen that the following equations are correct: (a, ~) -- (a, ~ ) ; (6) (act, fl) = (a, a fl) ~-- (a, fla ) ; (7) (aa, ~) ~- (a, fla ) = (A, a fi) (8) From Eq. (7) and (8) it follows that if u and U are orthogonal trices (u u --- u u ~- ln, U U U U ~- 1N), then (ua U, u fl U ) = (a, fl) (9) Another useful proposition is the following: if (a, b) ~ 0 for all symmetric (skew-symmetric) matrices, b, then a is skew-symmetric (symmetric). The solution of the problem is much simplified by an appeal to two theorems which are generalizations of well-known theorems on square matrices/f They will not be proven here. *Quoted from MacDuffee, "Theory of Matrices", Ergebn. d. Mathem., v. 2, No. 5 p. 80 (19~). tcourant and Hilbert, "Methoden der mathematischen Physik" Berlin, 1924; pp. 9 et seq., p. 25. MacDuffee, p. 78.

3 CARL ECKART AND GALE YOUNG 213 Theorem I. For any rea~ matrix a, two ortkogonal m~trices u and U c~. be found so that ~ ~ uau is a real diagonal matrix with no negative ele~ents. A diagonal matrix ~ (square or rectangular) is one for which i;; ~- 0 unless i ~-.i. If a diagonal matrix is rectangular, then there will be some rows or columns which consist entirely of zeros. For the following, this remark is of some importance, as will be seen. The equation of the theorem may also be written a~u" ~ U (10) whose right side may be called the canonic resolution of a. If n ~ N, ~ will have N~n columns of zeros and a is seen to depend only on the first n rows of U. If u, ~ and the first n rows of U are given, a is determined. Let v be the diagonal n ~ n matrix which consists of the first n columns of ~, and ~o the n ~( N matrix composed of the first n rows of U; then these remarks can be summarized by the equation a = u ~ v ~o (10.1) where ~o o) ,, but ~o ~o =/= 1~.. For numerical work, Eq. (10.1) preferable to Eq. (10) ; for formal manipulation, Eq. (10) is convenient. The numerical evaluation of u and v (or ~) can be accomplished from the consideration of the matrix a ~- a a" alone. This matrix is closely related to the matrix of correlation coefficients of the tests. It is seen that and since ~ ), ~ ~- i s adiagonal matrix, it fol lows that u i s oneof t he orthogonal matrices which transform the correlational matrix to diagonal form. The rows of u are unit vectors along the.principal axes of ~ and the squares of the diagonal elements of v (or ~) are the characteristic values of a; this shows that the latter can never be negative numbers, a result which can also be obtained more directly.* The methods for determining the principal axes and characteristic values of a symmetric matrix are also known,? so that these remarks may be considered as indicating the method for calculating u and ~. If none of the characteristic values of a is zero (this will presumably be the case in the overwhelming proportion of actual calculations) *Courant-Hilbert, p. 20. Courant-Hilbert, pp. 13, 16.

4 214 PSYCHOMETRIKA the matrix v will have a reciprocal, and co can be obtained by solving Eq. (10.1) co--- V-1 U t. The numerical values of the elements in the remaining rows of the matrix U will not be needed, but could be found if necessary. For simplicity of manipulation, it is convenient to proceed as though this has been done. The diagonal elements of 2 were called the "canonical multipliers" of a by Sylvester.* The multipliers and characteristic values of a correlational matrix are identical; in the case of a symmetric matrix, there may be a difference in sign; for a general square matrix, there is no simple relation between the two; for a rectangular matrix, the characteristic values are not defined. The correlational matrices, Sylvester called the "false squares" of a. In the foregoing (and in the usual treatment of factor theory) only the matrix a ~ a a" has been considered. However, the matrix A ~ a a is related to the correlation coefficients of the individuals in the same manner as a is related to the correlation coefficients of the tests. There is complete mathematical symmetry between the two correlation matrices. To every multiplier, there is associated a row of u and a row of U; this complex of n ~ N -~- 1 numbers may be called a canonic component of a. Theorem II. If a ~" and ~ a are both symmetric matrices, then and only then can two arthogonal matrices u and U be found such that ~ -~ u a U and ~ ~ u fl U" are both real diagonal matrices. Either one (but in general, not both) of the diagonal matrices may be further restricted to have no negative elements. This theorem is a generalization of the theorem that the principal axes of two symmetric matrices coincide if and only if ab -- ha. Solution of the problem The distance of fl from a is given by x, where x2~ (a,a) --2(a, fl) ~- (fl, fl) ; (11) x is a function of all the elements of fl, and these are to be determined go that its value is a minimum. The elements of fl are not all indepen-.dent, however, because of the requirement that its rank be less than the number of its rows or columns. Theorem I makes it possible *Messeng. Math., 1~ p. 45 (1889).

5 CARL ECKART AND GALE YOUNG 215 to eliminate some of the interdependence: suppose ~ to have been resolved into canonic form: fl~u ~ V (12) with ~ diagonal, and u and U orthogonal matrices. Then the rank of fl will be r if and only if/~ has this rank i.e., if just r of the diagonal elements of ~ are different from zero; the non-vanishing elements of ~ will be independent. However, the elements of u or U will not be independent, since these matrices must be orthogonal. It is not necessary to express these matrices in terms of independent parameters because of the following proposition:* if u is any orthogonal matrix and the independent variables that determine it are given any infinitesimal increments, the resulting increment of u is ~u=us, (13) where s is a skew-symmetric matrix whose elements are infinitesimal, but otherwise arbitrary. The Eq. (11) becomes, because of Eq. (12) and x2~ (a,a) --2(a,u ~ U).~t_ (~,~u) (14) Since x is to be a minimum, it follows that ~ x 2 ~ 0 when u is given the increment ~ u (Eq. (13)). Hence 0 ~ (a, -- s u" ~ U) ~ -- (a, s fl) ~ -- (a ~, (15) Since s is an arbitrary skew-symmetric matrix, it follows that a ~t must be symmetric. Discussing the increment of U in the same manner, it will be found that ~/Ya must also be symmetric, and hence, by Theorem II, the orthogonal matrices can be found so that Eq. (12) and (with X the diagonal matrix of the multipliers Then Eq. (11) becomes a ~u 2 U (12.1) of a) are both valid. x ~ ~ ()~--~, ;t--r) --~]~ 2 (2~ -- ~u~) (11.1) 2i and #~ being the diagonal elements of the corresponding matrices. It remains to determine the matrix ~ so that this expression has its minimum value, subject to the condition that just r of the #~ shall *Courant-Hitbert, p. 2.7.

6 216 PSYCHOMETRIKA be different from zero. It may be supposed that 21 >_~ 22 >_~ 2~ >=.-. ; then an obvious solution of the problem is ~2~, i~r ; ~0, i)r. (17) This is also the only solution unless ~ The procedure of finding fl may be su~arized: (1.) express in the ~nonic fore of Eq. (12.1) ; (2.) ~place all but r of the agonal elements of 2 by zeros, be~nning ~th the smallest and continuing in order of increasing ma~itude. (3.) The resulting matrix is ~, and fl is given by Eq. (12). The solution is u~que unless The minimum value of x ~ is x 2~ ~ ~2 (R being the rank of a and the multipliers being numbered as before). This leads readily to the following conclusion: if 1 is the len~h of a, the smallest upper bound for x is ~(1~/~)1~2. This can be u ilized in esti~ting the si~ificance of an approximation ~ a ob~ined by some other meth~ than the presen~ one. Another impo~nt conclusion is that if ~ is the bes~ appro~ma- ~ion (o~ ~n~ ~) ~o.. e~ n ~ = ~ (~ ~ ~ ~) ~U also be thebest a~roximation. (of rank r) ~ a =. ~ (~ =.~.). T~us. if ~ approximation ~o ~he s~re ~a~ix is found, ~hen ~he co~elational matri e~ calcula~d ~mm it ~1~ auwmatically be the best approximations W the correlational ~tn es calculated from the on.hal score matrix. This is not su~rising, but ~uires p~of; the pr~f is readily supplied from ~he foregoing result. Concerning a hypothetical interpretation of t]~e canonic components~ofl It is reasonable to inquire if, when a is a score matrix, those of its canonic components that enter into fl may be interpreted as the independent factors that determine the differences in the performance of various individuals. In order to discuss this question let p,- 1,.. n number the tests, i ~ 1, -.. N number the individuals, and e or ~ ~- 1, ~. r number the components of fl that have non-vanishing multipliers.. Then the component e consists of the n ~- N ~- 1 numbers up~, 2p>0, U~.

7 which satisfy CARL ECKART AND GALE YOUNG 217 the equations ~.., u#p u,,~, = ~,, ; ~ U~,~ U,,~ ~,~,, (18) fl~,~ ~ ~ u~p 2~ U~. (19) These equations suggest that if these numbers have any empirical significance, then N 1/" U~ will be the standard score of individual i for the ability to exercise the factor e, and n 1/2 ups, the standard score of the test p for its demand for the exercise of the factor. The appearance of the multipliers in Eq. (19) does not correspond to the accepted postulates of factor theory, and their interpretation is thus not immediate. To clarify this point, one may inquire after those empirical circumstances, which if they are actually realized, will lead one to accept the foregoing interpretation. One such set of circumstances would be the following: suppose that two sets of individuals, N1 and N,~ in number, have been tested by the same battery of n tests. The scores of the NI individuals can then be arranged in an n )~ N~ matrix as, those of the N~ individuals into an n ~( NI~ matrixa~[ ; or, the scores of all N~.~- N,, individuals can be arranged into an n X (N[.~- N,,) matrix a obtained by adjoining a~ and am First consider a[ and ai~; if these can be approximated by matrices fl~ and fl~ having the same u-matrices* (within reasonable limits of error), then the foregoing interpretation would be appropriate. In this case the u-matrix obtained by approximating a would also be the same as the others. The multipliers ~,i, 2~,,~ and 2~, and of course also the U~,~, U~,~ and U~ obtained from the three score-matrices would not be identical. In order to discuss their interrelations, let the index i now run from 1 to N~ -~- Nm the first N~ values designating individ. uals of the first set, etc. Then it can be shown that and ~ U~ = 2~,~ Up~,~, if i ~ Nz ; (20) = ;t~,~1 U~,~, if i > N~. (21) These are precisely the relations that would be expected according to the suggested interpretation if 2~,,~/N~ and ~.,~z 2/N~ were the variances of the tw(~ sets of individuals. This is not a tenable interpretation, however, because of the symmetric manner in which the *The sufficient condition for this equality is that a[ a~[ be a symmetric matrix, where a~ ~ a~a~, etc. This criterion can be applied even when u~ and u~ are not known.

8 PSYCHOMETRIKA individuals and the tests enter into the calculations. It may, however, be supposed that ~p.~2/nn~ is the product of the variance of the set of individuals with the variance of the set of tests. It should be noted that it has been implicitly assumed that the scores are not measured from the averages of the different sets of individuals, but from a fiducial zero which is the same for all sets.

### MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.

Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix.

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. Matrices Definition. An m-by-n matrix is a rectangular array of numbers that has m rows and n columns: a 11

### Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20

Matrix algebra January 20 Introduction Basics The mathematics of multiple regression revolves around ordering and keeping track of large arrays of numbers and solving systems of equations The mathematical

### Math 312 Homework 1 Solutions

Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.

MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Chapter 19. General Matrices. An n m matrix is an array. a 11 a 12 a 1m a 21 a 22 a 2m A = a n1 a n2 a nm. The matrix A has n row vectors

Chapter 9. General Matrices An n m matrix is an array a a a m a a a m... = [a ij]. a n a n a nm The matrix A has n row vectors and m column vectors row i (A) = [a i, a i,..., a im ] R m a j a j a nj col

### Similar matrices and Jordan form

Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

### 1 Introduction to Matrices

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

### LECTURE 1 I. Inverse matrices We return now to the problem of solving linear equations. Recall that we are trying to find x such that IA = A

LECTURE I. Inverse matrices We return now to the problem of solving linear equations. Recall that we are trying to find such that A = y. Recall: there is a matri I such that for all R n. It follows that

### 13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

### Using the Singular Value Decomposition

Using the Singular Value Decomposition Emmett J. Ientilucci Chester F. Carlson Center for Imaging Science Rochester Institute of Technology emmett@cis.rit.edu May 9, 003 Abstract This report introduces

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### 5.3 ORTHOGONAL TRANSFORMATIONS AND ORTHOGONAL MATRICES

5.3 ORTHOGONAL TRANSFORMATIONS AND ORTHOGONAL MATRICES Definition 5.3. Orthogonal transformations and orthogonal matrices A linear transformation T from R n to R n is called orthogonal if it preserves

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### Inner product. Definition of inner product

Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product

### 1.5 Elementary Matrices and a Method for Finding the Inverse

.5 Elementary Matrices and a Method for Finding the Inverse Definition A n n matrix is called an elementary matrix if it can be obtained from I n by performing a single elementary row operation Reminder:

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components

Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### Matrices, Determinants and Linear Systems

September 21, 2014 Matrices A matrix A m n is an array of numbers in rows and columns a 11 a 12 a 1n r 1 a 21 a 22 a 2n r 2....... a m1 a m2 a mn r m c 1 c 2 c n We say that the dimension of A is m n (we

### B such that AB = I and BA = I. (We say B is an inverse of A.) Definition A square matrix A is invertible (or nonsingular) if matrix

Matrix inverses Recall... Definition A square matrix A is invertible (or nonsingular) if matrix B such that AB = and BA =. (We say B is an inverse of A.) Remark Not all square matrices are invertible.

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6

Chapter 7 Matrices Definition An m n matrix is an array of numbers set out in m rows and n columns Examples (i ( 1 1 5 2 0 6 has 2 rows and 3 columns and so it is a 2 3 matrix (ii 1 0 7 1 2 3 3 1 is a

### Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

### 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each)

Math 33 AH : Solution to the Final Exam Honors Linear Algebra and Applications 1. True/False: Circle the correct answer. No justifications are needed in this exercise. (1 point each) (1) If A is an invertible

### CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES. From Exploratory Factor Analysis Ledyard R Tucker and Robert C.

CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES From Exploratory Factor Analysis Ledyard R Tucker and Robert C MacCallum 1997 180 CHAPTER 8 FACTOR EXTRACTION BY MATRIX FACTORING TECHNIQUES In

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### Matrices, transposes, and inverses

Matrices, transposes, and inverses Math 40, Introduction to Linear Algebra Wednesday, February, 202 Matrix-vector multiplication: two views st perspective: A x is linear combination of columns of A 2 4

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Inverses and powers: Rules of Matrix Arithmetic

Contents 1 Inverses and powers: Rules of Matrix Arithmetic 1.1 What about division of matrices? 1.2 Properties of the Inverse of a Matrix 1.2.1 Theorem (Uniqueness of Inverse) 1.2.2 Inverse Test 1.2.3

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### The Solution of Linear Simultaneous Equations

Appendix A The Solution of Linear Simultaneous Equations Circuit analysis frequently involves the solution of linear simultaneous equations. Our purpose here is to review the use of determinants to solve

### 2.1: MATRIX OPERATIONS

.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

### Mathematics Notes for Class 12 chapter 3. Matrices

1 P a g e Mathematics Notes for Class 12 chapter 3. Matrices A matrix is a rectangular arrangement of numbers (real or complex) which may be represented as matrix is enclosed by [ ] or ( ) or Compact form

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### The Inverse of a Matrix

The Inverse of a Matrix 7.4 Introduction In number arithmetic every number a ( 0) has a reciprocal b written as a or such that a ba = ab =. Some, but not all, square matrices have inverses. If a square

### 15.062 Data Mining: Algorithms and Applications Matrix Math Review

.6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

### 10. Graph Matrices Incidence Matrix

10 Graph Matrices Since a graph is completely determined by specifying either its adjacency structure or its incidence structure, these specifications provide far more efficient ways of representing a

### Direct Methods for Solving Linear Systems. Linear Systems of Equations

Direct Methods for Solving Linear Systems Linear Systems of Equations Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Math 313 Lecture #10 2.2: The Inverse of a Matrix

Math 1 Lecture #10 2.2: The Inverse of a Matrix Matrix algebra provides tools for creating many useful formulas just like real number algebra does. For example, a real number a is invertible if there is

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

### 9 Matrices, determinants, inverse matrix, Cramer s Rule

AAC - Business Mathematics I Lecture #9, December 15, 2007 Katarína Kálovcová 9 Matrices, determinants, inverse matrix, Cramer s Rule Basic properties of matrices: Example: Addition properties: Associative:

### Topic 1: Matrices and Systems of Linear Equations.

Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method

### Lecture 21: The Inverse of a Matrix

Lecture 21: The Inverse of a Matrix Winfried Just, Ohio University October 16, 2015 Review: Our chemical reaction system Recall our chemical reaction system A + 2B 2C A + B D A + 2C 2D B + D 2C If we write

### MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

Facts About Eigenvalues By Dr David Butler Definitions Suppose A is an n n matrix An eigenvalue of A is a number λ such that Av = λv for some nonzero vector v An eigenvector of A is a nonzero vector v

### 1 Introduction. 2 Matrices: Definition. Matrix Algebra. Hervé Abdi Lynne J. Williams

In Neil Salkind (Ed.), Encyclopedia of Research Design. Thousand Oaks, CA: Sage. 00 Matrix Algebra Hervé Abdi Lynne J. Williams Introduction Sylvester developed the modern concept of matrices in the 9th

### 18.06 Problem Set 4 Solution Due Wednesday, 11 March 2009 at 4 pm in 2-106. Total: 175 points.

806 Problem Set 4 Solution Due Wednesday, March 2009 at 4 pm in 2-06 Total: 75 points Problem : A is an m n matrix of rank r Suppose there are right-hand-sides b for which A x = b has no solution (a) What

### Inner Product Spaces and Orthogonality

Inner Product Spaces and Orthogonality week 3-4 Fall 2006 Dot product of R n The inner product or dot product of R n is a function, defined by u, v a b + a 2 b 2 + + a n b n for u a, a 2,, a n T, v b,

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### ( % . This matrix consists of \$ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&

Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important

### Inverses. Stephen Boyd. EE103 Stanford University. October 27, 2015

Inverses Stephen Boyd EE103 Stanford University October 27, 2015 Outline Left and right inverses Inverse Solving linear equations Examples Pseudo-inverse Left and right inverses 2 Left inverses a number

### Determinants. Dr. Doreen De Leon Math 152, Fall 2015

Determinants Dr. Doreen De Leon Math 52, Fall 205 Determinant of a Matrix Elementary Matrices We will first discuss matrices that can be used to produce an elementary row operation on a given matrix A.

### Review Jeopardy. Blue vs. Orange. Review Jeopardy

Review Jeopardy Blue vs. Orange Review Jeopardy Jeopardy Round Lectures 0-3 Jeopardy Round \$200 How could I measure how far apart (i.e. how different) two observations, y 1 and y 2, are from each other?

### = [a ij ] 2 3. Square matrix A square matrix is one that has equal number of rows and columns, that is n = m. Some examples of square matrices are

This document deals with the fundamentals of matrix algebra and is adapted from B.C. Kuo, Linear Networks and Systems, McGraw Hill, 1967. It is presented here for educational purposes. 1 Introduction In

### The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression

The Singular Value Decomposition in Symmetric (Löwdin) Orthogonalization and Data Compression The SVD is the most generally applicable of the orthogonal-diagonal-orthogonal type matrix decompositions Every

### These axioms must hold for all vectors ū, v, and w in V and all scalars c and d.

DEFINITION: A vector space is a nonempty set V of objects, called vectors, on which are defined two operations, called addition and multiplication by scalars (real numbers), subject to the following axioms

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Notes on Orthogonal and Symmetric Matrices MENU, Winter 2013

Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,

### The Determinant: a Means to Calculate Volume

The Determinant: a Means to Calculate Volume Bo Peng August 20, 2007 Abstract This paper gives a definition of the determinant and lists many of its well-known properties Volumes of parallelepipeds are

### Lecture 23: The Inverse of a Matrix

Lecture 23: The Inverse of a Matrix Winfried Just, Ohio University March 9, 2016 The definition of the matrix inverse Let A be an n n square matrix. The inverse of A is an n n matrix A 1 such that A 1

### 4: SINGLE-PERIOD MARKET MODELS

4: SINGLE-PERIOD MARKET MODELS Ben Goldys and Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2015 B. Goldys and M. Rutkowski (USydney) Slides 4: Single-Period Market

### is in plane V. However, it may be more convenient to introduce a plane coordinate system in V.

.4 COORDINATES EXAMPLE Let V be the plane in R with equation x +2x 2 +x 0, a two-dimensional subspace of R. We can describe a vector in this plane by its spatial (D)coordinates; for example, vector x 5

### Lecture 5 - Triangular Factorizations & Operation Counts

LU Factorization Lecture 5 - Triangular Factorizations & Operation Counts We have seen that the process of GE essentially factors a matrix A into LU Now we want to see how this factorization allows us

### MATHEMATICS FOR ENGINEERS BASIC MATRIX THEORY TUTORIAL 2

MATHEMATICS FO ENGINEES BASIC MATIX THEOY TUTOIAL This is the second of two tutorials on matrix theory. On completion you should be able to do the following. Explain the general method for solving simultaneous

### MAT Solving Linear Systems Using Matrices and Row Operations

MAT 171 8.5 Solving Linear Systems Using Matrices and Row Operations A. Introduction to Matrices Identifying the Size and Entries of a Matrix B. The Augmented Matrix of a System of Equations Forming Augmented

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### Section 5.3. Section 5.3. u m ] l jj. = l jj u j + + l mj u m. v j = [ u 1 u j. l mj

Section 5. l j v j = [ u u j u m ] l jj = l jj u j + + l mj u m. l mj Section 5. 5.. Not orthogonal, the column vectors fail to be perpendicular to each other. 5..2 his matrix is orthogonal. Check that

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### MAT 200, Midterm Exam Solution. a. (5 points) Compute the determinant of the matrix A =

MAT 200, Midterm Exam Solution. (0 points total) a. (5 points) Compute the determinant of the matrix 2 2 0 A = 0 3 0 3 0 Answer: det A = 3. The most efficient way is to develop the determinant along the

### MATH 551 - APPLIED MATRIX THEORY

MATH 55 - APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### MATH36001 Background Material 2015

MATH3600 Background Material 205 Matrix Algebra Matrices and Vectors An ordered array of mn elements a ij (i =,, m; j =,, n) written in the form a a 2 a n A = a 2 a 22 a 2n a m a m2 a mn is said to be

### Hypothesis Testing in the Classical Regression Model

LECTURE 5 Hypothesis Testing in the Classical Regression Model The Normal Distribution and the Sampling Distributions It is often appropriate to assume that the elements of the disturbance vector ε within

### Multiple regression - Matrices

Multiple regression - Matrices This handout will present various matrices which are substantively interesting and/or provide useful means of summarizing the data for analytical purposes. As we will see,

### On the general equation of the second degree

On the general equation of the second degree S Kesavan The Institute of Mathematical Sciences, CIT Campus, Taramani, Chennai - 600 113 e-mail:kesh@imscresin Abstract We give a unified treatment of the

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### CS3220 Lecture Notes: QR factorization and orthogonal transformations

CS3220 Lecture Notes: QR factorization and orthogonal transformations Steve Marschner Cornell University 11 March 2009 In this lecture I ll talk about orthogonal matrices and their properties, discuss

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Pearson Education, Inc.

2 Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Theorem 8: Let A be a square matrix. Then the following statements are equivalent. That is, for a given A, the statements are either all true

### APPLICATIONS OF MATRICES. Adj A is nothing but the transpose of the co-factor matrix [A ij ] of A.

APPLICATIONS OF MATRICES ADJOINT: Let A = [a ij ] be a square matrix of order n. Let Aij be the co-factor of a ij. Then the n th order matrix [A ij ] T is called the adjoint of A. It is denoted by adj

### Math 54. Selected Solutions for Week Is u in the plane in R 3 spanned by the columns

Math 5. Selected Solutions for Week 2 Section. (Page 2). Let u = and A = 5 2 6. Is u in the plane in R spanned by the columns of A? (See the figure omitted].) Why or why not? First of all, the plane in

### STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. Clarificationof zonationprocedure described onpp. 238-239

STATISTICS AND DATA ANALYSIS IN GEOLOGY, 3rd ed. by John C. Davis Clarificationof zonationprocedure described onpp. 38-39 Because the notation used in this section (Eqs. 4.8 through 4.84) is inconsistent

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Lecture L3 - Vectors, Matrices and Coordinate Transformations

S. Widnall 16.07 Dynamics Fall 2009 Lecture notes based on J. Peraire Version 2.0 Lecture L3 - Vectors, Matrices and Coordinate Transformations By using vectors and defining appropriate operations between

### Solution. Area(OABC) = Area(OAB) + Area(OBC) = 1 2 det( [ 5 2 1 2. Question 2. Let A = (a) Calculate the nullspace of the matrix A.

Solutions to Math 30 Take-home prelim Question. Find the area of the quadrilateral OABC on the figure below, coordinates given in brackets. [See pp. 60 63 of the book.] y C(, 4) B(, ) A(5, ) O x Area(OABC)

### Chapter 17. Orthogonal Matrices and Symmetries of Space

Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length