Linear Algebra for the Sciences. Thomas Kappeler, Riccardo Montalto

Size: px
Start display at page:

Download "Linear Algebra for the Sciences. Thomas Kappeler, Riccardo Montalto"

Transcription

1 Linear Algebra for the Sciences Thomas Kappeler, Riccardo Montalto

2 2

3 Contents Systems of linear equations 5. Linear systems with two equations and two unknowns Gaussian elimination Matrix calculus and related topics Matrix calculus Linear dependence, bases, coordinates Determinants Complex numbers and complex systems of linear equations Complex numbers and their calculus The fundamental theorem of algebra Systems of linear equations with complex coefficients Vector spaces and linear maps 6 4. Vector spaces and their linear subspaces Linear maps Inner products on R vector spaces Isometries and orthogonal matrices Vector product in R Inner products on C vector spaces Eigenvalues and eigenvectors Eigenvalues and eigenvectors of C linear maps on C vector spaces Eigenvalues and eigenvectors of R-linear maps on R-vector spaces Quadratic forms on R n Differential equations Introduction Systems of linear ODEs of first order with constant coefficients Linear ODEs of higher order with constant coefficients

4 4 CONTENTS

5 Chapter Systems of linear equations At the core of linear algebra is the solving of systems of linear equations. A linear system (S) with n unknowns and m equations has the form a x + a 2 x a n x n = b (Eq ) a 2 x + a 22 x a 2n x n = b 2 (Eq 2 ). a m x + a m2 x a mn x n = b m (Eq m ) The numbers a,..., a n, a 2,..., a 2n,..., a m,..., a mn are called the coefficients of the system. The system is said to be real if all coefficients a ij, i =,..., n, j =,..., m as well as the numbers b,..., b m are real. In this chapter we will only consider real systems of linear equations, but the techniques we develop to solve them also apply to complex systems, i.e., systems where a ij and b i are complex numbers which will only be introduced in a later chapter. Given the real system (S), a solution of (S) is a set of n real numbers x,..., x n so that equations (Eq ) (Eq m ) are satisfied simultaneously. The basic questions with regard to linear systems of the form (S) are the following ones: () Does (S) have a solution? (Existence) (2) Does (S) have at most solution? (Uniqueness) Or formulated in more general terms: (3) What are the properties of the set of solutions { } L := (x,..., x n ) : x R,..., x n R; (x,..., x n ) satisfies (S ) (S m ). Here and in the sequel, R denotes the set of real numbers. Note that in this terminology questions () and (2) can be reformulated as follows: ( ) Is L a nonempty set? (2 ) Does L have at most one element? In practical applications, systems of linear equations can be very large. One therefore needs theoretical concepts and numerical algorithms to investigate respectively solve such systems. 5

6 6 CHAPTER. SYSTEMS OF LINEAR EQUATIONS. Linear systems with two equations and two unknowns As an introduction we consider in this section real systems with two equations and two unknowns: { a x + a 2 x 2 = b a 2 x + a 22 x 2 = b 2. In such a case, we usually choose a simpler notation: x x, x 2 y { ax + by = e cx + dy = f. where a, b, c, d, e, f are in R. To get some experience, let us first treat the simpler case of one linear equation with one unknown ax = b (..) where a, b R. The solvability of this equation depends on the values of a and b: Case a 0: equation () has precisely one solution x = b a R or L = { b a Case a = 0 and b 0: equation () has no solution, i.e., L =. Case a = 0 and b = 0: any real number x R is a solution of (), i.e., L = R. Next let us treat the case of one equation and two unknowns: }. ax + by = c, a, b, c R (..2) As above we denote the set of solutions by L, { } L = (x, y) R 2 : ax + by = c. Case a = 0, b = 0, c = 0: L = R 2 Case a = 0, b = 0, c 0: L = Case (a, b) (0, 0): the set L is a straight line in R 2. In case b 0, this straight line is given by y = a b x + c b and In case b = 0 and a 0, L = {( x, a b x + c ) } : x R. b {( c ) L = a, y } : y R.

7 .. LINEAR SYSTEMS WITH TWO EQUATIONS AND TWO UNKNOWNS 7 Now let us go back to the linear system of two equations with two unknowns, ax + by = e (..3) cx + dy = f (..4) The set of solutions L of (..3), (..4) is then given by the intersection of the set of solutions of (..3) with the set of solutions of (..3), L = L L 2, where { } { } L = (x, y) R 2 : ax + by = e, L 2 = (x, y) R 2 : cx + dy = f. In case (a, b) (0, 0) and (c, d) (0, 0), L and L 2 are lines in R 2 and L is the intersection of them. Thus L can be a one point set (L and L 2 are not parallel), or a line (L = L 2 ) or the emptyset (L and L 2 are parallel but do not coincide). Let us now describe an algorithm how to determine the set of solutions of (..3), (..4) in a systematic way. You know this algorithm already from high school. To simplify the algorithm we assume that a 0. (..5) Step : eliminate x from equation (..4) by replacing (..4) by (..4) c (..3) It a means that the left hand side of (..4) is replaced by cx + dy c ( ) ax + by (use that a 0!) a whereas the right hand side of (..4) is replaced by f c a e. The new system of equations then reads as follows ax + by = e (..6) (d c ) a b y = f c a e. (..7) It is straightforward to see that under the assumption (..5), the set solutions of (..3), (..4) coincides with the set of solutions of (..6), (..7). In such a case we say that the two systems are equivalent. Step 2: The system (..6), (..7) is solved by first solving (..7) for y and then use (..6) to determine x: Case d c b 0: then (..7) has the unique solution a y = f c e a af ce d c = b ad bc a and when substituted into (..6) one obtains ( af ce ) ax = e b ad bc

8 8 CHAPTER. SYSTEMS OF LINEAR EQUATIONS or x = de bf ad bc. Hence the set of solutions L consists of one element L = {( de bf af ce )}, ad bc ad bc (..8) Case d c b = 0, f c e 0: equation (..7) has no solutions and hence L =. a a Case d c b = 0, f c e = 0: then any y R is a solution of (..7) and solutions of a a (..6) are given by x = e b y. Hence the set of solutions L is given by a a {( e L = a b ) a y, y } : y R. Motivated by formula (..8) for the solutions of (..6), (..7) in the case where d c a b 0 we make the following definitions: Definition... (i) A real 2 2 matrix (plural: matrices) is an array A of real numbers of the form ( ) a b A :=, a, b, c, d R ; c d (ii) the determinant of a 2 2 matrix A is defined as det(a) := ad bc. The notion of the determinant can be used to characterize the solvability of the system (..6), (..7) and to obtain formulas for its solutions. We state without proof the following Theorem... (i) The system of linear equations (..6), (..7) has a unique solution if and only if ( ) a b det 0. c d ( ) a b (ii) If det 0, then the unique solution of (..6), (..7) is given by the following c d formulas (Cramer s rule) ( ) ( ) e b a e det det f d c f x = ( ), y = ( ) a b a b det det c d c d

9 .2. GAUSSIAN ELIMINATION 9 Problem: Analyze the following system of linear equations { 2y + 4x = 3 x + y = 5 and find its set of solutions. Answer: Let A denote the coefficient matrix, ( ) 4 2 A =. Then det(a) = 6 0. Hence according to Theorem.., the system of equations has a unique solution given by ( ) ( ) det 5 x = = 7 det 6 6, y = 5 = Gaussian elimination Gaussian elimination is an algorithm to determine the set of solutions of an arbitrary system of linear equations with m equations and n unknowns, denoted by x,..., x n, a x + a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2. a m x + a m2 x a mn x n = b m where a ij R, i m (index for the equations) and j n (index for the unknowns). A compact way of writing the above system of equations is achieved using the symbol for the sum, n a ij x j = b i for any i m. (.2.) Of course we could also use different letters than i and j. We denote by L the set of solutions of (.2.), L := { (x,..., x n ) R n : n } a ij x j = b i for any i m. Note that L = m L i, L i := i= { (x,..., x n ) R n : n a ij x j = b i }.

10 0 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Definition.2.. We say that two systems of linear equations and n a ij x j = b i for any i m n c kj x j = d k for any k p are equivalent if their sets of solutions coincide. and An example of two equivalent systems with three unknowns is the following one: { 4x + 3x 2 + 2x 3 = x + x 2 + x 3 = 4 4x + 3x 2 + 2x 3 = x + x 2 + x 3 = 4 2x + 2x 2 + 2x 3 = 8 since the latter equation is obtained from the equation x + x 2 + x 3 = 4 by multiplying left and right hand side by the factor 2. The idea of Gaussian elimination is to replace a given system of linear equations in a systematic way by an equivalent one which is easy to solve. In Subsection. we have demonstrated this method in the case of two equations (m = 2) and two unknowns (n = 2). Gaussian elimination uses the following basic operations, referred to as row operations, which leave the set of solutions of a given system of linear equations invariant: (R) Exchange of two equations (rows) of a system of linear equations. Example: { 5x 2 + 5x 3 = 0 4x + 3x 2 + x 3 = { 4x + 3x 2 + x 3 = 5x 2 + 5x 3 = 0 It means that the equations get listed in a different order. (R2) Multiplication of an equation (row) by a real number α 0. Example: { 4x + 3x 2 + x 3 = 5x 2 + 5x 3 = 0 { 4x + 3x 2 + x 3 = x 2 + 3x 3 = 2 We have multiplied the left and right hand side of the second equation by the factor /5.

11 .2. GAUSSIAN ELIMINATION (R3) An equation (row) gets replaced by the equation obtained by adding to it the multiple of another equation. More formally, this can be expressed as follows: the kth equation n a kjx j = b k is replaced by the equation n n a kj x j + α a lj x j = b k + αb l for some where l m with l k or more explicitly, (a k + αa l )x + + (a kn + αa ln )x n = b k + αb l. Example: { x + x 2 = 5 4x + 2x 2 = 3 (Eq) (Eq2) (Eq2) (Eq2) 4(Eq) { x + x 2 = 5 2x 2 = 7 It is not difficult to verify that these basic row operations lead to equivalent linear systems. We state without proof the following Theorem.2.. The basic row operations lead to equivalent systems of linear equations. We now show how these basic row operations can be used to solve a system (S) of linear equations of the form a x + a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2. a m x + a m2 x a mn x n = b m where a ij ( i m, j n) and b i ( i m) are real numbers. To (S) we associate its coefficient matrix a a n A =... a m a mn A is an array of real numbers with m rows and n columns. Such an array of real numbers is called an m n matrix and written in a compact form as A = (a ij ) i m j n The augmented coefficient matrix of (S) is the following array of real numbers a a n b... a m a mn b m written in a compact form as (A b).

12 2 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Definition.2.2. (A b) is said to be in row echelon form (Zeilenstufenform) if it is of the form b where stands for nonzero elements of A. Note that below the echelon (Stufe), all the coefficients of A vanish. We point out that it is possible that the coefficient matrix A does not have any zero columns (Spalten). Furthermore, if it has zero rows (Zeilen), then these rows have to be at the bottom of A. Examples of augmented coefficient matrices in row echelon form are ( 0 ) 0 0 0, ( 0 ) ( ) 4 0 0, whereas the following ones are not in this form ( ) ( ) 0 0,, ( 0 0 ) ( ) , ( 0 0 ) 0 If the augmented coefficient matrix of a linear system is in row echelon form, it can easily be solved. To illustrate this let us look at few examples. Examples: () { 2x + x 2 = 2 3x 2 = 6 ( 2 ) is in row echelon form. Solve it by first determining x 2 by the second equation and then solving the first equation for x by substituting the obtained value of x 2 3x 2 = 6 x 2 = 2 2x = 2 x 2 2x = 0 x = 0 hence the set of solutions is given by L = {(0, 2)}.

13 .2. GAUSSIAN ELIMINATION 3 (2) { 2x + x 2 + x 3 = 2 3x 3 = 6 ( 2 ) is in row echelon form. Solve the second equation for x 3 and then the first equation for x : 3x 3 = 6 x 3 = 2 ; x 2 is a free variable; 2x = 2 x 2 x 3 x = 2 x 2 and hence L = { ( } 2 x 2, x 2, 2) : x 2 R, which is a straight line in R 3 trough (0, 0, 2) in direction (, 2, 0). 2 (3) The augmented coefficient matrix is in row echelon form. The corresponding system of linear equations can be solved as follows: since 0 x + 0 x 2 = 3 has no solutions, one concludes that L =. 2 (4) The augmented coefficient matrix is in row echelon form. The corresponding system of linear equations can be solved as follows: is satisfied for any x, x 2 R; Hence L = {(, 2)}. (5) The augmented coefficient matrix 0 x + 0 x 2 = 0 3x 2 = 6 x 2 = 2 2x = x 2 x = 2. ( ) 0 2 is in row echelon form x 3 = 6, x 2 = 2x 3 =, x free variable, hence L = { } (x,, 6) : x R which is a straight line in R 3 trough the point (0,, 6) in direction (, 0, 0).

14 4 CHAPTER. SYSTEMS OF LINEAR EQUATIONS ( ) (6) The augmented coefficient matrix is in row echelon form hence 3x 4 = 6 x 4 = 2 ; x 3 and x 4 are a free variables ; L = x = 2x 2 x 4 = 2x 2 2, { } ( 2x 2 2, x 2, x 3, 2) : x 2, x 3 R which is a plane in R 4 containing the point ( 2, 0, 0, 2) and spanned by the vectors (0, 0,, 0) and ( 2,, 0, 0). ( ) (7) The augmented coefficient matrix is in row echelon form Hence x 4 is a free variable ; 3x 3 = 6 x 3 = 2 L = x = 2x 2 x 3 = 2x 2 2. { } ( 2x 2 2, x 2, 2, x 4 ) : x 2, x 4 R which is a plane in R 4 containing the point ( 2, 0, 2, 0) and spanned by the vectors (0, 0, 0, ) and ( 2,, 0, 0). Gaussian elimination is a method of transforming a given augmented coefficient matrix with the help of basic row operations into row echelon form. How can this be achieved? We explain the procedure with a few examples. It is convenient to introduce for the three basic row operations the following notations: R i k : exchange rows i and k R k αr k : replace k-th row R k by αr k, α 0. R k R k + αr l : replace k-th row by adding to it αr l where l k and α R. Examples ( ) () The augmented coefficient matrix is not in row echelon form. Apply 2 2 R 2 to get ( ) ( ) 2 (2) The augmented coefficient matrix is not in row echelon form. The first row is ok; in the second row we have to eliminate 4; hence R 2 R 2 2R yielding ( ) 2 0 2

15 .2. GAUSSIAN ELIMINATION 5 (3) The augmented coefficient matrix 2 0 is not in row echelon form. The first row is ok; in second and third row we have to eliminate 2 respectively 4. Hence R 2 R 2 2R and R 3 R 3 4R Now R and R 2 are ok, but we need to eliminate 3 from the last row. R 3 R 3 3R 2, yielding which is in row echelon form. Hence (4) The augmented coefficient matrix is not in row echelon form. R is ok, but we need to eliminate the first coefficients from subsequent rows: yielding R 2 R 2 + R, R 3 R 3 + 2R, R 5 R 5 R, Rows R, R 2 are ok, but we need to eliminate the third coefficients in the rows R 3, R 4, R 5. R 3 R 3 2R 2, R 4 R 4 R 2, R 5 R 5 R 2, yielding

16 6 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Now R, R 2, R 3 are ok, but we need to eliminate the last coefficients in R 4 and R 5, i.e., R 4 R 4 R 3, R 5 R 5 R 3, leading to which is in row echelon form. 0 (5) The augmented coefficient matrix 0 0 is not in row echelon form. 2 0 R is ok, but we need to eliminate the first coefficients in R 2, R 3, i.e., yielding R 2 R 2 + R, R 3 R 3 + 2R, To bring the lattest augmented coefficient matrix in row echelon form we need to exchange the second and the third row, R 2 3, leading to which is in row echelon form. We now want to go one step further and describe the set of solutions of a system of linear equations in more detail. We begin by making some preliminary considerations. Consider the system (S) { x + 4x 2 = b 5x + 2x 2 = b 2. The corresponding augmented coefficient matrix is given by ( ) 4 b (A b) =. 5 2 b 2 Let us compare it with the system ( S), obtained by exchanging the two columns of A. Introducing as new unknowns y, y 2 this system reads { 4y + y 2 = b 2y + 5y 2 = b 2

17 .2. GAUSSIAN ELIMINATION 7 and the corresponding augmented coefficient matrix is given by (à b) where à = ( ) Denote by L and L the set of solutions of (S) respectively ( S). It is easy to see that the map (x, x 2 ) (y, y 2 ) := (x 2, x ) induces a bijection between L and L. It means that any solution (x, x 2 ) of (S) leads to the solution y := x 2, y 2 := x of ( S) and conversely, any solution (y, y 2 ) of ( S) leads to a solution x := y 2, x 2 := y of (S), or said differently, by renumerating the unknowns x, x 2, we can read off the set of solutions of ( S) from the one of (S). This procedure can be used to bring an augmented coefficient matrix (A b) in row echelon form into an even simpler form: assume that A has m rows and n columns. By renumerating the unknowns, which corresponds to a permutation of the columns of A, (A b) can be brought into the echelon form (à b), ã 0 ã ã 33. à = 0 ã kk b where k is an integer with 0 k min(m, n) and ã 0, ã 22 0,..., ã kk 0. If k = 0, then à is the matrix whose entries are all zero. Note that now all echelons have height and length equal to one. Using the row operations (R2) and (R3), (à b) can be simplified. First we apply (R2) to the rows R i, i k, (R i ) ã ii (R i ) yielding ã 2 ã 3 ã k 0 ã à =. 0 ã (k )k b

18 8 CHAPTER. SYSTEMS OF LINEAR EQUATIONS and then we apply (R3) to remove all coefficients ã ij with < i < j k to obtain â (k+) â n â 2(k+) â 2n... Â = 0 0 â k(k+) â kn b referred to as being in refined echelon form. The system of linear equations corresponding to this latter augmented coefficient matrix (Â b) is then the following one: y + n j=k+ âjy j = b. y k + n j=k+ âkjy j = b k n 0 y j = b i k + i m. Note that the unknowns are denoted by y,..., y n since the original unknowns x,..., x n might have been permuted, that 0 k min(m, n), and that the set of solutions of the latter system is given by L, introduced above. The sets L and L can now easily be determined. We have to distinguish between different cases: CASE : k < m and there exists i with k + i m so that b i 0. Then L = and hence L =. CASE 2: either [k = m] or [k < m and b k+ = 0,..., b m = 0]. Then the system above reduces to the system of equations n y i + â ij y j = b i, for any i k. (.2.2) j=k+ CASE 2A: if in addition k = n, then the system (.2.2) reads y i = b i for any i n. It means that L = {( b,..., b n )} and therefore the system with augmented coefficient matrix (A b) we started with, has a unique solution. CASE 2B: if in addition k < n, then the system (.2.2) reads y i = b n i â ij y j for any i k j=k+ and the unknowns y k+,..., y n are free variables, also referred to as parameters and denoted by t k+,..., t n. The set of solutions L is then given by n n ) } {( b â j t j,..., b k â kj t j, t k+,..., t n : t k+,..., t n R arbitrary. j=k+ j=k+

19 .2. GAUSSIAN ELIMINATION 9 Hence the system (.2.2) and therefore also the original system with augmented coefficient matrix (A b) has infinitely many solutions. The map R n k R n, given by b â (k+) â (k+2) â n.... t k+. t n bk 0 + t k â k(k+) is a parameter representation of L t k+2 â k(k+2) Let us now illustrate the discussed procedure with a few examples: () Consider the case of one equation and two unknowns, Then 0. 0 a x + a 2 x 2 = b, a t n â kn x + â 2 x 2 = b, â 2 = a 2, b = b a a and we are in the CASE 2B with k =, m =, n = 2. The set of solutions L = L (no renumeration of unknowns were necessary) has the following parameter representation ) ( ) ( b R R 2 â2, t 2 + t 2. 0 It is a straight line in R 2, passing through the point ( b, 0) and having the direction ( â 2, ). (2) Consider the case of one equation and three unknowns, a x + a 2 x 2 + a 3 x 3 = b, a 0. We divide the equation by a and obtain where x + â 2 x 2 + â 3 x 3 = b â 2 = a 2, â 3 = a 3, b = b a a a and we are again in the case 2-B with k =, m =, n = 3. The set of solutions L = L has the following parameter representation ( ) b â 2 â 3 R 2 R 3 t2, 0 + t t 2 + t

20 20 CHAPTER. SYSTEMS OF LINEAR EQUATIONS It is a plane in R 3 passing through the point ( b, 0, 0) and spanned by the vectors ( â 2,, 0) and ( â 3, 0, ). (3) Consider the following system x + x 2 + x 3 = 4, x x 2 2x 3 = 0. (.2.3) The corresponding augmented coefficient matrix (A b) is given by ( ) Apply row operation (R3) and replace R 2 by R 2 R to get ( ) Apply row operation (R2), R 2 2 R 2 to get ( ) and finally apply again row operation (R3), R R R 2, yielding ( ) , which is in refined echelon form. We are in the CASE 2B with k = 2, m = 2, n = 3. Since we have not permuted the unknowns, L = L and a parameter representation of L is given by R R 3, 2 t t 3 0 /2 3/2 which is a straight line in R 3, passing through the point (2, 2, 0) and having the direction (/2, 3/2, ). (4) Consider x + 2x 2 x 3 = 2x + x 2 + x 3 = 0 3x + 0 x 2 + 3x 3 =. Apply row operation (R3) by replacing R 2 R 2 2R, R 3 R 3 3R, yielding

21 .2. GAUSSIAN ELIMINATION 2 Apply (R3) once were, R 3 R 3 2R 2, leading to the following augmented coefficient matrix in echelon form Now apply (R2), R 2 3 R 2 to get and finally we apply (R3) once more, R R 2R 2, to get the following augmented coefficient matrix in refined echelon form We are in CASE 2B with k = 2, m = 3, n = 3 and L = L. Hence a parameter representation of L is given by R R 3, /3 t 3 2/3 + t 3 0 which is a straight line in R 3 passing through the point ( /3, 2/3, 0) and having the direction (,, ). From our analysis one can deduce the following Theorem.2.2. If (S) is a system of m linear equations and n unknowns with m < n, then its set of solutions is either empty or infinite. Remark.2.3. To see that Theorem.2.2 holds, one argues as follows: bring the augmented coefficient matrix in refined row echelon form. Then k min(m, n) = m < n since by assumption m < n. Hence CASE 2A cannot occur and we are either in CASE (L = ) or in CASE 2B (L infinite). An important class of linear systems is the one where the number of equations is the same as the number of unknowns, m = n. Definition.2.3. A matrix A is called quadratic if the number of its rows equals the number of its columns.

22 22 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Definition.2.4. We say that a n n matrix A = (a ij ) i,j n is a diagonal matrix if A = diaga, where diaga = (d ij ) i,j n is the n n matrix with d ii = a ii i n, d ij = 0 i j. It is called the n n identity matrix if a ii = for any i n and a ij = 0 for i j. We denote it by Id n or Id n n. Going through the procedure described above for transforming the augmented coefficient matrix (A b) of a given system (S) of n linear equations with n unknowns into refined echelon form, one sees that (S) has a unique solution if and only if it is possible to bring (A b) without renumbering the unknowns into the form (Id n b), yielding the solution x = b,..., x n = b n. Definition.2.5. We say that a n n matrix A is regular if it can be transformed by the row operations (R) (R3) to the identity matrix Id n. Otherwise A is called singular. Theorem.2.4. Assume that (S) is a system of n linear equations and n unknowns with augmented coefficients matrix (A b), i.e., n a ij x j = b i, i n. Then the following holds: (i) If A is regular, then for any b R n, n a ij x j = b i, i n. has a unique solution. (ii) If A is singular, then for any b R n, the system n a ij x j = b i, i n. has either no solution at all or infinitely many. Example: Assume that A is a singular n n matrix. Then the system with augmented coefficient matrix (A 0) has infinitely many solutions. Corollary.2.5. Assume that A is a n n matrix, A = (a ij ) i,j n. If there exists b R n so that n a ij x j = b i, i n, has a unique solution, then for any b R n, n a ij x j = b i, i n. has a unique solution.

23 .2. GAUSSIAN ELIMINATION 23 Definition.2.6. A system of the form n a ij x j = b i, i m is called a homogeneous system of linear equations if b = (b,..., b m ) = (0,..., 0) and otherwise inhomogeneous. Given a inhomogeneous system the system n a ij x j = b i, i m, n a ij x j = 0, i m is referred to as the corresponding homogeneous system. Note that a homogeneous system of linear equations has always the zero solution. Theorem.2.2 then leads to the following corollary. Corollary.2.6. A homogeneous system of m linear equations with n unknowns and m < n has infinitely many solutions. We summarize the results for a system of n linear equations with n unknowns as follows. Corollary.2.7. Assume that we are given a system (S) n a ij x j = b i, i n, where b = (b,..., b n ) R n. Then the following statements are equivalent: (i) If (S) has a unique solution. (ii) The homogeneous system corresponding to (S) has only the zero solution. (iii) A = (a ij ) i,j n is regular. Finally, we already remark at this point that the set of solutions L of the linear system n a ijx j = b, i m, and the set of solutions L h of the corresponding homogeneous system n a ijx j = 0, i m, have the following properties which can be verified in a straightforward way: (P) for any x, x L h and any λ R, one has x + x L h, and λx L h. (P2) for any x, x L, one has x x L h. (P3) for any x L and x L h, one has x + x L. We will come back to these properties after having introduced the notion of vector spaces.

24 24 CHAPTER. SYSTEMS OF LINEAR EQUATIONS

25 Chapter 2 Matrix calculus and related topics The aim of this chapter to introduce the notion of a matrix and to discuss the elementary properties of matrices as well as related topics. 2. Matrix calculus The aim of this section is to discuss the basics of the matrix calculus. We denote by Mat m n (R) or by R m n the set of all real m n matrices a a n A = (a ij ) i m =.. j n a m a mn Definition 2.. (Sum, multiplication by scalars). (i) For any A = (a ij ) i m j n B = (b ij ) i m in R m n we denote by A + B the m n matrix given by j n (ii) For any A = (a ij ) i m j n (a ij + b ij ) i m R m n. j n in R m n and λ R we denote by λa the m n matrix (λa ij ) i m R m n. j n Note that for the sum of two matrices A and B to be defined, they have to have the same number of rows and the same number of columns. So, e.g., the two matrices ( ) 2 and (6 7) 3 4 cannot be added. The definition of the multiplication of matrices is more complicated. It is motivated by the interpretation of a m n matrix as a linear map from R m to R n which we will discuss in the subsequent chapter in detail. We will see that the multiplication of matrices corresponds to the composition of the corresponding linear maps. At this point however it is only important to know that the definition of the multiplication of matrices is very well motivated. and 25

26 26 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Definition Assume that A = (a il ) i m R m n and B = (b lj ) l n R n k. l n j k Then the product of A and B, denoted by A B or AB for short, is the m k matrix C = (c ij ) i m with coefficients given by j k c ij := n a il b lj. l= Note that in the computation of the coefficient c ij only the i-th row of A, (a i,..., a in ), and the j-th column of B, b j. b nj are involved. Examples () Let A = (2) Let A = (3) Let A = ( ) 2, B = 3 4 ( ) 2, B = 3 4 AB = ( ) 2, B = 3 4 AB = (4) Let A = ( 2), B = ( ). Then A R 2 2 2, B R 2 and AB is well defined. AB = ( ) ( ) = R ( ) 0. Then A, B R and ( ) = ( ) 7 2 R ( ) 0. Then A R , B R 2 3 and ( ) = ( ) 3. Then A R 4 2, B R 2 and AB = = R ( R) ( ) 7 2 R and BA = ( ) = ( ) 3 6 R The following theorem states elementary properties of the matrix multiplication.

27 2.. MATRIX CALCULUS 27 Theorem 2... The following holds: (i) matrix multiplication is associative: for any A R m n, B R n k, C R k l, (ii) matrix multiplication is distributive: (ii) for any A, B R m n, C R n k (ii2) for any A R m n, B, C R n k (AB)C = A(BC) ; (A + B)C = AC + BC ; A(B + C) = AB + AC ; (iii) operation with the identity matrix: for any A R m n, A Id n = A, Id m A = A. To get acquainted with matrix mutiplication let us verify item (i) of Theorem 2.. in the case m = 2, n = 2, k = 2. For any A, B, C R 2 2, the identity A(BC) = (AB)C is verified as follows: let D := BC, E := AB. We claim that AD = EC. Indeed and (AD) ij = (EC) ij = 2 a ik d kj = k= 2 e il c lj = l= 2 k= a ik 2 b kl c lj = l= 2 ( 2 a ik b kl )c lj = l= k= 2 k= l= 2 l= k= 2 a ik b kl c lj 2 a ik b kl c lj. Matrix multiplication is not commutative, ( i.e., ) in general, ( for A, ) B R n n, one has 0 0 AB BA. As an example consider A = and B =. Then AB BA 0 2 since ( ) ( ) ( ) AB = = whereas BA = ( ) ( ) = ( ) 0 2 As a consequence, in general (AB) 2 A 2 B 2. However matrices in certain special classes commute with each other. The set of diagonal n n matrices is such a class. Indeed if A, B R n n are both diagonal matrices, A = diag(a), B = diag(b), then AB is also a diagonal matrix, AB = diag(ab), with implying that AB = BA. (AB) ii = a ii b ii, i n, We say that two n n matrices A, B commute if AB = BA. AB = BA. They anticommute if

28 28 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Definition A matrix A R n n is called invertible if there exists B R n n so that AB = BA = Id n. One can easily verify that for any given A R n n there exists at most one matrix B R n n so that AB = BA = Id n. Indeed assume that C R n n satisfies AC = CA = Id n. Then B = B Id n = B(AC) = (BA)C = Id n C = C. Hence if A R n n is invertible, there exists a unique matrix B R n n so that AB = BA = Id n. This matrix is denoted by A and is called the inverse of A. Note that the notion of the inverse is only defined for quadratic matrices. Examples in R 2 2 : () The null matrix (2) Similarly, ( ) 0 0 is not invertible since for any B R ( ) 0 0 B = 0 0 ( ) ( ) b b 2 = b 3 b 4 ( ) 0 is not invertible since for any B R ( ) 0 B = 0 0 ( ) ( ) b b 2 = b 3 b 4 (3) The identity matrix Id 2 is invertible and Id 2 = Id 2. ( ) 3 0 (4) The diagonal matrix A = is invertible and 0 4 (5) The matrix A = ( ) A 0 = ( ) 0 0 Id ( ) b3 b 4 Id ( ) 2 is invertible. One can easily verify that 3 4 A = 2 ( ) ( ) a b How can the inverse of an invertible 2 2 matrix A = be computed? In the case c d where det(a) 0, it turns out that A is invertible and its inverse A is given by ( ) ( ) A d b d b det(a) det(a) = =. det(a) c a c det(a) a det(a)

29 2.. MATRIX CALCULUS 29 Indeed, one has ( ) ( ) A d b a b A = det(a) c a c d ( ) da bc db bd = = Id det(a) ca + ac cb + ad 2. Similarly one verifies that AA = Id 2. Before we describe a procedure to find the inverse of an invertible n n matrix, let us state some general results on invertible n n matrices. First let us introduce GL R (n) := { } A R n n : A is invertible where we remark that GL stands for general linear. Theorem The following holds: (i) For any A, B GL R (n), one has AB GL R (n) and (AB) = B A. (ii) For any A GL R (n), A GL R (n) and (A ) = A. (2..) To get acquainted with the notion of the inverse of an invertible matrix, let us verify the above statement (i): first note that if A, B GL R (n), then A, B are well defined and so is B A. To see that B A is the inverse of AB we compute and similarly (B A )(AB) = B (A A)B = B Id n B = B B = Id n (AB)B A = A(BB )A = A Id n A = AA = Id n. Hence by definition of the inverse we have that AB is invertible and (AB) is given by B A. To see that (ii) holds we argue similarly. Note that in general, A B is not the inverse of AB, but of BA. Hence in case A and B do not commute, neither do A and B. The important questions with regard to the invertibility of a quadratic matrix A are the following ones: () How can we decide if A is invertible? (2) In case A is invertible, how can we find its inverse?

30 30 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS It turns out that the two questions are closely related and can be answered by the following procedure: consider the system (S) of n linear equations with n unknowns a x + + a n x n = b. a n x + + a nn x n = b n where A = (a ij ) i,j n and b = (b,..., b n ) R n. We want to write this system in matrix notation. For this purpose we consider b as a n matrix and similarly, we do so for x, b b n x b =., x =. Since A is a n n matrix, the matrix multiplication of A and x is well defined and Ax R n. Note that n (Ax) j = a jk x k = a j x + + a jn x n. k= x n. Hence the above linear system (S), when written in matrix notation, takes the form Ax = b. Now let us assume that A is invertible. Then the matrix multiplication of A and Ax is well defined and A (Ax) = (A A)x = Id n x = x. Hence multiplying left and right hand side of Ax = b with A, we get If we choose x = A b. b = e () = 0., 0 then x = A e () is the first column of A. More generally, if 0. 0 b = e (j) = 0. 0

31 2.. MATRIX CALCULUS 3 where is the j-th component of b = e (j) and j n, then A e (j) is the j-th column of A. Summarizing, we have seen that we can decide if A is invertible and if so, determine A, by solving the following systems of linear equations Ax = e (), Ax = e (2),, Ax = e (n). In case A is regular, the solutions x (),, x (n) of the latter equations are uniquely determined and are the columns of A. In that case, we are thus led to apply the following procedure for determining the inverse of A: form the following version of the augmented coefficient matrix a a n a n a nn 0 0 Use Gaussian elimination to determine the solutions x (),, x (n). consequence, one gets the following As an immediate Theorem The quadratic matrix A is invertible if and only if A is regular. In case A is invertible, the linear system Ax = b has the solution x = A b. We recall that a n n matrix A is regular if it can be transformed into the identity matrix Id n by the row operations (R) (R3). In such a case, the above version of the augmented coefficient matrix gets transformed into a a n a n a nn b b n b n b nn and A is given by the matrix (b ij ) i,j n. Let us now illustrate the procedure with a few examples: for each of the matrices A below, decide if A is invertible and if so, determine its inverse. ( ) () For A = consider the augmented coefficient matrix and apply Gaussian elimination ( ) 0 0

32 32 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS (i) R 2 R 2 + R : ( ) (ii) R 2 2 R 2 : ( ) (iii) R R R 2 : ( 0 thus A = ( ) /2 /2 = /2 /2 2 ) ( ). Note that the result coincides with the one obtained by the formula ( ) d b. det(a) c a 2 2 (2) For A = consider the augmented coefficients matrix 2 3 and apply Gaussian elimination: (i) R R 2 R, R 3 R 3 2R : (ii) R 3 R R 2 : (iii) R 3 4 R 3, R 2 3 R 2 :

33 2.. MATRIX CALCULUS 33 (iv) (R ) R 2R 3, R 2 R 2 + R 3 : (v) R R + 2R 2 : and ( ) 2 3 (3) For A = 4 6 A = consider the augmented coefficients matrix ( 2 3 ) and apply Gaussian elimination: (i) R 2 R 2 + 2R : ( 2 3 ) It follows that A is not regular and hence according to Theorem 2..3, A is not invertible. We finish this section by introducing the notion of the transpose of a matrix. Definition Given A R m n, we denote by A T the n m matrix for which the i-th row is given by the j-th column of A. More formally, Examples: () ( ) 2 A = R 3 4 A T = 4 (2) A = 2 5 R 3 2 A T = 3 6 A T = (b ij ) i n, with b ij := a ji j m ( ) 3 R ( ) 2 3 R

34 34 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS (3) A = 2 R 3 A T = ( 2 3) R 3 3 Definition A quadratic matrix A = (a ij ) i,j n R n n is said to be symmetric if A = A T or, written coefficient wise Examples: () A = (2) A = ( ) 2 is symmetric. 2 3 ( ) 2 is not symmetric. 3 a ij = a ji i, j n. (3) If A R n n is a diagonal matrix, then A is symmetric. (Recall that A is a diagonal matrix if A = diag(a).) Theorem (i) For any A R m n, B R n k, (AB) T = B T A T (ii) For any A GL R (n), also A T GL R (n) and (A T ) = (A ) T. To get more acquainted with the notion of the transpose of a matrix, let us verify the statements above: given A R m n, B R n k, one has AB R m k and for any i m, j k, (AB) ij = n a il b lj = n (A T ) li (B T ) jl = n (B T ) jl (A T ) li = (B T A T ) ji l= l= l= and on the other hand, by the definition of the transpose matrix, ((AB) T ) ji = (AB) ij. Combining the two identities yields (AB) T = B T A T. To see that for any A GL R (n) also A T GL R (n), the candidate for the inverse of A T is the matrix (A ) T. Indeed one has (A T )(A ) T = (A A) T = Id T n = Id n since Id n = Id T n, since Id n is diagonal. Similarly (A ) T A T (i) = (AA ) T = Id T n = Id n.

35 2.2. LINEAR DEPENDENCE, BASES, COORDINATES Linear dependence, bases, coordinates In this section we introduce the important notions of linear (in)dependence of vectors in R k, of a basis of R k and of coordinates of a vector in R k with respect to a basis. Later in this course we will discuss these notions in the more general framework of vector spaces of which R k is an example. Elements of R k are called vectors and written as a = (a,..., a k ) where a j, j k are called the components of a. Alternatively, we view a as a k matrix a a =. Definition Assume that a (),, a (n) are vectors in R k. A vector b R k is said to be a linear combination of the vectors a (),, a (n) if there exist real numbers α,, α n so that n b = α a () + + α n a (n) or b = α j a (j). Example: Consider the vectors a () = (, 2), a (2) = (2, ) R 2. Then b = (b, b 2 ) = (, 5) is a linear combination of a () and a (2) since b = 3a () a (2) : indeed a k. b =, (3a () a (2) ) = 3 2 = and b 2 = 5, (3a () a (2) ) 2 = 3 2 = 5. Definition Assume that a (),, a (n) are vectors in R k. They are said to be linearly dependent if there exists i n such that a (i) is a linear combination of a (j), j i, i.e., if there exists α j R, j i, so that a (i) = j i j n α j a (j). The vectors a (),, a (n) are said to be linearly independent if they are not linearly dependent. Example: We claim that the two vectors a () = (, 2), a (2) = (2, ) are linearly independent in R 2. To see it, we assume that they are linearly dependent and then show that this is not possible. If a () and a (2) are linearly independent, then either there exists α R so that a (2) = α a () 2 = α, = α 2 α = 2, α = /2 or there exists α 2 R with a () = α 2 a (2) = 2 α 2, 2 = α 2 α 2 = /2, α 2 = 2.

36 36 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS In both cases we arrive at a contraddiction and hence a (), a (2) cannot be linearly dependent. Thus by definition they are linearly independent. Examples: Decide if the following vectors are linearly dependent: () a () = (, 2), a (2) = (3, 5) in R 2 (2) a () = (, 2, 3), a (2) = (4, 5, 6) in R 3 (3) a () = (2, 4, 8), a (2) = (3, 6, 2) in R 3 (4) a () = (, 3), a (2) = ( 2, 6) in R 2. Important questions with regard to the notions of linear combination and linear independence are the following ones: How can we decide whether given vectors a (),..., a (n) in R k are linearly independent? Given vectors b, a (),..., a (n) in R k how can we decide whether b is a linear combination of a (),..., a (n) and if so, how can we find α,..., α n in R so that b = n α j a (j). Are the numbers α,..., α n uniquely determined? It turns out that these questions are closely related with each other and can be rephrased in terms of systems of linear equations: assume that b, a (),..., a (n) are vectors in R k. We view them as k matrices and write b b =. b k, a () = a (). a () k = a. a k,..., a (n) = a (n). a (n) k = Denote by A the k n matrix whose columns are given by a (),..., a (n), a... a n a ()... a (n) A =.. =... a k... a kn a () k... a (n) k Recall that b is a linear combination ov the vectors a (),..., a (n) if there exist real numbers α,..., α n so that n b = α j a (j). a n. a kn.

37 2.2. LINEAR DEPENDENCE, BASES, COORDINATES 37 Written componentwise, it means that or in matrix notation b = Aα, where b = α a () + + α n a (n) = n a jα j. b k = α a () k + + α n a (n) k = n a kjα j α α =. α n R n. Note that A R k n and hence the matrix multiplication of A by α is well defined. Hence we can express the question if b is a linear combination of a (),..., a (n) in terms of the matrix calculus as follows: b is a linear combination of a (),..., a (n) if and only if the linear system Ax = b has a solution x R n. a (),..., a (n) are linearly dependent if and only if the linear homogeneous system Ax = 0 has a solution x R n, x 0. a (),..., a (n) are linearly independent if and only if the linear homogeneous system Ax = 0 has only the trivial solution x = 0 R n. In the important case where k = n, we thus have, in view of the definition of a regular matrix, the following Theorem Assume that a (),..., a (n) are vectors in R n. Then the following two statements are equivalent: (i) a (),..., a (n) are linearly independent; (ii) the n n matrix a ()... a (n) A =.. a () n... a (n) n is regular. Example: Let b = (5, 5,, 2), a () = (,, 0, ), a (2) = (0, 2,, ), a (3) = ( 2,, 0, ). Show that b is alinear combination of a (), a (2), a (3). Solution: it suffices to study the linear system Ax = b where x = x 2 and A is the x matrix a () a (2) a (3) 0 2 A =.. = a () 4 a (2) 4 a (3) 4 x

38 38 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Consider the augmented coefficient matrix R 2 R 2 R, R 4 R 4 R R R 3 R 3 + 2R 2, R 4 R 4 R R 4 R 4 3R Therefore, x 3 = 2, x 2 =, x = 5 + 2x 3 = hence b = a () + a (2) + ( 2) a (3) = a () + a (2) 2a (3). Definition Vectors a (),..., a (n) in R k are called a basis of R k if any vector b R k can be represented in a unique way as linear combination of a (),..., a (n), i.e. if for every b R k there exist unique real numbers x,..., x n such that b = n x j a (j). The numbers x,..., x n are called the coordinates of b with respect to the basis a (),..., a (n).

39 2.2. LINEAR DEPENDENCE, BASES, COORDINATES 39 Theorem (i) Any basis of R k consists of k vectors. (ii) If the vectors a (),..., a (k) in R k are linearly independent, then they form a basis in R k. (iii) If a (),..., a (n) are vectors in R k with n < k, then there exist vectors a (n+),..., a (k) in R k so that a (),..., a (n), a (n+),..., a (k) form a basis of R k. In words, a collection of linearly independent vectors of R k can always be completed to a basis of R k. Examples: The vectors e () = (, 0,..., 0), e (2) = (0,, 0,..., 0),..., e (n) = (0,..., 0, ) form a basis of R n, referred to as the standard basis of R n. Indeed any vector b = (b,..., b n ) R n can be written uniquely as b = n b j e (j). Note that b,..., b n e (),..., e (n) of R n. are the coordinates of b with respect to the standard basis a () = (, ), a (2) = (2, ) is a basis of R 2 : according to Theorem 2.2.2, it suffices to verify that a (), a (2) are linearly independent in R 2 and according to Theorem 2.2., this is the case iff ( ) a () a (2) A = a () 2 a (2) 2 = ( ) 2 is regular. Since det(a) = 2 = 0, A is indeed regular. Find the coordinates of the vector b = (, 3) R 2 with respect to the basis a () = (, ), a (2) = (2, ) of R 2. We need to solve the linear system Ax = b, whose augmented coefficient matrix is given by ( 2 ) 3 R 2 R 2 R ( and hence x 2 = 2, x = 2x 2 = 5. The coordinates of b withe respect to a () and a (2) are thus 5 and 2, b = 5a () 2a (2), x = 5, x 2 = 2. Note that the coordinates of b with respect to the standard basis of R 2 are and 3. An important question is how the coefficients 5, 2 and, 3 are related with each other. Let us consider the latter question in a more general context. Assume that [a] := [a (),..., a (n) ] and b := [b (),..., b (n) ] )

40 40 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS are bases of R n and consider a vector u R n. Then u = n α j a (j), u = n β j b (j) where α,..., α n are the coordinates of u with respect to [a] and β,..., β n the ones of u with respect to [b]. We would like to have a method of computing β j, j n from α j, j n. To this end we have to express the vectors a (j) as linear combination of the vectors b (),..., b (n) : Then or a (j) = n t... t n t ij b (i), T :=.. t n... t nn i= n β i b (i) = u = i= n α j a (j) = n β i b (i) = i= n α j n t ij b (i) i= n ( n α j t ij )b (i). i= Since the coordinates of u with respect to the basis [b] are uniquely determined one conludes that β i = n t ijα j for any i n. In matrix notation, we thus obtain the relation α β = T α, α =. α n, β = β. It turns out that a convenient notation for T is the following one T = Id [a] [b]. β n. (2.2.) The j-th column of T is the vector of coordinates of a (j) with respect to the basis [b]. Similarly, we express b (j) as a linear combination of a (),..., a (n), n s... s n b (j) = s ij a (j), S =... i= s n... s nn We then obtain α = Sβ, S = id [b] [a]. (2.2.2) Theorem Assume that [a] and [b] are bases of R n. Then (i) S and T are regular n n matrices, hence invertible. (ii) T = S.

41 2.2. LINEAR DEPENDENCE, BASES, COORDINATES 4 Note that item (ii) can be deduced by (2.2.), (2.2.2) and item (i): indeed β = T α (2.2.2) β = T Sβ α = Sβ (2.2.) α = ST α. From item (i), it follows that T S = Id n = ST, i.e., T = S. Example: Consider the standard basis [e] = [e (), e (2), e (3) ] and the basis [b] = [b (), b (2), b (3) ] of R 3 given by b () = (,, ), b (2) = (0,, 2), b (3) = (2,, ). Let us compute S = Id [b] [e]. The first column of S is the vector of coordinates of b () with respect to the standard basis [e], and similarly, and Hence b () = e () + e (2) + e (3) b (2) = 0 e () + e (2) + 2 e (3) b (3) = 2 e () + e (2) e (3). 0 2 S = 2 With Gaussian elimination, we then compute T = Id [e] [b] = S, T = The coordinates β, β 2, β 3 of u = (, 2, 3) R 3 with respect to the basis [b], u = 3 β jb (j), can then be computed as follows: compute the coordinates of u with respect to the basis [e] : α =, α 2 = 2, α 3 = β = Id [e] [b] α = T α = = Hence u = b () + b (2) + 0 b (3). How to remember the formula? β = Id [e] [b] α { β = new coordinates, α = old coordinates [b] = new basis, [e] = old basis We will come back on this topic in Chapter 3, when we discuss the notion of a linear map.

42 42 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS 2.3 Determinants ( ) a a We already introduced the notion of the determinant of a 2 2 matrix A = 2, a 2 a 22 and established important properties: det(a) = a a 22 a 2 a 2 A is regular if and only if det(a) 0 (simple criterion to decide if a 2 2 matrix is regular or not) Cramer s rule for solving Ax = b, b R 2. Write ( ) ( ) a () a =, a (2) a2 =, b = a 2 a 22 ( b b 2 ). Then x = det(b a(2) ) det(a () a (2) ), x 2 = det(a() b) det(a () a (2) ). We now want to address the question if the notion of determinant can be extended to n n matrices so that corresponding properties hold as for 2 2 matrices. The answer is yes! Among the many equivalent ways of defining the determinant of n n matrices, we choose a recursive definition, which defines the determinant of a n n matrix in terms of determinants of certain (n ) (n ) matrices. First we need to introduce some more notations. For A R n n and i, j n, we denote by A (i,j) R (n ) (n ) the (n ) (n ) matrix, obtained from A by deleting the i-th row and the j-th column. Example: For 2 3 A = R one has ( ) ( ) ( ) A (,) =, A (,3) =, A (,2) = ( ) ( ) ( ) A (2,) =, A (2,2) =, A (2,3) = To motivate the inductive definition of the determinant of a n n matrix, we first consider the cases n =, n = 2. One has n = : A = (a ) R det(a) = a n = 2 : ( ) a a A = 2 a 2 a 22 det(a) = a a 22 a 2 a 2

43 2.3. DETERMINANTS 43 which can be written as det(a) = a det(a (,) ) a 2 det(a (,2) ) = ( ) + a det(a (,) ) + ( ) +2 a 2 det(a (,2) ) 2 = ( ) +j a j det(a (,j) ). Definition For any A R n n with n 3, det(a) = n ( ) +j a j det(a (,j) ). (2.3.) Since A (,j) is a (n ) (n ) matrix for any j n, this is indeed a recursive definition. We refer to (2.3.) as the expansion of det(a) with respect to the first row. Example: Then 2 3 A = 4 2 R det(a) = ( ) + det(a (,) ) + ( ) +2 2 det(a (,2) ) + ( ) +3 3 det(a (,3) ). Since one gets A (,) = ( ) 2, A (,2) = 0 ( ) 4, A (,3) = ( ) 4 2, 0 det(a) = (2 0) 2(4 ) + 3(4 0 2 ) = = 0. Let us state some elementary important properties of the determinant. Theorem For any A R n n and k n the following holds: (i) n det(a) = ( ) k+j a kj det(a (k,j) ) (expansion of det(a) with respect to the k-th row) (ii) n det(a) = ( ) j+k a jk det(a (j,k) ) (expansion of det(a) with respect to the k-th column) (iii) det(a) = det(a T ).

44 44 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Note that item (iii) of the latter theorem follows from item (ii), since (A T ) j = a j and (A T ) (,j) = A (j,). To state the next theorem let us introduce some more notations. For A R n n, denote by a (),..., a (n) its columns and by A (),..., A (n) its rows. We then have A = (a () a (n) ), A = A (). A (n). Theorem For any A R n n, the following identities hold: (i) for any j < i n, and det(a () a (j) a (i) a (n) ) = det(a () a (i) a (j) a (n) ) (ii) For any λ R and i n and for any b R n A () A ().. A (j) A (i) det. = det.. A (i) A (j).. A (n) A (n) det(a () λa (i) a (n) ) = λ det(a () a (i) a (n) ) det(a () (a (i) + b) a (n) ) = det(a () a (i) a (n) ) + det(a () b a (n) ) and an analogous statement holds for the rows of A. (iii) for any i, j n and i j, λ R, det(a () (a (i) + λa (j) ) a (n) ) = det(a () a i a (n) ). An analogous statement is valid for the rows of A. (iv) If A is upper triangular, namely a ij = 0 i > j, then det(a) = a a 22 a nn = n a jj. Expressed in words, item (ii) says that det(a) is linear with respect to its i-th column, i n (cf Chapter 3 for the notion of linear maps). Similarly, det(a) is linear with respect to its i-th row, i n.

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

More information

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

More information

NON SINGULAR MATRICES. DEFINITION. (Non singular matrix) An n n A is called non singular or invertible if there exists an n n matrix B such that

NON SINGULAR MATRICES. DEFINITION. (Non singular matrix) An n n A is called non singular or invertible if there exists an n n matrix B such that NON SINGULAR MATRICES DEFINITION. (Non singular matrix) An n n A is called non singular or invertible if there exists an n n matrix B such that AB = I n = BA. Any matrix B with the above property is called

More information

A matrix over a field F is a rectangular array of elements from F. The symbol

A matrix over a field F is a rectangular array of elements from F. The symbol Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F) denotes the collection of all m n matrices over F Matrices will usually be denoted

More information

Topic 1: Matrices and Systems of Linear Equations.

Topic 1: Matrices and Systems of Linear Equations. Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method

More information

Matrix Inverse and Determinants

Matrix Inverse and Determinants DM554 Linear and Integer Programming Lecture 5 and Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1 2 3 4 and Cramer s rule 2 Outline 1 2 3 4 and

More information

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

More information

(a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.

(a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product

More information

UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

More information

Notes on Determinant

Notes on Determinant ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

More information

1.5 Elementary Matrices and a Method for Finding the Inverse

1.5 Elementary Matrices and a Method for Finding the Inverse .5 Elementary Matrices and a Method for Finding the Inverse Definition A n n matrix is called an elementary matrix if it can be obtained from I n by performing a single elementary row operation Reminder:

More information

Determinants. Dr. Doreen De Leon Math 152, Fall 2015

Determinants. Dr. Doreen De Leon Math 152, Fall 2015 Determinants Dr. Doreen De Leon Math 52, Fall 205 Determinant of a Matrix Elementary Matrices We will first discuss matrices that can be used to produce an elementary row operation on a given matrix A.

More information

Matrix Algebra LECTURE 1. Simultaneous Equations Consider a system of m linear equations in n unknowns: y 1 = a 11 x 1 + a 12 x 2 + +a 1n x n,

Matrix Algebra LECTURE 1. Simultaneous Equations Consider a system of m linear equations in n unknowns: y 1 = a 11 x 1 + a 12 x 2 + +a 1n x n, LECTURE 1 Matrix Algebra Simultaneous Equations Consider a system of m linear equations in n unknowns: y 1 a 11 x 1 + a 12 x 2 + +a 1n x n, (1) y 2 a 21 x 1 + a 22 x 2 + +a 2n x n, y m a m1 x 1 +a m2 x

More information

MATH36001 Background Material 2015

MATH36001 Background Material 2015 MATH3600 Background Material 205 Matrix Algebra Matrices and Vectors An ordered array of mn elements a ij (i =,, m; j =,, n) written in the form a a 2 a n A = a 2 a 22 a 2n a m a m2 a mn is said to be

More information

Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants

Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants Hartmut Führ fuehr@matha.rwth-aachen.de Lehrstuhl A für Mathematik, RWTH Aachen October 30, 2008 Overview

More information

Diagonal, Symmetric and Triangular Matrices

Diagonal, Symmetric and Triangular Matrices Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

More information

DETERMINANTS. b 2. x 2

DETERMINANTS. b 2. x 2 DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

More information

MATH 240 Fall, Chapter 1: Linear Equations and Matrices

MATH 240 Fall, Chapter 1: Linear Equations and Matrices MATH 240 Fall, 2007 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 9th Ed. written by Prof. J. Beachy Sections 1.1 1.5, 2.1 2.3, 4.2 4.9, 3.1 3.5, 5.3 5.5, 6.1 6.3, 6.5, 7.1 7.3 DEFINITIONS

More information

( % . This matrix consists of $ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&

( % . This matrix consists of $ 4 5  5' the coefficients of the variables as they appear in the original system. The augmented 3  2 2 # 2  3 4& Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important

More information

Inverses and powers: Rules of Matrix Arithmetic

Inverses and powers: Rules of Matrix Arithmetic Contents 1 Inverses and powers: Rules of Matrix Arithmetic 1.1 What about division of matrices? 1.2 Properties of the Inverse of a Matrix 1.2.1 Theorem (Uniqueness of Inverse) 1.2.2 Inverse Test 1.2.3

More information

1 Determinants and the Solvability of Linear Systems

1 Determinants and the Solvability of Linear Systems 1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely side-stepped

More information

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix.

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. Matrices Definition. An m-by-n matrix is a rectangular array of numbers that has m rows and n columns: a 11

More information

The Characteristic Polynomial

The Characteristic Polynomial Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

More information

Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws.

Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Matrix Algebra A. Doerr Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Some Basic Matrix Laws Assume the orders of the matrices are such that

More information

Mathematics Notes for Class 12 chapter 3. Matrices

Mathematics Notes for Class 12 chapter 3. Matrices 1 P a g e Mathematics Notes for Class 12 chapter 3. Matrices A matrix is a rectangular arrangement of numbers (real or complex) which may be represented as matrix is enclosed by [ ] or ( ) or Compact form

More information

Chapter 8. Matrices II: inverses. 8.1 What is an inverse?

Chapter 8. Matrices II: inverses. 8.1 What is an inverse? Chapter 8 Matrices II: inverses We have learnt how to add subtract and multiply matrices but we have not defined division. The reason is that in general it cannot always be defined. In this chapter, we

More information

LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA. September 23, 2010 LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

More information

Math 315: Linear Algebra Solutions to Midterm Exam I

Math 315: Linear Algebra Solutions to Midterm Exam I Math 35: Linear Algebra s to Midterm Exam I # Consider the following two systems of linear equations (I) ax + by = k cx + dy = l (II) ax + by = 0 cx + dy = 0 (a) Prove: If x = x, y = y and x = x 2, y =

More information

1 Eigenvalues and Eigenvectors

1 Eigenvalues and Eigenvectors Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

More information

Solving Systems of Linear Equations

Solving Systems of Linear Equations LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

More information

2.5 Gaussian Elimination

2.5 Gaussian Elimination page 150 150 CHAPTER 2 Matrices and Systems of Linear Equations 37 10 the linear algebra package of Maple, the three elementary 20 23 1 row operations are 12 1 swaprow(a,i,j): permute rows i and j 3 3

More information

Chapter 1 - Matrices & Determinants

Chapter 1 - Matrices & Determinants Chapter 1 - Matrices & Determinants Arthur Cayley (August 16, 1821 - January 26, 1895) was a British Mathematician and Founder of the Modern British School of Pure Mathematics. As a child, Cayley enjoyed

More information

EC9A0: Pre-sessional Advanced Mathematics Course

EC9A0: Pre-sessional Advanced Mathematics Course University of Warwick, EC9A0: Pre-sessional Advanced Mathematics Course Peter J. Hammond & Pablo F. Beker 1 of 55 EC9A0: Pre-sessional Advanced Mathematics Course Slides 1: Matrix Algebra Peter J. Hammond

More information

NOTES on LINEAR ALGEBRA 1

NOTES on LINEAR ALGEBRA 1 School of Economics, Management and Statistics University of Bologna Academic Year 205/6 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

More information

1 Determinants. Definition 1

1 Determinants. Definition 1 Determinants The determinant of a square matrix is a value in R assigned to the matrix, it characterizes matrices which are invertible (det 0) and is related to the volume of a parallelpiped described

More information

INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL

INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL SOLUTIONS OF THEORETICAL EXERCISES selected from INTRODUCTORY LINEAR ALGEBRA WITH APPLICATIONS B. KOLMAN, D. R. HILL Eighth Edition, Prentice Hall, 2005. Dr. Grigore CĂLUGĂREANU Department of Mathematics

More information

Direct Methods for Solving Linear Systems. Linear Systems of Equations

Direct Methods for Solving Linear Systems. Linear Systems of Equations Direct Methods for Solving Linear Systems Linear Systems of Equations Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University

More information

1 Gaussian Elimination

1 Gaussian Elimination Contents 1 Gaussian Elimination 1.1 Elementary Row Operations 1.2 Some matrices whose associated system of equations are easy to solve 1.3 Gaussian Elimination 1.4 Gauss-Jordan reduction and the Reduced

More information

Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

More information

4. MATRICES Matrices

4. MATRICES Matrices 4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

More information

2.1: Determinants by Cofactor Expansion. Math 214 Chapter 2 Notes and Homework. Evaluate a Determinant by Expanding by Cofactors

2.1: Determinants by Cofactor Expansion. Math 214 Chapter 2 Notes and Homework. Evaluate a Determinant by Expanding by Cofactors 2.1: Determinants by Cofactor Expansion Math 214 Chapter 2 Notes and Homework Determinants The minor M ij of the entry a ij is the determinant of the submatrix obtained from deleting the i th row and the

More information

Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

More information

1 Introduction to Matrices

1 Introduction to Matrices 1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

More information

a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2. Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

More information

Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M 1 such that

Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M 1 such that 0. Inverse Matrix Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M such that M M = I = M M. Inverse of a 2 2 Matrix Let M and N be the matrices: a b d b M =, N = c

More information

The Solution of Linear Simultaneous Equations

The Solution of Linear Simultaneous Equations Appendix A The Solution of Linear Simultaneous Equations Circuit analysis frequently involves the solution of linear simultaneous equations. Our purpose here is to review the use of determinants to solve

More information

Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold:

Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold: Linear Systems Example: Find x, x, x such that the following three equations hold: x + x + x = 4x + x + x = x + x + x = 6 We can write this using matrix-vector notation as 4 {{ A x x x {{ x = 6 {{ b General

More information

Summary of week 8 (Lectures 22, 23 and 24)

Summary of week 8 (Lectures 22, 23 and 24) WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry

More information

5.3 Determinants and Cramer s Rule

5.3 Determinants and Cramer s Rule 290 5.3 Determinants and Cramer s Rule Unique Solution of a 2 2 System The 2 2 system (1) ax + by = e, cx + dy = f, has a unique solution provided = ad bc is nonzero, in which case the solution is given

More information

Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20

Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20 Matrix algebra January 20 Introduction Basics The mathematics of multiple regression revolves around ordering and keeping track of large arrays of numbers and solving systems of equations The mathematical

More information

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

More information

= [a ij ] 2 3. Square matrix A square matrix is one that has equal number of rows and columns, that is n = m. Some examples of square matrices are

= [a ij ] 2 3. Square matrix A square matrix is one that has equal number of rows and columns, that is n = m. Some examples of square matrices are This document deals with the fundamentals of matrix algebra and is adapted from B.C. Kuo, Linear Networks and Systems, McGraw Hill, 1967. It is presented here for educational purposes. 1 Introduction In

More information

13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions.

13 MATH FACTS 101. 2 a = 1. 7. The elements of a vector have a graphical interpretation, which is particularly easy to see in two or three dimensions. 3 MATH FACTS 0 3 MATH FACTS 3. Vectors 3.. Definition We use the overhead arrow to denote a column vector, i.e., a linear segment with a direction. For example, in three-space, we write a vector in terms

More information

Lecture 6. Inverse of Matrix

Lecture 6. Inverse of Matrix Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

More information

Introduction to Matrix Algebra I

Introduction to Matrix Algebra I Appendix A Introduction to Matrix Algebra I Today we will begin the course with a discussion of matrix algebra. Why are we studying this? We will use matrix algebra to derive the linear regression model

More information

Practice Math 110 Final. Instructions: Work all of problems 1 through 5, and work any 5 of problems 10 through 16.

Practice Math 110 Final. Instructions: Work all of problems 1 through 5, and work any 5 of problems 10 through 16. Practice Math 110 Final Instructions: Work all of problems 1 through 5, and work any 5 of problems 10 through 16. 1. Let A = 3 1 1 3 3 2. 6 6 5 a. Use Gauss elimination to reduce A to an upper triangular

More information

2.1: MATRIX OPERATIONS

2.1: MATRIX OPERATIONS .: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

More information

Systems of Linear Equations

Systems of Linear Equations Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

More information

Solving Linear Systems, Continued and The Inverse of a Matrix

Solving Linear Systems, Continued and The Inverse of a Matrix , Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

More information

Matrices, Determinants and Linear Systems

Matrices, Determinants and Linear Systems September 21, 2014 Matrices A matrix A m n is an array of numbers in rows and columns a 11 a 12 a 1n r 1 a 21 a 22 a 2n r 2....... a m1 a m2 a mn r m c 1 c 2 c n We say that the dimension of A is m n (we

More information

Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively. Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

More information

MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1. MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

More information

MAT188H1S Lec0101 Burbulla

MAT188H1S Lec0101 Burbulla Winter 206 Linear Transformations A linear transformation T : R m R n is a function that takes vectors in R m to vectors in R n such that and T (u + v) T (u) + T (v) T (k v) k T (v), for all vectors u

More information

Name: Section Registered In:

Name: Section Registered In: Name: Section Registered In: Math 125 Exam 3 Version 1 April 24, 2006 60 total points possible 1. (5pts) Use Cramer s Rule to solve 3x + 4y = 30 x 2y = 8. Be sure to show enough detail that shows you are

More information

Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible:

Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible: Cramer s Rule and the Adjugate Using determinants, it is possible to express the solution to a system of equations whose coefficient matrix is invertible: Theorem [Cramer s Rule] If A is an invertible

More information

1 Sets and Set Notation.

1 Sets and Set Notation. LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

More information

Vector Spaces II: Finite Dimensional Linear Algebra 1

Vector Spaces II: Finite Dimensional Linear Algebra 1 John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition

More information

Sergei Silvestrov, Christopher Engström, Karl Lundengård, Johan Richter, Jonas Österberg. November 13, 2014

Sergei Silvestrov, Christopher Engström, Karl Lundengård, Johan Richter, Jonas Österberg. November 13, 2014 Sergei Silvestrov,, Karl Lundengård, Johan Richter, Jonas Österberg November 13, 2014 Analysis Todays lecture: Course overview. Repetition of matrices elementary operations. Repetition of solvability of

More information

1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form 1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

More information

Math 312 Homework 1 Solutions

Math 312 Homework 1 Solutions Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

More information

9 Matrices, determinants, inverse matrix, Cramer s Rule

9 Matrices, determinants, inverse matrix, Cramer s Rule AAC - Business Mathematics I Lecture #9, December 15, 2007 Katarína Kálovcová 9 Matrices, determinants, inverse matrix, Cramer s Rule Basic properties of matrices: Example: Addition properties: Associative:

More information

Facts About Eigenvalues

Facts About Eigenvalues Facts About Eigenvalues By Dr David Butler Definitions Suppose A is an n n matrix An eigenvalue of A is a number λ such that Av = λv for some nonzero vector v An eigenvector of A is a nonzero vector v

More information

2.5 Elementary Row Operations and the Determinant

2.5 Elementary Row Operations and the Determinant 2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)

More information

Solutions to Review Problems

Solutions to Review Problems Chapter 1 Solutions to Review Problems Chapter 1 Exercise 42 Which of the following equations are not linear and why: (a x 2 1 + 3x 2 2x 3 = 5. (b x 1 + x 1 x 2 + 2x 3 = 1. (c x 1 + 2 x 2 + x 3 = 5. (a

More information

Homework: 2.1 (page 56): 7, 9, 13, 15, 17, 25, 27, 35, 37, 41, 46, 49, 67

Homework: 2.1 (page 56): 7, 9, 13, 15, 17, 25, 27, 35, 37, 41, 46, 49, 67 Chapter Matrices Operations with Matrices Homework: (page 56):, 9, 3, 5,, 5,, 35, 3, 4, 46, 49, 6 Main points in this section: We define a few concept regarding matrices This would include addition of

More information

APPLICATIONS OF MATRICES. Adj A is nothing but the transpose of the co-factor matrix [A ij ] of A.

APPLICATIONS OF MATRICES. Adj A is nothing but the transpose of the co-factor matrix [A ij ] of A. APPLICATIONS OF MATRICES ADJOINT: Let A = [a ij ] be a square matrix of order n. Let Aij be the co-factor of a ij. Then the n th order matrix [A ij ] T is called the adjoint of A. It is denoted by adj

More information

Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i. Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

More information

Factorization Theorems

Factorization Theorems Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

More information

On the general equation of the second degree

On the general equation of the second degree On the general equation of the second degree S Kesavan The Institute of Mathematical Sciences, CIT Campus, Taramani, Chennai - 600 113 e-mail:kesh@imscresin Abstract We give a unified treatment of the

More information

Section 6.1 - Inner Products and Norms

Section 6.1 - Inner Products and Norms Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,

More information

Using row reduction to calculate the inverse and the determinant of a square matrix

Using row reduction to calculate the inverse and the determinant of a square matrix Using row reduction to calculate the inverse and the determinant of a square matrix Notes for MATH 0290 Honors by Prof. Anna Vainchtein 1 Inverse of a square matrix An n n square matrix A is called invertible

More information

by the matrix A results in a vector which is a reflection of the given

by the matrix A results in a vector which is a reflection of the given Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

More information

1 Spherical Kinematics

1 Spherical Kinematics ME 115(a): Notes on Rotations 1 Spherical Kinematics Motions of a 3-dimensional rigid body where one point of the body remains fixed are termed spherical motions. A spherical displacement is a rigid body

More information

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

More information

Solution of Linear Systems

Solution of Linear Systems Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

More information

Inverses. Stephen Boyd. EE103 Stanford University. October 27, 2015

Inverses. Stephen Boyd. EE103 Stanford University. October 27, 2015 Inverses Stephen Boyd EE103 Stanford University October 27, 2015 Outline Left and right inverses Inverse Solving linear equations Examples Pseudo-inverse Left and right inverses 2 Left inverses a number

More information

Using the Singular Value Decomposition

Using the Singular Value Decomposition Using the Singular Value Decomposition Emmett J. Ientilucci Chester F. Carlson Center for Imaging Science Rochester Institute of Technology emmett@cis.rit.edu May 9, 003 Abstract This report introduces

More information

MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.

MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar

More information

Recall that two vectors in are perpendicular or orthogonal provided that their dot

Recall that two vectors in are perpendicular or orthogonal provided that their dot Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

More information

Determinants. Chapter Properties of the Determinant

Determinants. Chapter Properties of the Determinant Chapter 4 Determinants Chapter 3 entailed a discussion of linear transformations and how to identify them with matrices. When we study a particular linear transformation we would like its matrix representation

More information

Lecture 11. Shuanglin Shao. October 2nd and 7th, 2013

Lecture 11. Shuanglin Shao. October 2nd and 7th, 2013 Lecture 11 Shuanglin Shao October 2nd and 7th, 2013 Matrix determinants: addition. Determinants: multiplication. Adjoint of a matrix. Cramer s rule to solve a linear system. Recall that from the previous

More information

Economics 102C: Advanced Topics in Econometrics 2 - Tooling Up: The Basics

Economics 102C: Advanced Topics in Econometrics 2 - Tooling Up: The Basics Economics 102C: Advanced Topics in Econometrics 2 - Tooling Up: The Basics Michael Best Spring 2015 Outline The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices The Evaluation Problem:

More information

8 Square matrices continued: Determinants

8 Square matrices continued: Determinants 8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You

More information

Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6

Chapter 7. Matrices. Definition. An m n matrix is an array of numbers set out in m rows and n columns. Examples. ( 1 1 5 2 0 6 Chapter 7 Matrices Definition An m n matrix is an array of numbers set out in m rows and n columns Examples (i ( 1 1 5 2 0 6 has 2 rows and 3 columns and so it is a 2 3 matrix (ii 1 0 7 1 2 3 3 1 is a

More information

MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0,

MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0, MATH 23: SYSTEMS OF LINEAR EQUATIONS Systems of Linear Equations In the plane R 2 the general form of the equation of a line is ax + by = c and that the general equation of a plane in R 3 will be we call

More information

PROVING STATEMENTS IN LINEAR ALGEBRA

PROVING STATEMENTS IN LINEAR ALGEBRA Mathematics V2010y Linear Algebra Spring 2007 PROVING STATEMENTS IN LINEAR ALGEBRA Linear algebra is different from calculus: you cannot understand it properly without some simple proofs. Knowing statements

More information

Matrix Algebra and Applications

Matrix Algebra and Applications Matrix Algebra and Applications Dudley Cooke Trinity College Dublin Dudley Cooke (Trinity College Dublin) Matrix Algebra and Applications 1 / 49 EC2040 Topic 2 - Matrices and Matrix Algebra Reading 1 Chapters

More information

Similarity and Diagonalization. Similar Matrices

Similarity and Diagonalization. Similar Matrices MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

More information

Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems. Matrix Factorization Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

More information

MATH 551 - APPLIED MATRIX THEORY

MATH 551 - APPLIED MATRIX THEORY MATH 55 - APPLIED MATRIX THEORY FINAL TEST: SAMPLE with SOLUTIONS (25 points NAME: PROBLEM (3 points A web of 5 pages is described by a directed graph whose matrix is given by A Do the following ( points

More information