Linear Algebra for the Sciences. Thomas Kappeler, Riccardo Montalto

Transcription

1 Linear Algebra for the Sciences Thomas Kappeler, Riccardo Montalto

2 2

3 Contents Systems of linear equations 5. Linear systems with two equations and two unknowns Gaussian elimination Matrix calculus and related topics Matrix calculus Linear dependence, bases, coordinates Determinants Complex numbers and complex systems of linear equations Complex numbers and their calculus The fundamental theorem of algebra Systems of linear equations with complex coefficients Vector spaces and linear maps 6 4. Vector spaces and their linear subspaces Linear maps Inner products on R vector spaces Isometries and orthogonal matrices Vector product in R Inner products on C vector spaces Eigenvalues and eigenvectors Eigenvalues and eigenvectors of C linear maps on C vector spaces Eigenvalues and eigenvectors of R-linear maps on R-vector spaces Quadratic forms on R n Differential equations Introduction Systems of linear ODEs of first order with constant coefficients Linear ODEs of higher order with constant coefficients

4 4 CONTENTS

5 Chapter Systems of linear equations At the core of linear algebra is the solving of systems of linear equations. A linear system (S) with n unknowns and m equations has the form a x + a 2 x a n x n = b (Eq ) a 2 x + a 22 x a 2n x n = b 2 (Eq 2 ). a m x + a m2 x a mn x n = b m (Eq m ) The numbers a,..., a n, a 2,..., a 2n,..., a m,..., a mn are called the coefficients of the system. The system is said to be real if all coefficients a ij, i =,..., n, j =,..., m as well as the numbers b,..., b m are real. In this chapter we will only consider real systems of linear equations, but the techniques we develop to solve them also apply to complex systems, i.e., systems where a ij and b i are complex numbers which will only be introduced in a later chapter. Given the real system (S), a solution of (S) is a set of n real numbers x,..., x n so that equations (Eq ) (Eq m ) are satisfied simultaneously. The basic questions with regard to linear systems of the form (S) are the following ones: () Does (S) have a solution? (Existence) (2) Does (S) have at most solution? (Uniqueness) Or formulated in more general terms: (3) What are the properties of the set of solutions { } L := (x,..., x n ) : x R,..., x n R; (x,..., x n ) satisfies (S ) (S m ). Here and in the sequel, R denotes the set of real numbers. Note that in this terminology questions () and (2) can be reformulated as follows: ( ) Is L a nonempty set? (2 ) Does L have at most one element? In practical applications, systems of linear equations can be very large. One therefore needs theoretical concepts and numerical algorithms to investigate respectively solve such systems. 5

6 6 CHAPTER. SYSTEMS OF LINEAR EQUATIONS. Linear systems with two equations and two unknowns As an introduction we consider in this section real systems with two equations and two unknowns: { a x + a 2 x 2 = b a 2 x + a 22 x 2 = b 2. In such a case, we usually choose a simpler notation: x x, x 2 y { ax + by = e cx + dy = f. where a, b, c, d, e, f are in R. To get some experience, let us first treat the simpler case of one linear equation with one unknown ax = b (..) where a, b R. The solvability of this equation depends on the values of a and b: Case a 0: equation () has precisely one solution x = b a R or L = { b a Case a = 0 and b 0: equation () has no solution, i.e., L =. Case a = 0 and b = 0: any real number x R is a solution of (), i.e., L = R. Next let us treat the case of one equation and two unknowns: }. ax + by = c, a, b, c R (..2) As above we denote the set of solutions by L, { } L = (x, y) R 2 : ax + by = c. Case a = 0, b = 0, c = 0: L = R 2 Case a = 0, b = 0, c 0: L = Case (a, b) (0, 0): the set L is a straight line in R 2. In case b 0, this straight line is given by y = a b x + c b and In case b = 0 and a 0, L = {( x, a b x + c ) } : x R. b {( c ) L = a, y } : y R.

7 .. LINEAR SYSTEMS WITH TWO EQUATIONS AND TWO UNKNOWNS 7 Now let us go back to the linear system of two equations with two unknowns, ax + by = e (..3) cx + dy = f (..4) The set of solutions L of (..3), (..4) is then given by the intersection of the set of solutions of (..3) with the set of solutions of (..3), L = L L 2, where { } { } L = (x, y) R 2 : ax + by = e, L 2 = (x, y) R 2 : cx + dy = f. In case (a, b) (0, 0) and (c, d) (0, 0), L and L 2 are lines in R 2 and L is the intersection of them. Thus L can be a one point set (L and L 2 are not parallel), or a line (L = L 2 ) or the emptyset (L and L 2 are parallel but do not coincide). Let us now describe an algorithm how to determine the set of solutions of (..3), (..4) in a systematic way. You know this algorithm already from high school. To simplify the algorithm we assume that a 0. (..5) Step : eliminate x from equation (..4) by replacing (..4) by (..4) c (..3) It a means that the left hand side of (..4) is replaced by cx + dy c ( ) ax + by (use that a 0!) a whereas the right hand side of (..4) is replaced by f c a e. The new system of equations then reads as follows ax + by = e (..6) (d c ) a b y = f c a e. (..7) It is straightforward to see that under the assumption (..5), the set solutions of (..3), (..4) coincides with the set of solutions of (..6), (..7). In such a case we say that the two systems are equivalent. Step 2: The system (..6), (..7) is solved by first solving (..7) for y and then use (..6) to determine x: Case d c b 0: then (..7) has the unique solution a y = f c e a af ce d c = b ad bc a and when substituted into (..6) one obtains ( af ce ) ax = e b ad bc

8 8 CHAPTER. SYSTEMS OF LINEAR EQUATIONS or x = de bf ad bc. Hence the set of solutions L consists of one element L = {( de bf af ce )}, ad bc ad bc (..8) Case d c b = 0, f c e 0: equation (..7) has no solutions and hence L =. a a Case d c b = 0, f c e = 0: then any y R is a solution of (..7) and solutions of a a (..6) are given by x = e b y. Hence the set of solutions L is given by a a {( e L = a b ) a y, y } : y R. Motivated by formula (..8) for the solutions of (..6), (..7) in the case where d c a b 0 we make the following definitions: Definition... (i) A real 2 2 matrix (plural: matrices) is an array A of real numbers of the form ( ) a b A :=, a, b, c, d R ; c d (ii) the determinant of a 2 2 matrix A is defined as det(a) := ad bc. The notion of the determinant can be used to characterize the solvability of the system (..6), (..7) and to obtain formulas for its solutions. We state without proof the following Theorem... (i) The system of linear equations (..6), (..7) has a unique solution if and only if ( ) a b det 0. c d ( ) a b (ii) If det 0, then the unique solution of (..6), (..7) is given by the following c d formulas (Cramer s rule) ( ) ( ) e b a e det det f d c f x = ( ), y = ( ) a b a b det det c d c d

9 .2. GAUSSIAN ELIMINATION 9 Problem: Analyze the following system of linear equations { 2y + 4x = 3 x + y = 5 and find its set of solutions. Answer: Let A denote the coefficient matrix, ( ) 4 2 A =. Then det(a) = 6 0. Hence according to Theorem.., the system of equations has a unique solution given by ( ) ( ) det 5 x = = 7 det 6 6, y = 5 = Gaussian elimination Gaussian elimination is an algorithm to determine the set of solutions of an arbitrary system of linear equations with m equations and n unknowns, denoted by x,..., x n, a x + a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2. a m x + a m2 x a mn x n = b m where a ij R, i m (index for the equations) and j n (index for the unknowns). A compact way of writing the above system of equations is achieved using the symbol for the sum, n a ij x j = b i for any i m. (.2.) Of course we could also use different letters than i and j. We denote by L the set of solutions of (.2.), L := { (x,..., x n ) R n : n } a ij x j = b i for any i m. Note that L = m L i, L i := i= { (x,..., x n ) R n : n a ij x j = b i }.

10 0 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Definition.2.. We say that two systems of linear equations and n a ij x j = b i for any i m n c kj x j = d k for any k p are equivalent if their sets of solutions coincide. and An example of two equivalent systems with three unknowns is the following one: { 4x + 3x 2 + 2x 3 = x + x 2 + x 3 = 4 4x + 3x 2 + 2x 3 = x + x 2 + x 3 = 4 2x + 2x 2 + 2x 3 = 8 since the latter equation is obtained from the equation x + x 2 + x 3 = 4 by multiplying left and right hand side by the factor 2. The idea of Gaussian elimination is to replace a given system of linear equations in a systematic way by an equivalent one which is easy to solve. In Subsection. we have demonstrated this method in the case of two equations (m = 2) and two unknowns (n = 2). Gaussian elimination uses the following basic operations, referred to as row operations, which leave the set of solutions of a given system of linear equations invariant: (R) Exchange of two equations (rows) of a system of linear equations. Example: { 5x 2 + 5x 3 = 0 4x + 3x 2 + x 3 = { 4x + 3x 2 + x 3 = 5x 2 + 5x 3 = 0 It means that the equations get listed in a different order. (R2) Multiplication of an equation (row) by a real number α 0. Example: { 4x + 3x 2 + x 3 = 5x 2 + 5x 3 = 0 { 4x + 3x 2 + x 3 = x 2 + 3x 3 = 2 We have multiplied the left and right hand side of the second equation by the factor /5.

11 .2. GAUSSIAN ELIMINATION (R3) An equation (row) gets replaced by the equation obtained by adding to it the multiple of another equation. More formally, this can be expressed as follows: the kth equation n a kjx j = b k is replaced by the equation n n a kj x j + α a lj x j = b k + αb l for some where l m with l k or more explicitly, (a k + αa l )x + + (a kn + αa ln )x n = b k + αb l. Example: { x + x 2 = 5 4x + 2x 2 = 3 (Eq) (Eq2) (Eq2) (Eq2) 4(Eq) { x + x 2 = 5 2x 2 = 7 It is not difficult to verify that these basic row operations lead to equivalent linear systems. We state without proof the following Theorem.2.. The basic row operations lead to equivalent systems of linear equations. We now show how these basic row operations can be used to solve a system (S) of linear equations of the form a x + a 2 x a n x n = b a 2 x + a 22 x a 2n x n = b 2. a m x + a m2 x a mn x n = b m where a ij ( i m, j n) and b i ( i m) are real numbers. To (S) we associate its coefficient matrix a a n A =... a m a mn A is an array of real numbers with m rows and n columns. Such an array of real numbers is called an m n matrix and written in a compact form as A = (a ij ) i m j n The augmented coefficient matrix of (S) is the following array of real numbers a a n b... a m a mn b m written in a compact form as (A b).

12 2 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Definition.2.2. (A b) is said to be in row echelon form (Zeilenstufenform) if it is of the form b where stands for nonzero elements of A. Note that below the echelon (Stufe), all the coefficients of A vanish. We point out that it is possible that the coefficient matrix A does not have any zero columns (Spalten). Furthermore, if it has zero rows (Zeilen), then these rows have to be at the bottom of A. Examples of augmented coefficient matrices in row echelon form are ( 0 ) 0 0 0, ( 0 ) ( ) 4 0 0, whereas the following ones are not in this form ( ) ( ) 0 0,, ( 0 0 ) ( ) , ( 0 0 ) 0 If the augmented coefficient matrix of a linear system is in row echelon form, it can easily be solved. To illustrate this let us look at few examples. Examples: () { 2x + x 2 = 2 3x 2 = 6 ( 2 ) is in row echelon form. Solve it by first determining x 2 by the second equation and then solving the first equation for x by substituting the obtained value of x 2 3x 2 = 6 x 2 = 2 2x = 2 x 2 2x = 0 x = 0 hence the set of solutions is given by L = {(0, 2)}.

13 .2. GAUSSIAN ELIMINATION 3 (2) { 2x + x 2 + x 3 = 2 3x 3 = 6 ( 2 ) is in row echelon form. Solve the second equation for x 3 and then the first equation for x : 3x 3 = 6 x 3 = 2 ; x 2 is a free variable; 2x = 2 x 2 x 3 x = 2 x 2 and hence L = { ( } 2 x 2, x 2, 2) : x 2 R, which is a straight line in R 3 trough (0, 0, 2) in direction (, 2, 0). 2 (3) The augmented coefficient matrix is in row echelon form. The corresponding system of linear equations can be solved as follows: since 0 x + 0 x 2 = 3 has no solutions, one concludes that L =. 2 (4) The augmented coefficient matrix is in row echelon form. The corresponding system of linear equations can be solved as follows: is satisfied for any x, x 2 R; Hence L = {(, 2)}. (5) The augmented coefficient matrix 0 x + 0 x 2 = 0 3x 2 = 6 x 2 = 2 2x = x 2 x = 2. ( ) 0 2 is in row echelon form x 3 = 6, x 2 = 2x 3 =, x free variable, hence L = { } (x,, 6) : x R which is a straight line in R 3 trough the point (0,, 6) in direction (, 0, 0).

14 4 CHAPTER. SYSTEMS OF LINEAR EQUATIONS ( ) (6) The augmented coefficient matrix is in row echelon form hence 3x 4 = 6 x 4 = 2 ; x 3 and x 4 are a free variables ; L = x = 2x 2 x 4 = 2x 2 2, { } ( 2x 2 2, x 2, x 3, 2) : x 2, x 3 R which is a plane in R 4 containing the point ( 2, 0, 0, 2) and spanned by the vectors (0, 0,, 0) and ( 2,, 0, 0). ( ) (7) The augmented coefficient matrix is in row echelon form Hence x 4 is a free variable ; 3x 3 = 6 x 3 = 2 L = x = 2x 2 x 3 = 2x 2 2. { } ( 2x 2 2, x 2, 2, x 4 ) : x 2, x 4 R which is a plane in R 4 containing the point ( 2, 0, 2, 0) and spanned by the vectors (0, 0, 0, ) and ( 2,, 0, 0). Gaussian elimination is a method of transforming a given augmented coefficient matrix with the help of basic row operations into row echelon form. How can this be achieved? We explain the procedure with a few examples. It is convenient to introduce for the three basic row operations the following notations: R i k : exchange rows i and k R k αr k : replace k-th row R k by αr k, α 0. R k R k + αr l : replace k-th row by adding to it αr l where l k and α R. Examples ( ) () The augmented coefficient matrix is not in row echelon form. Apply 2 2 R 2 to get ( ) ( ) 2 (2) The augmented coefficient matrix is not in row echelon form. The first row is ok; in the second row we have to eliminate 4; hence R 2 R 2 2R yielding ( ) 2 0 2

15 .2. GAUSSIAN ELIMINATION 5 (3) The augmented coefficient matrix 2 0 is not in row echelon form. The first row is ok; in second and third row we have to eliminate 2 respectively 4. Hence R 2 R 2 2R and R 3 R 3 4R Now R and R 2 are ok, but we need to eliminate 3 from the last row. R 3 R 3 3R 2, yielding which is in row echelon form. Hence (4) The augmented coefficient matrix is not in row echelon form. R is ok, but we need to eliminate the first coefficients from subsequent rows: yielding R 2 R 2 + R, R 3 R 3 + 2R, R 5 R 5 R, Rows R, R 2 are ok, but we need to eliminate the third coefficients in the rows R 3, R 4, R 5. R 3 R 3 2R 2, R 4 R 4 R 2, R 5 R 5 R 2, yielding

16 6 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Now R, R 2, R 3 are ok, but we need to eliminate the last coefficients in R 4 and R 5, i.e., R 4 R 4 R 3, R 5 R 5 R 3, leading to which is in row echelon form. 0 (5) The augmented coefficient matrix 0 0 is not in row echelon form. 2 0 R is ok, but we need to eliminate the first coefficients in R 2, R 3, i.e., yielding R 2 R 2 + R, R 3 R 3 + 2R, To bring the lattest augmented coefficient matrix in row echelon form we need to exchange the second and the third row, R 2 3, leading to which is in row echelon form. We now want to go one step further and describe the set of solutions of a system of linear equations in more detail. We begin by making some preliminary considerations. Consider the system (S) { x + 4x 2 = b 5x + 2x 2 = b 2. The corresponding augmented coefficient matrix is given by ( ) 4 b (A b) =. 5 2 b 2 Let us compare it with the system ( S), obtained by exchanging the two columns of A. Introducing as new unknowns y, y 2 this system reads { 4y + y 2 = b 2y + 5y 2 = b 2

17 .2. GAUSSIAN ELIMINATION 7 and the corresponding augmented coefficient matrix is given by (à b) where à = ( ) Denote by L and L the set of solutions of (S) respectively ( S). It is easy to see that the map (x, x 2 ) (y, y 2 ) := (x 2, x ) induces a bijection between L and L. It means that any solution (x, x 2 ) of (S) leads to the solution y := x 2, y 2 := x of ( S) and conversely, any solution (y, y 2 ) of ( S) leads to a solution x := y 2, x 2 := y of (S), or said differently, by renumerating the unknowns x, x 2, we can read off the set of solutions of ( S) from the one of (S). This procedure can be used to bring an augmented coefficient matrix (A b) in row echelon form into an even simpler form: assume that A has m rows and n columns. By renumerating the unknowns, which corresponds to a permutation of the columns of A, (A b) can be brought into the echelon form (à b), ã 0 ã ã 33. à = 0 ã kk b where k is an integer with 0 k min(m, n) and ã 0, ã 22 0,..., ã kk 0. If k = 0, then à is the matrix whose entries are all zero. Note that now all echelons have height and length equal to one. Using the row operations (R2) and (R3), (à b) can be simplified. First we apply (R2) to the rows R i, i k, (R i ) ã ii (R i ) yielding ã 2 ã 3 ã k 0 ã à =. 0 ã (k )k b

18 8 CHAPTER. SYSTEMS OF LINEAR EQUATIONS and then we apply (R3) to remove all coefficients ã ij with < i < j k to obtain â (k+) â n â 2(k+) â 2n... Â = 0 0 â k(k+) â kn b referred to as being in refined echelon form. The system of linear equations corresponding to this latter augmented coefficient matrix (Â b) is then the following one: y + n j=k+ âjy j = b. y k + n j=k+ âkjy j = b k n 0 y j = b i k + i m. Note that the unknowns are denoted by y,..., y n since the original unknowns x,..., x n might have been permuted, that 0 k min(m, n), and that the set of solutions of the latter system is given by L, introduced above. The sets L and L can now easily be determined. We have to distinguish between different cases: CASE : k < m and there exists i with k + i m so that b i 0. Then L = and hence L =. CASE 2: either [k = m] or [k < m and b k+ = 0,..., b m = 0]. Then the system above reduces to the system of equations n y i + â ij y j = b i, for any i k. (.2.2) j=k+ CASE 2A: if in addition k = n, then the system (.2.2) reads y i = b i for any i n. It means that L = {( b,..., b n )} and therefore the system with augmented coefficient matrix (A b) we started with, has a unique solution. CASE 2B: if in addition k < n, then the system (.2.2) reads y i = b n i â ij y j for any i k j=k+ and the unknowns y k+,..., y n are free variables, also referred to as parameters and denoted by t k+,..., t n. The set of solutions L is then given by n n ) } {( b â j t j,..., b k â kj t j, t k+,..., t n : t k+,..., t n R arbitrary. j=k+ j=k+

19 .2. GAUSSIAN ELIMINATION 9 Hence the system (.2.2) and therefore also the original system with augmented coefficient matrix (A b) has infinitely many solutions. The map R n k R n, given by b â (k+) â (k+2) â n.... t k+. t n bk 0 + t k â k(k+) is a parameter representation of L t k+2 â k(k+2) Let us now illustrate the discussed procedure with a few examples: () Consider the case of one equation and two unknowns, Then 0. 0 a x + a 2 x 2 = b, a t n â kn x + â 2 x 2 = b, â 2 = a 2, b = b a a and we are in the CASE 2B with k =, m =, n = 2. The set of solutions L = L (no renumeration of unknowns were necessary) has the following parameter representation ) ( ) ( b R R 2 â2, t 2 + t 2. 0 It is a straight line in R 2, passing through the point ( b, 0) and having the direction ( â 2, ). (2) Consider the case of one equation and three unknowns, a x + a 2 x 2 + a 3 x 3 = b, a 0. We divide the equation by a and obtain where x + â 2 x 2 + â 3 x 3 = b â 2 = a 2, â 3 = a 3, b = b a a a and we are again in the case 2-B with k =, m =, n = 3. The set of solutions L = L has the following parameter representation ( ) b â 2 â 3 R 2 R 3 t2, 0 + t t 2 + t

20 20 CHAPTER. SYSTEMS OF LINEAR EQUATIONS It is a plane in R 3 passing through the point ( b, 0, 0) and spanned by the vectors ( â 2,, 0) and ( â 3, 0, ). (3) Consider the following system x + x 2 + x 3 = 4, x x 2 2x 3 = 0. (.2.3) The corresponding augmented coefficient matrix (A b) is given by ( ) Apply row operation (R3) and replace R 2 by R 2 R to get ( ) Apply row operation (R2), R 2 2 R 2 to get ( ) and finally apply again row operation (R3), R R R 2, yielding ( ) , which is in refined echelon form. We are in the CASE 2B with k = 2, m = 2, n = 3. Since we have not permuted the unknowns, L = L and a parameter representation of L is given by R R 3, 2 t t 3 0 /2 3/2 which is a straight line in R 3, passing through the point (2, 2, 0) and having the direction (/2, 3/2, ). (4) Consider x + 2x 2 x 3 = 2x + x 2 + x 3 = 0 3x + 0 x 2 + 3x 3 =. Apply row operation (R3) by replacing R 2 R 2 2R, R 3 R 3 3R, yielding

21 .2. GAUSSIAN ELIMINATION 2 Apply (R3) once were, R 3 R 3 2R 2, leading to the following augmented coefficient matrix in echelon form Now apply (R2), R 2 3 R 2 to get and finally we apply (R3) once more, R R 2R 2, to get the following augmented coefficient matrix in refined echelon form We are in CASE 2B with k = 2, m = 3, n = 3 and L = L. Hence a parameter representation of L is given by R R 3, /3 t 3 2/3 + t 3 0 which is a straight line in R 3 passing through the point ( /3, 2/3, 0) and having the direction (,, ). From our analysis one can deduce the following Theorem.2.2. If (S) is a system of m linear equations and n unknowns with m < n, then its set of solutions is either empty or infinite. Remark.2.3. To see that Theorem.2.2 holds, one argues as follows: bring the augmented coefficient matrix in refined row echelon form. Then k min(m, n) = m < n since by assumption m < n. Hence CASE 2A cannot occur and we are either in CASE (L = ) or in CASE 2B (L infinite). An important class of linear systems is the one where the number of equations is the same as the number of unknowns, m = n. Definition.2.3. A matrix A is called quadratic if the number of its rows equals the number of its columns.

22 22 CHAPTER. SYSTEMS OF LINEAR EQUATIONS Definition.2.4. We say that a n n matrix A = (a ij ) i,j n is a diagonal matrix if A = diaga, where diaga = (d ij ) i,j n is the n n matrix with d ii = a ii i n, d ij = 0 i j. It is called the n n identity matrix if a ii = for any i n and a ij = 0 for i j. We denote it by Id n or Id n n. Going through the procedure described above for transforming the augmented coefficient matrix (A b) of a given system (S) of n linear equations with n unknowns into refined echelon form, one sees that (S) has a unique solution if and only if it is possible to bring (A b) without renumbering the unknowns into the form (Id n b), yielding the solution x = b,..., x n = b n. Definition.2.5. We say that a n n matrix A is regular if it can be transformed by the row operations (R) (R3) to the identity matrix Id n. Otherwise A is called singular. Theorem.2.4. Assume that (S) is a system of n linear equations and n unknowns with augmented coefficients matrix (A b), i.e., n a ij x j = b i, i n. Then the following holds: (i) If A is regular, then for any b R n, n a ij x j = b i, i n. has a unique solution. (ii) If A is singular, then for any b R n, the system n a ij x j = b i, i n. has either no solution at all or infinitely many. Example: Assume that A is a singular n n matrix. Then the system with augmented coefficient matrix (A 0) has infinitely many solutions. Corollary.2.5. Assume that A is a n n matrix, A = (a ij ) i,j n. If there exists b R n so that n a ij x j = b i, i n, has a unique solution, then for any b R n, n a ij x j = b i, i n. has a unique solution.

23 .2. GAUSSIAN ELIMINATION 23 Definition.2.6. A system of the form n a ij x j = b i, i m is called a homogeneous system of linear equations if b = (b,..., b m ) = (0,..., 0) and otherwise inhomogeneous. Given a inhomogeneous system the system n a ij x j = b i, i m, n a ij x j = 0, i m is referred to as the corresponding homogeneous system. Note that a homogeneous system of linear equations has always the zero solution. Theorem.2.2 then leads to the following corollary. Corollary.2.6. A homogeneous system of m linear equations with n unknowns and m < n has infinitely many solutions. We summarize the results for a system of n linear equations with n unknowns as follows. Corollary.2.7. Assume that we are given a system (S) n a ij x j = b i, i n, where b = (b,..., b n ) R n. Then the following statements are equivalent: (i) If (S) has a unique solution. (ii) The homogeneous system corresponding to (S) has only the zero solution. (iii) A = (a ij ) i,j n is regular. Finally, we already remark at this point that the set of solutions L of the linear system n a ijx j = b, i m, and the set of solutions L h of the corresponding homogeneous system n a ijx j = 0, i m, have the following properties which can be verified in a straightforward way: (P) for any x, x L h and any λ R, one has x + x L h, and λx L h. (P2) for any x, x L, one has x x L h. (P3) for any x L and x L h, one has x + x L. We will come back to these properties after having introduced the notion of vector spaces.

24 24 CHAPTER. SYSTEMS OF LINEAR EQUATIONS

25 Chapter 2 Matrix calculus and related topics The aim of this chapter to introduce the notion of a matrix and to discuss the elementary properties of matrices as well as related topics. 2. Matrix calculus The aim of this section is to discuss the basics of the matrix calculus. We denote by Mat m n (R) or by R m n the set of all real m n matrices a a n A = (a ij ) i m =.. j n a m a mn Definition 2.. (Sum, multiplication by scalars). (i) For any A = (a ij ) i m j n B = (b ij ) i m in R m n we denote by A + B the m n matrix given by j n (ii) For any A = (a ij ) i m j n (a ij + b ij ) i m R m n. j n in R m n and λ R we denote by λa the m n matrix (λa ij ) i m R m n. j n Note that for the sum of two matrices A and B to be defined, they have to have the same number of rows and the same number of columns. So, e.g., the two matrices ( ) 2 and (6 7) 3 4 cannot be added. The definition of the multiplication of matrices is more complicated. It is motivated by the interpretation of a m n matrix as a linear map from R m to R n which we will discuss in the subsequent chapter in detail. We will see that the multiplication of matrices corresponds to the composition of the corresponding linear maps. At this point however it is only important to know that the definition of the multiplication of matrices is very well motivated. and 25

26 26 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Definition Assume that A = (a il ) i m R m n and B = (b lj ) l n R n k. l n j k Then the product of A and B, denoted by A B or AB for short, is the m k matrix C = (c ij ) i m with coefficients given by j k c ij := n a il b lj. l= Note that in the computation of the coefficient c ij only the i-th row of A, (a i,..., a in ), and the j-th column of B, b j. b nj are involved. Examples () Let A = (2) Let A = (3) Let A = ( ) 2, B = 3 4 ( ) 2, B = 3 4 AB = ( ) 2, B = 3 4 AB = (4) Let A = ( 2), B = ( ). Then A R 2 2 2, B R 2 and AB is well defined. AB = ( ) ( ) = R ( ) 0. Then A, B R and ( ) = ( ) 7 2 R ( ) 0. Then A R , B R 2 3 and ( ) = ( ) 3. Then A R 4 2, B R 2 and AB = = R ( R) ( ) 7 2 R and BA = ( ) = ( ) 3 6 R The following theorem states elementary properties of the matrix multiplication.

27 2.. MATRIX CALCULUS 27 Theorem 2... The following holds: (i) matrix multiplication is associative: for any A R m n, B R n k, C R k l, (ii) matrix multiplication is distributive: (ii) for any A, B R m n, C R n k (ii2) for any A R m n, B, C R n k (AB)C = A(BC) ; (A + B)C = AC + BC ; A(B + C) = AB + AC ; (iii) operation with the identity matrix: for any A R m n, A Id n = A, Id m A = A. To get acquainted with matrix mutiplication let us verify item (i) of Theorem 2.. in the case m = 2, n = 2, k = 2. For any A, B, C R 2 2, the identity A(BC) = (AB)C is verified as follows: let D := BC, E := AB. We claim that AD = EC. Indeed and (AD) ij = (EC) ij = 2 a ik d kj = k= 2 e il c lj = l= 2 k= a ik 2 b kl c lj = l= 2 ( 2 a ik b kl )c lj = l= k= 2 k= l= 2 l= k= 2 a ik b kl c lj 2 a ik b kl c lj. Matrix multiplication is not commutative, ( i.e., ) in general, ( for A, ) B R n n, one has 0 0 AB BA. As an example consider A = and B =. Then AB BA 0 2 since ( ) ( ) ( ) AB = = whereas BA = ( ) ( ) = ( ) 0 2 As a consequence, in general (AB) 2 A 2 B 2. However matrices in certain special classes commute with each other. The set of diagonal n n matrices is such a class. Indeed if A, B R n n are both diagonal matrices, A = diag(a), B = diag(b), then AB is also a diagonal matrix, AB = diag(ab), with implying that AB = BA. (AB) ii = a ii b ii, i n, We say that two n n matrices A, B commute if AB = BA. AB = BA. They anticommute if

28 28 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Definition A matrix A R n n is called invertible if there exists B R n n so that AB = BA = Id n. One can easily verify that for any given A R n n there exists at most one matrix B R n n so that AB = BA = Id n. Indeed assume that C R n n satisfies AC = CA = Id n. Then B = B Id n = B(AC) = (BA)C = Id n C = C. Hence if A R n n is invertible, there exists a unique matrix B R n n so that AB = BA = Id n. This matrix is denoted by A and is called the inverse of A. Note that the notion of the inverse is only defined for quadratic matrices. Examples in R 2 2 : () The null matrix (2) Similarly, ( ) 0 0 is not invertible since for any B R ( ) 0 0 B = 0 0 ( ) ( ) b b 2 = b 3 b 4 ( ) 0 is not invertible since for any B R ( ) 0 B = 0 0 ( ) ( ) b b 2 = b 3 b 4 (3) The identity matrix Id 2 is invertible and Id 2 = Id 2. ( ) 3 0 (4) The diagonal matrix A = is invertible and 0 4 (5) The matrix A = ( ) A 0 = ( ) 0 0 Id ( ) b3 b 4 Id ( ) 2 is invertible. One can easily verify that 3 4 A = 2 ( ) ( ) a b How can the inverse of an invertible 2 2 matrix A = be computed? In the case c d where det(a) 0, it turns out that A is invertible and its inverse A is given by ( ) ( ) A d b d b det(a) det(a) = =. det(a) c a c det(a) a det(a)

29 2.. MATRIX CALCULUS 29 Indeed, one has ( ) ( ) A d b a b A = det(a) c a c d ( ) da bc db bd = = Id det(a) ca + ac cb + ad 2. Similarly one verifies that AA = Id 2. Before we describe a procedure to find the inverse of an invertible n n matrix, let us state some general results on invertible n n matrices. First let us introduce GL R (n) := { } A R n n : A is invertible where we remark that GL stands for general linear. Theorem The following holds: (i) For any A, B GL R (n), one has AB GL R (n) and (AB) = B A. (ii) For any A GL R (n), A GL R (n) and (A ) = A. (2..) To get acquainted with the notion of the inverse of an invertible matrix, let us verify the above statement (i): first note that if A, B GL R (n), then A, B are well defined and so is B A. To see that B A is the inverse of AB we compute and similarly (B A )(AB) = B (A A)B = B Id n B = B B = Id n (AB)B A = A(BB )A = A Id n A = AA = Id n. Hence by definition of the inverse we have that AB is invertible and (AB) is given by B A. To see that (ii) holds we argue similarly. Note that in general, A B is not the inverse of AB, but of BA. Hence in case A and B do not commute, neither do A and B. The important questions with regard to the invertibility of a quadratic matrix A are the following ones: () How can we decide if A is invertible? (2) In case A is invertible, how can we find its inverse?

30 30 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS It turns out that the two questions are closely related and can be answered by the following procedure: consider the system (S) of n linear equations with n unknowns a x + + a n x n = b. a n x + + a nn x n = b n where A = (a ij ) i,j n and b = (b,..., b n ) R n. We want to write this system in matrix notation. For this purpose we consider b as a n matrix and similarly, we do so for x, b b n x b =., x =. Since A is a n n matrix, the matrix multiplication of A and x is well defined and Ax R n. Note that n (Ax) j = a jk x k = a j x + + a jn x n. k= x n. Hence the above linear system (S), when written in matrix notation, takes the form Ax = b. Now let us assume that A is invertible. Then the matrix multiplication of A and Ax is well defined and A (Ax) = (A A)x = Id n x = x. Hence multiplying left and right hand side of Ax = b with A, we get If we choose x = A b. b = e () = 0., 0 then x = A e () is the first column of A. More generally, if 0. 0 b = e (j) = 0. 0

31 2.. MATRIX CALCULUS 3 where is the j-th component of b = e (j) and j n, then A e (j) is the j-th column of A. Summarizing, we have seen that we can decide if A is invertible and if so, determine A, by solving the following systems of linear equations Ax = e (), Ax = e (2),, Ax = e (n). In case A is regular, the solutions x (),, x (n) of the latter equations are uniquely determined and are the columns of A. In that case, we are thus led to apply the following procedure for determining the inverse of A: form the following version of the augmented coefficient matrix a a n a n a nn 0 0 Use Gaussian elimination to determine the solutions x (),, x (n). consequence, one gets the following As an immediate Theorem The quadratic matrix A is invertible if and only if A is regular. In case A is invertible, the linear system Ax = b has the solution x = A b. We recall that a n n matrix A is regular if it can be transformed into the identity matrix Id n by the row operations (R) (R3). In such a case, the above version of the augmented coefficient matrix gets transformed into a a n a n a nn b b n b n b nn and A is given by the matrix (b ij ) i,j n. Let us now illustrate the procedure with a few examples: for each of the matrices A below, decide if A is invertible and if so, determine its inverse. ( ) () For A = consider the augmented coefficient matrix and apply Gaussian elimination ( ) 0 0

32 32 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS (i) R 2 R 2 + R : ( ) (ii) R 2 2 R 2 : ( ) (iii) R R R 2 : ( 0 thus A = ( ) /2 /2 = /2 /2 2 ) ( ). Note that the result coincides with the one obtained by the formula ( ) d b. det(a) c a 2 2 (2) For A = consider the augmented coefficients matrix 2 3 and apply Gaussian elimination: (i) R R 2 R, R 3 R 3 2R : (ii) R 3 R R 2 : (iii) R 3 4 R 3, R 2 3 R 2 :

33 2.. MATRIX CALCULUS 33 (iv) (R ) R 2R 3, R 2 R 2 + R 3 : (v) R R + 2R 2 : and ( ) 2 3 (3) For A = 4 6 A = consider the augmented coefficients matrix ( 2 3 ) and apply Gaussian elimination: (i) R 2 R 2 + 2R : ( 2 3 ) It follows that A is not regular and hence according to Theorem 2..3, A is not invertible. We finish this section by introducing the notion of the transpose of a matrix. Definition Given A R m n, we denote by A T the n m matrix for which the i-th row is given by the j-th column of A. More formally, Examples: () ( ) 2 A = R 3 4 A T = 4 (2) A = 2 5 R 3 2 A T = 3 6 A T = (b ij ) i n, with b ij := a ji j m ( ) 3 R ( ) 2 3 R

34 34 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS (3) A = 2 R 3 A T = ( 2 3) R 3 3 Definition A quadratic matrix A = (a ij ) i,j n R n n is said to be symmetric if A = A T or, written coefficient wise Examples: () A = (2) A = ( ) 2 is symmetric. 2 3 ( ) 2 is not symmetric. 3 a ij = a ji i, j n. (3) If A R n n is a diagonal matrix, then A is symmetric. (Recall that A is a diagonal matrix if A = diag(a).) Theorem (i) For any A R m n, B R n k, (AB) T = B T A T (ii) For any A GL R (n), also A T GL R (n) and (A T ) = (A ) T. To get more acquainted with the notion of the transpose of a matrix, let us verify the statements above: given A R m n, B R n k, one has AB R m k and for any i m, j k, (AB) ij = n a il b lj = n (A T ) li (B T ) jl = n (B T ) jl (A T ) li = (B T A T ) ji l= l= l= and on the other hand, by the definition of the transpose matrix, ((AB) T ) ji = (AB) ij. Combining the two identities yields (AB) T = B T A T. To see that for any A GL R (n) also A T GL R (n), the candidate for the inverse of A T is the matrix (A ) T. Indeed one has (A T )(A ) T = (A A) T = Id T n = Id n since Id n = Id T n, since Id n is diagonal. Similarly (A ) T A T (i) = (AA ) T = Id T n = Id n.

35 2.2. LINEAR DEPENDENCE, BASES, COORDINATES Linear dependence, bases, coordinates In this section we introduce the important notions of linear (in)dependence of vectors in R k, of a basis of R k and of coordinates of a vector in R k with respect to a basis. Later in this course we will discuss these notions in the more general framework of vector spaces of which R k is an example. Elements of R k are called vectors and written as a = (a,..., a k ) where a j, j k are called the components of a. Alternatively, we view a as a k matrix a a =. Definition Assume that a (),, a (n) are vectors in R k. A vector b R k is said to be a linear combination of the vectors a (),, a (n) if there exist real numbers α,, α n so that n b = α a () + + α n a (n) or b = α j a (j). Example: Consider the vectors a () = (, 2), a (2) = (2, ) R 2. Then b = (b, b 2 ) = (, 5) is a linear combination of a () and a (2) since b = 3a () a (2) : indeed a k. b =, (3a () a (2) ) = 3 2 = and b 2 = 5, (3a () a (2) ) 2 = 3 2 = 5. Definition Assume that a (),, a (n) are vectors in R k. They are said to be linearly dependent if there exists i n such that a (i) is a linear combination of a (j), j i, i.e., if there exists α j R, j i, so that a (i) = j i j n α j a (j). The vectors a (),, a (n) are said to be linearly independent if they are not linearly dependent. Example: We claim that the two vectors a () = (, 2), a (2) = (2, ) are linearly independent in R 2. To see it, we assume that they are linearly dependent and then show that this is not possible. If a () and a (2) are linearly independent, then either there exists α R so that a (2) = α a () 2 = α, = α 2 α = 2, α = /2 or there exists α 2 R with a () = α 2 a (2) = 2 α 2, 2 = α 2 α 2 = /2, α 2 = 2.

36 36 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS In both cases we arrive at a contraddiction and hence a (), a (2) cannot be linearly dependent. Thus by definition they are linearly independent. Examples: Decide if the following vectors are linearly dependent: () a () = (, 2), a (2) = (3, 5) in R 2 (2) a () = (, 2, 3), a (2) = (4, 5, 6) in R 3 (3) a () = (2, 4, 8), a (2) = (3, 6, 2) in R 3 (4) a () = (, 3), a (2) = ( 2, 6) in R 2. Important questions with regard to the notions of linear combination and linear independence are the following ones: How can we decide whether given vectors a (),..., a (n) in R k are linearly independent? Given vectors b, a (),..., a (n) in R k how can we decide whether b is a linear combination of a (),..., a (n) and if so, how can we find α,..., α n in R so that b = n α j a (j). Are the numbers α,..., α n uniquely determined? It turns out that these questions are closely related with each other and can be rephrased in terms of systems of linear equations: assume that b, a (),..., a (n) are vectors in R k. We view them as k matrices and write b b =. b k, a () = a (). a () k = a. a k,..., a (n) = a (n). a (n) k = Denote by A the k n matrix whose columns are given by a (),..., a (n), a... a n a ()... a (n) A =.. =... a k... a kn a () k... a (n) k Recall that b is a linear combination ov the vectors a (),..., a (n) if there exist real numbers α,..., α n so that n b = α j a (j). a n. a kn.

37 2.2. LINEAR DEPENDENCE, BASES, COORDINATES 37 Written componentwise, it means that or in matrix notation b = Aα, where b = α a () + + α n a (n) = n a jα j. b k = α a () k + + α n a (n) k = n a kjα j α α =. α n R n. Note that A R k n and hence the matrix multiplication of A by α is well defined. Hence we can express the question if b is a linear combination of a (),..., a (n) in terms of the matrix calculus as follows: b is a linear combination of a (),..., a (n) if and only if the linear system Ax = b has a solution x R n. a (),..., a (n) are linearly dependent if and only if the linear homogeneous system Ax = 0 has a solution x R n, x 0. a (),..., a (n) are linearly independent if and only if the linear homogeneous system Ax = 0 has only the trivial solution x = 0 R n. In the important case where k = n, we thus have, in view of the definition of a regular matrix, the following Theorem Assume that a (),..., a (n) are vectors in R n. Then the following two statements are equivalent: (i) a (),..., a (n) are linearly independent; (ii) the n n matrix a ()... a (n) A =.. a () n... a (n) n is regular. Example: Let b = (5, 5,, 2), a () = (,, 0, ), a (2) = (0, 2,, ), a (3) = ( 2,, 0, ). Show that b is alinear combination of a (), a (2), a (3). Solution: it suffices to study the linear system Ax = b where x = x 2 and A is the x matrix a () a (2) a (3) 0 2 A =.. = a () 4 a (2) 4 a (3) 4 x

38 38 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Consider the augmented coefficient matrix R 2 R 2 R, R 4 R 4 R R R 3 R 3 + 2R 2, R 4 R 4 R R 4 R 4 3R Therefore, x 3 = 2, x 2 =, x = 5 + 2x 3 = hence b = a () + a (2) + ( 2) a (3) = a () + a (2) 2a (3). Definition Vectors a (),..., a (n) in R k are called a basis of R k if any vector b R k can be represented in a unique way as linear combination of a (),..., a (n), i.e. if for every b R k there exist unique real numbers x,..., x n such that b = n x j a (j). The numbers x,..., x n are called the coordinates of b with respect to the basis a (),..., a (n).

39 2.2. LINEAR DEPENDENCE, BASES, COORDINATES 39 Theorem (i) Any basis of R k consists of k vectors. (ii) If the vectors a (),..., a (k) in R k are linearly independent, then they form a basis in R k. (iii) If a (),..., a (n) are vectors in R k with n < k, then there exist vectors a (n+),..., a (k) in R k so that a (),..., a (n), a (n+),..., a (k) form a basis of R k. In words, a collection of linearly independent vectors of R k can always be completed to a basis of R k. Examples: The vectors e () = (, 0,..., 0), e (2) = (0,, 0,..., 0),..., e (n) = (0,..., 0, ) form a basis of R n, referred to as the standard basis of R n. Indeed any vector b = (b,..., b n ) R n can be written uniquely as b = n b j e (j). Note that b,..., b n e (),..., e (n) of R n. are the coordinates of b with respect to the standard basis a () = (, ), a (2) = (2, ) is a basis of R 2 : according to Theorem 2.2.2, it suffices to verify that a (), a (2) are linearly independent in R 2 and according to Theorem 2.2., this is the case iff ( ) a () a (2) A = a () 2 a (2) 2 = ( ) 2 is regular. Since det(a) = 2 = 0, A is indeed regular. Find the coordinates of the vector b = (, 3) R 2 with respect to the basis a () = (, ), a (2) = (2, ) of R 2. We need to solve the linear system Ax = b, whose augmented coefficient matrix is given by ( 2 ) 3 R 2 R 2 R ( and hence x 2 = 2, x = 2x 2 = 5. The coordinates of b withe respect to a () and a (2) are thus 5 and 2, b = 5a () 2a (2), x = 5, x 2 = 2. Note that the coordinates of b with respect to the standard basis of R 2 are and 3. An important question is how the coefficients 5, 2 and, 3 are related with each other. Let us consider the latter question in a more general context. Assume that [a] := [a (),..., a (n) ] and b := [b (),..., b (n) ] )

40 40 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS are bases of R n and consider a vector u R n. Then u = n α j a (j), u = n β j b (j) where α,..., α n are the coordinates of u with respect to [a] and β,..., β n the ones of u with respect to [b]. We would like to have a method of computing β j, j n from α j, j n. To this end we have to express the vectors a (j) as linear combination of the vectors b (),..., b (n) : Then or a (j) = n t... t n t ij b (i), T :=.. t n... t nn i= n β i b (i) = u = i= n α j a (j) = n β i b (i) = i= n α j n t ij b (i) i= n ( n α j t ij )b (i). i= Since the coordinates of u with respect to the basis [b] are uniquely determined one conludes that β i = n t ijα j for any i n. In matrix notation, we thus obtain the relation α β = T α, α =. α n, β = β. It turns out that a convenient notation for T is the following one T = Id [a] [b]. β n. (2.2.) The j-th column of T is the vector of coordinates of a (j) with respect to the basis [b]. Similarly, we express b (j) as a linear combination of a (),..., a (n), n s... s n b (j) = s ij a (j), S =... i= s n... s nn We then obtain α = Sβ, S = id [b] [a]. (2.2.2) Theorem Assume that [a] and [b] are bases of R n. Then (i) S and T are regular n n matrices, hence invertible. (ii) T = S.

41 2.2. LINEAR DEPENDENCE, BASES, COORDINATES 4 Note that item (ii) can be deduced by (2.2.), (2.2.2) and item (i): indeed β = T α (2.2.2) β = T Sβ α = Sβ (2.2.) α = ST α. From item (i), it follows that T S = Id n = ST, i.e., T = S. Example: Consider the standard basis [e] = [e (), e (2), e (3) ] and the basis [b] = [b (), b (2), b (3) ] of R 3 given by b () = (,, ), b (2) = (0,, 2), b (3) = (2,, ). Let us compute S = Id [b] [e]. The first column of S is the vector of coordinates of b () with respect to the standard basis [e], and similarly, and Hence b () = e () + e (2) + e (3) b (2) = 0 e () + e (2) + 2 e (3) b (3) = 2 e () + e (2) e (3). 0 2 S = 2 With Gaussian elimination, we then compute T = Id [e] [b] = S, T = The coordinates β, β 2, β 3 of u = (, 2, 3) R 3 with respect to the basis [b], u = 3 β jb (j), can then be computed as follows: compute the coordinates of u with respect to the basis [e] : α =, α 2 = 2, α 3 = β = Id [e] [b] α = T α = = Hence u = b () + b (2) + 0 b (3). How to remember the formula? β = Id [e] [b] α { β = new coordinates, α = old coordinates [b] = new basis, [e] = old basis We will come back on this topic in Chapter 3, when we discuss the notion of a linear map.

42 42 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS 2.3 Determinants ( ) a a We already introduced the notion of the determinant of a 2 2 matrix A = 2, a 2 a 22 and established important properties: det(a) = a a 22 a 2 a 2 A is regular if and only if det(a) 0 (simple criterion to decide if a 2 2 matrix is regular or not) Cramer s rule for solving Ax = b, b R 2. Write ( ) ( ) a () a =, a (2) a2 =, b = a 2 a 22 ( b b 2 ). Then x = det(b a(2) ) det(a () a (2) ), x 2 = det(a() b) det(a () a (2) ). We now want to address the question if the notion of determinant can be extended to n n matrices so that corresponding properties hold as for 2 2 matrices. The answer is yes! Among the many equivalent ways of defining the determinant of n n matrices, we choose a recursive definition, which defines the determinant of a n n matrix in terms of determinants of certain (n ) (n ) matrices. First we need to introduce some more notations. For A R n n and i, j n, we denote by A (i,j) R (n ) (n ) the (n ) (n ) matrix, obtained from A by deleting the i-th row and the j-th column. Example: For 2 3 A = R one has ( ) ( ) ( ) A (,) =, A (,3) =, A (,2) = ( ) ( ) ( ) A (2,) =, A (2,2) =, A (2,3) = To motivate the inductive definition of the determinant of a n n matrix, we first consider the cases n =, n = 2. One has n = : A = (a ) R det(a) = a n = 2 : ( ) a a A = 2 a 2 a 22 det(a) = a a 22 a 2 a 2

43 2.3. DETERMINANTS 43 which can be written as det(a) = a det(a (,) ) a 2 det(a (,2) ) = ( ) + a det(a (,) ) + ( ) +2 a 2 det(a (,2) ) 2 = ( ) +j a j det(a (,j) ). Definition For any A R n n with n 3, det(a) = n ( ) +j a j det(a (,j) ). (2.3.) Since A (,j) is a (n ) (n ) matrix for any j n, this is indeed a recursive definition. We refer to (2.3.) as the expansion of det(a) with respect to the first row. Example: Then 2 3 A = 4 2 R det(a) = ( ) + det(a (,) ) + ( ) +2 2 det(a (,2) ) + ( ) +3 3 det(a (,3) ). Since one gets A (,) = ( ) 2, A (,2) = 0 ( ) 4, A (,3) = ( ) 4 2, 0 det(a) = (2 0) 2(4 ) + 3(4 0 2 ) = = 0. Let us state some elementary important properties of the determinant. Theorem For any A R n n and k n the following holds: (i) n det(a) = ( ) k+j a kj det(a (k,j) ) (expansion of det(a) with respect to the k-th row) (ii) n det(a) = ( ) j+k a jk det(a (j,k) ) (expansion of det(a) with respect to the k-th column) (iii) det(a) = det(a T ).

44 44 CHAPTER 2. MATRIX CALCULUS AND RELATED TOPICS Note that item (iii) of the latter theorem follows from item (ii), since (A T ) j = a j and (A T ) (,j) = A (j,). To state the next theorem let us introduce some more notations. For A R n n, denote by a (),..., a (n) its columns and by A (),..., A (n) its rows. We then have A = (a () a (n) ), A = A (). A (n). Theorem For any A R n n, the following identities hold: (i) for any j < i n, and det(a () a (j) a (i) a (n) ) = det(a () a (i) a (j) a (n) ) (ii) For any λ R and i n and for any b R n A () A ().. A (j) A (i) det. = det.. A (i) A (j).. A (n) A (n) det(a () λa (i) a (n) ) = λ det(a () a (i) a (n) ) det(a () (a (i) + b) a (n) ) = det(a () a (i) a (n) ) + det(a () b a (n) ) and an analogous statement holds for the rows of A. (iii) for any i, j n and i j, λ R, det(a () (a (i) + λa (j) ) a (n) ) = det(a () a i a (n) ). An analogous statement is valid for the rows of A. (iv) If A is upper triangular, namely a ij = 0 i > j, then det(a) = a a 22 a nn = n a jj. Expressed in words, item (ii) says that det(a) is linear with respect to its i-th column, i n (cf Chapter 3 for the notion of linear maps). Similarly, det(a) is linear with respect to its i-th row, i n.