Matrix Calculations: Inverse and asis Transformation A. Kissinger (and H. Geuvers) Institute for Computing and Information ciences Intelligent ystems Version: spring 25 A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations / 42
Outline Existence and uniqueness of inverse Existence and uniqueness of inverse A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 2 / 42
Recall: Inverse matrix Definition Let A be a n n ( square ) matrix. This A has an inverse if there is an n n matrix A with: A A I and A A I Note Matrix multiplication is not commutative, so it could (a priori) be the case that: A has a right inverse: a such that A I and A has a (different) left inverse: a C such that C A I. However, this doesn t happen. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 4 / 42
Uniqueness of the inverse Theorem If a matrix A has a left inverse and a right inverse, then they are equal. If A I and C A I, then C. Proof. Multiply both sides of the first equation by C: Corollary C A C I C If a matrix A has an inverse, it is unique. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 5 / 42
When does a matrix have an inverse? Theorem (Existence of inverses) An n n matrix has an inverse (or: is invertible) if and only if it has n pivots in its echelon form. oon, we will introduce another criterion for a matrix to be invertible, using determinants. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 6 / 42
Explicitly computing the inverse, part I uppose we wish to find A for A We need to find x, y, u, v with: ( ) ( ) a b x y c d u v Multiplying the matrices on the LH: ( ) ax + bu cx + du ay + bv cy + dv...gives a system of 4 equations: ax + bu cx + du ay + bv cy + dv ( ) a b c d ( ) ( ) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 7 / 42
Computing the inverse: the 2 2 case, part II plitting this into two systems: { ax + bu and cx + du { ay + bv cy + dv olving the first system for (u, x) and the second system for (v, y) gives: u c ad bc x d ad bc and v a ad bc y b ad bc (assuming bc ad ). Then: ( ) A x y u v Conclusion: A ad bc ( d ad bc c ad bc ( d ) b c a b ) ad bc a ad bc learn this formula by heart A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 8 / 42
Computing the inverse: the 2 2 case, part III ummarizing: Theorem (Existence of an inverse of a 2 2 matrix) A 2 2 matrix A ( ) a b c d has an inverse (or: is invertible) if and only if ad bc, in which case its inverse is ( ) A d b ad bc c a A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 9 / 42
Applying the general formula to the swingers Recall P ( ).8., so a.2.9 8, b, c 2, d 9 ad bc 72 2 7 7 so the inverse exists! Thus: ( ) P d b ad bc c a ( ).9. Then indeed: 7 (.9..2.8 ) 7 ( ).8..2.9.2.8 7 ( ).7.7 ( ) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations / 42
Existence and uniqueness of inverse What a determinant does For a square matrix A, the deteminant det(a) is a number (in R) It satisfies: det(a) A is not invertible A does not exist A has < n pivots in its echolon form have useful properties, but calculating determinants involves some work. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 2 / 42
Determinant of a 2 2 matrix Assume A ( ) a b c d Recall that the inverse A exists if and only if ad bc, and in that case is: ( ) A d b ad bc c a In this 2 2-case we define: ( ) a b det c d a b c d ad bc Thus, indeed: det(a) A does not exist. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 3 / 42
Determinant of a 2 2 matrix: example Recall the political transisition matrix ( ) ( ).8. P 8.2.9 2 9 Then: det(p) 8 9 2 72 2 7 7 We have already seen that P exists, so the determinant must be non-zero. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 4 / 42
Determinant of a 3 3 matrix a a 2 a 3 Assume A a 2 a 22 a 23 a 3 a 32 a 33 Then one defines: a a 2 a 3 det A a 2 a 22 a 23 a 3 a 32 a 33 +a a 22 a 23 a 32 a 33 a 2 a 2 a 3 a 32 a 33 + a 3 a 2 a 3 a 22 a 23 Methodology: take entries a i from first column, with alternating signs (+, -) take determinant from square submatrix obtained by deleting the first column and the i-th row A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 5 / 42
Determinant of a 3 3 matrix, example 2 5 3 4 2 3 4 5 2 + 2 2 3 4 ( ) ( ) ( ) 3 5 2 2 8 + 3 3 22 29 A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 6 / 42
The general, n n case a a n.. a n... a nn a 22 a 2n a 2 a n a 32 a 3n +a.. a 2 a n2... a nn.. a n2... a nn a 2 a n + a 3 ± a n.. a (n )2... a (n )n (where the last sign ± is + if n is odd and - if n is even) Then, each of the smaller determinants is computed recursively. (A lot of work! ut there are smarter ways...) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 7 / 42
ome properties of determinants Theorem For A and two n n matrices, det(a ) det(a) det(). The following are corollaries of the Theorem: det(a ) det( A). If A has an inverse, then det(a ) det(a). det(a k ) (det(a)) k, for any k N. Proofs of the first two: det(a ) det(a) det() det() det(a) det( A). (Note that det(a) and det() are simply numbers). If A has an inverse A then det(a) det(a ) det(a A ) det(i ), so det(a ) det(a). A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 8 / 42
Applications Existence and uniqueness of inverse detect when a matrix is invertible Though we showed an inefficient way to compute determinants, there is an efficient algorithm using, you guessed it...gaussian elimination! olutions to non-homogeneous systems can be expressed directly in terms of determinants using Cramer s rule (wiki it!) Most importantly: determinants will be used to calculate eigenvalues in the next lecture A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 9 / 42
ases and coefficients A basis for a vector space V is a set of vectors {v,..., v n } in V such that: They are linearly independent: a v +... + a n v n all a i 2 They span V, i.e. for all v V, there exist a i such that: v a v +... + a n v n A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 2 / 42
ases, equivalently Equivalently: a basis for a vector space V is a set of vectors {v,..., v n } in V such that: They uniquely span V, i.e. for all v V, there exist unique a i such that: v a v +... + a n v n It s useful to think of column vectors just as notation for this sum: a. a n : a v +... + a n v n Previously, we haven t bothered to write, but it is important! A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 22 / 42
Example: two bases for R 2 Let V R 2, and let {(, ), (, )} be the standard basis. Vectors expressed in the standard basis give exactly what you expect: ( ) ( ) ( ) ( ) a a a + b b b ut expressing a vector in another basis can give something totally different! For example, let {(, ), (, )}: ( ) a b a ( ) + b ( ) ( ) (a + b) b A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 23 / 42
ame vector, different outfits Hence the same vector can look different, depending on the choice of basis: ( ) ( ) (a + b) a b b Examples: ( ) ) ( ( ) ) ( ( ) 3 ) ( ( ) 2 ( ) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 24 / 42
Why??? Existence and uniqueness of inverse Many find the idea of multiple bases confusing the first time around. {(, ), (, )} is a perfectly good basis for R 2. Why bother with others? ome vector spaces don t have one obvious choice of basis. Example: subspaces R n. 2 ometimes it is way more efficient to write a vector with respect to a different basis, e.g.: 9378234 43823 224 54232498.. 3 The choice of basis for vectors affects how we write matrices as well. Often this can be done cleverly. Example: JPEGs, Google A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 25 / 42
Transforming bases, part I How can we transform a vector form the standard basis to a new basis, e.g. {(, ), (, )}? In order to express (a, b) R 2 in we need to find x, y R such that: (a ) b x ( ) + y ( ) : ( ) x y olving the equations gives: y b and x a b Example The vector v (, ) R 2 is represented w.r.t. the basis as: ( ) ( ) ( ) ( ) 9 9 + (use a, b in the formulas for x, y given above.) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 26 / 42
Transforming bases, part II Easier: given a vector written in {(, ), (, )}, how can we write it in the standard basis? Just use the definition: ( ) ( ) ( ) ( ) x x + y x + y y y Or, as matrix multiplication: ( ) ( ) ( ) x x + y y y }{{}}{{}}{{} T in basis in basis Let T be the matrix whose columns are the basis vectors. Then T transforms a vector written in into a vector written in. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 27 / 42
Transforming bases, part III How do we go back? Need T which does this: ( ) ( a a b ) b b olution: use the inverse! T : (T ) Example: (T )...which indeed gives: ( ) ( ) ( ) ( ) a b ( a b ) b A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 28 / 42
Transforming bases, part IV How about two non-standard bases? ( ) ( ) ( {, } C { 2 Problem: translate a vector from olution: do this in two steps: T C T v }{{}...then translate from to C ( ) a to b T v }{{} first translate from to... ), ( ) a b ( ) } 2 (T C ) T v C A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 29 / 42
Transforming bases, example For bases: ( { ), ( ) } C {...we need to find a and b such that ( ) ( ) a a b b C ( 2 ), Translating both sides to the standard basis gives: ( ) ( ) ( ) ( ) a a 2 2 b b This we can solve using the matrix-inverse: ( ) ( ) a ( b 2 2 ) ( ) } 2 ( ) a b A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 3 / 42
Transforming bases, example For: ( ) a we compute b }{{} in basis C ( 2 2 ( ) ( ) 2 2 ) } {{ } T C ( 2 2 ( ) }{{} T 4 4 ) ( ) a b }{{} in basis ( ) 4 ( ) 2 99 2 2 which gives: ( ) a b }{{} in basis C 4 ( ) ( ) 2 99 a 2 2 b }{{}}{{} T C in basis A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 3 / 42
asis transformation theorem Theorem Let be the standard basis for R n and let {v,..., v n } and C {w,..., w n } be other bases. Then there is an invertible n n basis transformation matrix T C such that: a a. T C. with.. a n a n a n a C n 2 T is the matrix which has the vectors in as columns, and T C : (T C ) T 3 T C (T C ) a a A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 32 / 42
Matrices in other bases ince vectors can be written with respect to different bases, so too can matrices. For example, let g be the linear map defined by: ( ) ( ) ( ) ( ) g( ) g( ) Then, naturally, we would represent g using the matrix: ( ) ecause indeed: ( ) ( ) ( ) and ( ) ( ) ( ) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 33 / 42
On the other hand... Lets look at what g does to another basis: ( ) ( ) {, } First (, ) : ( ) g( ) g( Then, by linearity: ( )... g( ) + g( ( ) ) g( ( ) ) ( ) + ( ) + ( ) ( ) )... ( ) ( ) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 34 / 42
On the other hand... imilarly (, ) : g( ( ) ( ) {, } ( ) ( ) ) g( ) g( ( ) ( ) )... Then, by linearity: ( )... g( ) g( ( ) ) ( ) ( ) ( ) ( ) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 35 / 42
A new matrix Existence and uniqueness of inverse From this: ( ) g( ) ( ) g( ( ) ( ) ) It follows that we should instead us this matrix to represent g: ( ) ecause indeed: ( ) ( ) ( ) and ( ) ( ) ( ) A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 36 / 42
A new matrix Existence and uniqueness of inverse o on different bases, g acts in totally different way! ( ) ( ) ( ) ( ) g( ) g( ) g( ( ) ) ( ) g( ( ) ( ) )...and hence gets a totally different matrix: ( ) ( ) vs. A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 37 / 42
Transforming bases, part II Theorem Assume again we have two bases, C for R n. If a linear map f : R n R n has matrix A w.r.t. to basis, then, w.r.t. to basis C, f has matrix A : A T C A T C Thus, via T C and T C one tranforms -matrices into C-matrices. In particular, a matrix can be translated from the standard basis to basis via: A T A T A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 38 / 42
Example basis transformation, part I Consider the standard basis {(, ), (, )} for R 2, and as alternative basis {(, ), (, 2)} Let the linear map f : R 2 R 2, w.r.t. the standard basis, be given by the matrix: ( ) A 2 3 What is the representation A of f w.r.t. basis? ( ) ince is the standard basis, T contains the 2 -vectors as its columns A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 39 / 42
Example basis transformation, part II The basis transformation matrix T in the other direction is obtained as matrix inverse: T ( ( ) ) ( ) ( ) T 2 2 2 2 2 Hence: A T ( A ) T ( ) 2 2 ( ) ( 2 3 ) 2 2 2 ( 3 2 ) 2 4 4 2 ( ) 4 2 2 2 2 ( ) 2 A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 4 / 42
Example basis transformation, part III Consider a vector v R 2 which can be represented in bases and respectively as: ( ) ( ) 5 5 and 4 That is, we have: Then, if we apply A we get: v : T A v 4 2 ( ) 5 4 ( ) 5 4 2 ( ) to v, written in the basis, 2 3 ( ) 2 3 ( ) 5 4 ( ) 22 A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 4 / 42
Example basis transformation, part IV ( ) On the other hand, if we apply A 2 2 2 2 to v we get: ( ) ( ) ( ) A 2 2 5 v 2 2 4 2...which we interpret as a vector written in. Comparing the two results: ( ) ( 2 ) + 2 2 ( ) ( ) ( ) 2 22 22...we get the same outcome! In fact: this is always the case. It can be shown using the definitions of A, v and properties of inverses (i.e. no matrixrekenen necessary!). A. Kissinger (and H. Geuvers) Version: spring 25 Matrix Calculations 42 / 42