A tutorial on inverting 3 by 3 matrices with cross products

Size: px

Start display at page:

Download "A tutorial on inverting 3 by 3 matrices with cross products"

Duane Lambert
7 years ago
Views:

1 A tutorial on inverting by atries with ross produts By Cedrik Collob Abstrat. his tutorial introdues the idea of inverting a by atrix and alulating its erinant with ross produts hopefully in a siple anner intelligible to any reader with inial atheatial and engineering skills. he forulas are derived twie with different approahes and every step is purposely ade unneessarily ailed for ease of understanding. he goal of this douent is to help the reader ipleent and use the ideas behind this tutorial iediately after reading it.. Introdution he inversion of atries is a reurring step in several Matheatis and Coputer Siene algoriths. Although there already exist several generi algoriths for the inversion of atries, there are several doains suh as Ebedded software, Coputer Graphis and Video Gaes for whih perforane is ritial and for whih generi algoriths used with low diension atries are not good enough. A subset of the perforane ritial probles enountered in those doains an be redued and solved by a linear syste of size by ; therefore the need for an effiient way to inverse a by atrix. On top of that hardware SIMD arhitetures have been providing for several years the eans to opute ross produts very effiiently, therefore aking the tehnique introdued in this tutorial both a siple and effiient ethod. he rest of the tutorial is organized as follow: Setion introdues the theory behind general square atrix inversion. Setion introdues the differene between theory and in use algoriths. Setion 4 siplifies the results of Setion for sall by atries. Setion 5 shows how to ahieve the sae results as Setion 4 using ross produts. Setion 6 onludes this tutorial.. Basi linear algebra refresher Let M be a square atrix of diension n. We note M ( ) representing the row indies and, n with i, n j representing the olun indies. M an also n be visualized in the following for M. Notie that n is the last n nn eleent of the first line and n is the first eleent of the last line. ollob@yahoo.o /

2 Although outside the sope of this tutorial we an deonstrate that M is invertible if and only if its erinant that will be referened as ( M ) is not zero. When M is invertible, its inverse is written as M and an be erined by the following forula: M adj( M ) ( M ) () he notation adj( M ) represents the adjugate atrix of M. he forula eans that the inverse of M is the adjugate saled by the inverse of the erinant whih is a salar. he adjugate and erinant still need to be explained so that the previous forula akes sense and be at all useable. he adjugate of M is also a square atrix of diension n, and its eleents a are defined by the following forula: a ji (notie that i and j are inverted on the right hand side of the equation so that to transpose the atrix). he oeffiients are alled ofators and are defined with the erinant of the square atrix of diension n reated by reoving the line i and olun j fro M., j, j+ n i j i, i, j i, j+ + i, n ( ) i +, i +, j i +, j+ i +, n nn n, j n, j+ nn () he erinant of M an be alulated with the following forula: n ( M ) () i he hoie of j in () does not atter and for a atrix of diension one, ( M ). As you ight have notied this reate a reursive forula sine ( M ) requires the value of the ofators, and the ofators are defined with erinants! However you an also notie that the size of the atries involved in the ofator is always one less than the size of the atrix of whih we alulate the erinant. herefore the reursion will reah atries of size one and therefore will end. I reoend any good Linear Algebra ourse book, for Frenh readers [] is a good referene

3 . Matrix inversion in pratie Although the inversion of atries with adjugate has theoretial value, it is iportant to ention that this for is very rarely used in pratie to inverse atries. For several lasses of proble inversing the atrix just to find one x so that Mx y when you have M and y an be prohibitory expensive, and it akes ore sense to reate iterative algoriths that onverge to x. he ost oon ethods to proeed this way are Jaobi, Gauss-Seidel and the Conjugate gradient. Even for the ases where the inverse is really needed there are usually better algoriths. he ost oon ethods 4 are Gauss-Jordan, Shipley-Colean [4], LU deoposition, QR deoposition. 4. he by ase If we apply () and () to the square atrix M, we get the following: ( M ) + + (4) ( ) ( ) ( ) ( ) ( ) ( ) (5) (6) (7) (8) (9) (0) See [] for a good start with oon ethods for solving linear systes and their desriptions. 4 See [] for a good start with atrix inversion ethods.

4 Using (5), (6) and (7) with (4) gives: ( ) ( ) ( ) () () () ( ) ( ) ( ) ( M ) + + (4) Using (5), (6), (7), (8), (9), (0), (), () and () with () gives: M ( M ) (5) You an verify by hand that MM I, by ultiplying M by M. 5. Calulating the erinant with a dot produt and a ross produt Although the forula (5) an be optiized, it is based on salar operations and do not take advantage of SIMD features oonly available on ost CPUs. On soe proessor arhitetures an alternative forulation using SIMD an provide ore effiient results. A oon feature with SIMD is the availability of vetors of several diensions that stand for the MD of Multiple Data in SIMD, in ost oon platfors available today those vetors are four diensional vetors storing floating point values or bits integers. Stepping bak fro the final for reahed in the last forula and looking at (4), we an see that ( M ) looks like a dot produt of two three-diensional vetors. Defining C whih is the first olun of M, and R [ ] whih is the first row of adj( M ), we notie that ( M ) RC or using a dot produt for: ( M ) R. C (6) 4

5 Now if we look arefully at R we get: R C C (7) C and C the seond and third rows of M, therefore herefore using (7) in (6) gives: C and C. ( ) ( M ) C. C C (8) 6. Inverting the atrix with ross produts he forula (8) is quite siple and nie. Moreover through the alulus we also realized that the first row of adj( M ) was the ross produt of the seond and third oluns of M, whih was interesting and enourage to look at the other rows of adj( M ). If we all respetively R and R the seond and third rows of adj( M ), and all respetively C and C the seond and third rows of adj( M ) R C C (9) R C C (0) Using (7), (8), (9) and (0) we get the elegant and SIMD friendly forula: C. C C M C C C C C C ( ) ( ) () 5

6 7. An alternative way to get the ross produt inversion forula he ross produt of two vetors is by definition noral to both vetors used to alulate the ross produt, if we hoose C and ( i, j),,,, then i C with { } { } j ( ) and C j ( Ci C j ) C. C C 0 i i j. 0 () If we write two atries A and B with the following rows and oluns notations: A A A A [ ] B B B B, where A, A and A are three diensional horizontal vetors., where B, B and B are three diensional vertial vetors. A A. B A. B A. B AB A B B B A. B A. B A. B A A. B A. B A. B hen [ ] () Iagine that B is M and we want A to be its inverse M. 0 0 So we want AB 0 0, therefore we need to ake sure that A. B 0 and 0 0 A. B 0. We know B and B sine they are oluns of M, and we want to find a vetor that is noral to both. Using () it is very easy with A k ( B ^ B ) identify A k ( B ^ B ) and A k ( B ^ B ). Following the sae ethod we an. herefore ( ^ ) ( ^ ) ( ^ ) (4) A k B B k B B k B B We now need to identify k, k and k by using (4) in () whih gives ( ) k 0 0 AB 0 kb. ( B B ) k ( ) 6

7 herefore k k k ( ) ( ) ( ) (5) (6) (7) Another interesting property if E, F, G are arbitrary vetors is that we an yle vetors to the left or to the right in the equation below. ( ) E. F G F.( G E) G.( E F) (8) Note that we only need to deonstrate one equality, sine hanging variable naes leads to the third equality. E e e e and f g fg fg F G f g fg fg, therefore f g fg fg ( ) ( ) ( ) ( ) E. F G e f g f g + e f g f g + e f g f g (9) Expanding (9) gives E.( F G) e fg e fg + e fg e fg + e fg e fg (0) Fatoring (0) by f, f and f gives ( ) ( ) ( ) ( ) E. F G f e g e g + f e g e g + f e g e g () g e ge ge G E g e ge ge g e ge ge therefore see fro () that ( ) E. F G F.( G E). Using (8) with (5), (6) and (7) we get k k k ( ) () 7

8 And finally using () and (4) we reah the sae result as in the previous setion A B B B B B B ( ) [ ] ^ ^ ^ 8. Conlusion his douent has shown with two distint approahes that it is possible to invert a three by three atrix with three ross produts, one dot produt and a atrix transpose. It has also shown that the erinant of the sae atrix ould be alulated with one ross produt and one dot produt. Although inverting a atrix is a solved proble, finding optial algoriths is still an unsolved proble. Of related iportane is the fat that the oputing oplexity of atrix inversion is equivalent to atrix ultipliation 5, and also equivalent to the oplexity of solving a olletion of linear equations and oputing the erinant of a atrix 6. herefore any progress ade by any of those area of researh iediately will benefit the other areas. A lot of progress has been ade sine the seinal disovery [7], ade by Strassen in 969. However there is still ajor interest to investigate and disover new and iproved algoriths to address those areas, and the reader of this tutorial an take part in this researh effort. 9. Referenes. J.M Arnaudies, Cours de Matheatiques. Algebre, Bordas (99). R. Barrett et al., eplates for the Solution of Linear Systes: Building Bloks for Iterative Methods, nd Edition, SIAM (994). W.H. Press et al., Nuerial Reipes in C++: he Art of Sientifi Coputing, Cabridge University Press (00) 4. J.P. Pante, Coputer Algoriths in Power Syste Analysis, World Wide Web site at the address (999) 5..H. Coren, et al., Introdution to Algoriths, Seond Edition, MI Press and MGraw-Hill (00) 6. V. Pan, How an we speed up atrix ultipliation?, SIAM Rev. 6 (984) 7. V. Strassen, Gaussian Eliination is not Optial, Nuer. Math. (969) 5 See [5] for ore ails. 6 See [6] for ore ails. 8

tr(a + B) = tr(a) + tr(b) tr(ca) = c tr(a)

tr(a + B) = tr(a) + tr(b) tr(ca) = c tr(a) Chapter 3 Determinant 31 The Determinant Funtion We follow an intuitive approah to introue the efinition of eterminant We alreay have a funtion efine on ertain matries: the trae The trae assigns a numer