Chapter 04.06 Gaussia Elimiatio After readig this chapter, you should be able to:. solve a set of simultaeous liear equatios usig Naïve Gauss elimiatio,. lear the pitfalls of the Naïve Gauss elimiatio method,. uderstad the effect of roud-off error whe solvig a set of liear equatios with the Naïve Gauss elimiatio method, 4. lear how to modify the Naïve Gauss elimiatio method to the Gaussia elimiatio with partial pivotig method to avoid pitfalls of the former method, 5. fid the determiat of a square matrix usig Gaussia elimiatio, ad 6. uderstad the relatioship betwee the determiat of a coefficiet matrix ad the solutio of simultaeous liear equatios. How is a set of equatios solved umerically? Oe of the most popular techiques for solvig simultaeous liear equatios is the Gaussia elimiatio method. The approach is desiged to solve a geeral set of equatios ad ukows a x + a + ax +... + a x b a x + a + ax +... + a x b...... a x + a + ax +... + ax b Gaussia elimiatio cosists of two steps. Forward Elimiatio of Ukows: I this step, the ukow is elimiated i each equatio startig with the first equatio. This way, the equatios are reduced to oe equatio ad oe ukow i each equatio.. Back Substitutio: I this step, startig from the last equatio, each of the ukows is foud. Forward Elimiatio of Ukows: I the first step of forward elimiatio, the first ukow, x is elimiated from all rows below the first row. The first equatio is selected as the pivot equatio to elimiate x. So, 04.06.
04.06. Chapter 04.06 to elimiate x i the secod equatio, oe divides the first equatio by a (hece called the pivot elemet) ad the multiplies it by a. This is the same as multiplyig the first equatio by a / a to give a a a a x + a +... + a x b a a a Now, this equatio ca be subtracted from the secod equatio to give a a a a a... a a x b b a + + a a or a +... + a x b where a a a a a a a a a a th This procedure of elimiatig x, is ow repeated for the third equatio to the equatio to reduce the set of equatios as a x + a + ax +... + a x b a + a x +... + a x b a + a x +... + a x b......... a x + a x +... + a x b This is the ed of the first step of forward elimiatio. Now for the secod step of forward elimiatio, we start with the secod equatio as the pivot equatio ad a as the pivot elemet. So, to elimiate x i the third equatio, oe divides the secod equatio by a (the pivot elemet) ad the multiply it by a. This is the same as multiplyig the secod equatio by a / a ad subtractig it from the third equatio. This makes the coefficiet of x zero i the third equatio. The same procedure is ow repeated for the fourth equatio till th the equatio to give a x + a + ax +... + a x b a + a x +... + a x b a x +... + a x b...... a x +... + a x b
Gaussia Elimiatio 04.06. The ext steps of forward elimiatio are coducted by usig the third equatio as a pivot equatio ad so o. That is, there will be a total of steps of forward elimiatio. At the ed of steps of forward elimiatio, we get a set of equatios that look like ax + a + a x +... + a x b a + a x +... + a x b a x +... + a x b...... ( ) a x ( b ) Back Substitutio: Now the equatios are solved startig from the last equatio as it has oly oe ukow. ( ) b x ( ) a th The the secod last equatio, that is the ( ) equatio, has two ukows: x ad x, th but x is already kow. This reduces the ( ) equatio also to oe ukow. Back substitutio hece ca be represeted for all equatios by the formula ad bi aij j i+ i ( i) aii ( i) ( i) x j x for i,,, x b ( ) ( ) a Example The upward velocity of a rocket is give at three differet times i Table. Table Velocity vs. time data. Time, t (s) Velocity, v (m/s) 5 06.8 8 77. 79. The velocity data is approximated by a polyomial as v ( t) at + at + a, 5 t The coefficiets a, a, ad a for the above expressio are give by
04.06.4 Chapter 04.06 5 5 a 06.8 64 8 a 77. 44 a 79. Fid the values of a, a, ad a usig the Naïve Gauss elimiatio method. Fid the velocity at t 6, 7.5, 9, secods. Solutio Forward Elimiatio of Ukows Sice there are three equatios, there will be two steps of forward elimiatio of ukows. First step Divide Row by 5 ad the multiply it by 64, that is, multiply Row by 64/5.56. ([ 5 5 ] [ 06.8] ). 56 gives Row as [ 64.8.56] [ 7.408] Subtract the result from Row 64 8 77. [ 64.8.56] [ 7.408] 0 4.8.56 96.08 to get the resultig equatios as 5 5 a 06.8 0 4.8.56 a 96.08 44 a 79. Divide Row by 5 ad the multiply it by 44, that is, multiply Row by 44/5 5.76. ([ 5 5 ] [ 06.8] ) 5. 76 gives Row as [ 44 8.8 5.76] [ 65.68] Subtract the result from Row 44 79. [ 44 8.8 5.76] [ 65.68] 0 6.8 4.76 5.968 to get the resultig equatios as 5 5 a 06.8 0 4.8.56 a 96.08 0 6.8 4.76 a 5.968 Secod step We ow divide Row by 4.8 ad the multiply by 6.8, that is, multiply Row by 6.8/ 4.8.5. ([ 0 4.8.56] [ 96.08] ). 5 gives Row as [ 0 6.8 5.46] [ 6.78] Subtract the result from Row
Gaussia Elimiatio 04.06.5 [ 0 6.8 4.76] [ 5.968] [ 0 6.8 5.46] [ 6.78] 0 0 0.7 0.76 to get the resultig equatios as 5 5 a 06.8 0 4.8.56 a 96.08 0 0 0.7 a 0.76 Back substitutio From the third equatio 0.7 a 0.76 0. 76 a 0. 7.0857 Substitutig the value of a i the secod equatio, 4.8a.56a 96.08 96.08 +.56a a 4.8 96.08 +.56.0857 4.8 9.6905 Substitutig the value of a ad a i the first equatio, 5a + 5a + a 06.8 06.8 5a a a 5 06.8 5 9.6905.0857 5 0.9047 Hece the solutio vector is a 0.9047 a 9.6905 a.0857 The polyomial that passes through the three data poits is the v ( t) at + at + a 0.9047t + 9.6905t +.0857, 5 t Sice we wat to fid the velocity at t 6, 7.5, 9 ad secods, we could simply substitute each value of t i v ( t) 0.9047t + 9.6905t +. 0857 ad fid the correspodig velocity. For example, at t 6
04.06.6 Chapter 04.06 v ( 6) 0.9047( 6) + 9.6905( 6) +.0857 9.686 m/s However we could also fid all the eeded values of velocity at t 6, 7.5, 9, secods usig matrix multiplicatio. t v ( t) [ 0. 9047 9. 6905 0857. ] t So if we wat to fid v ( 6), v( 7.5), v( 9), v( ), it is give by [ v( 6) v( 7.5) v( 9) v( ) ] [ 0.9047 9.6905.0857] v ( 6) 9.686 m/s v ( 7.5) 65.04 m/s v ( 9) 0.88 m/s v ( ) 5.88 m/s 6 [ 0.9047 9.6905.0857] 6 7.5 7.5 9 9 6 56.5 6 7.5 5.88 [ 9.686 65.04 0.88 ] 8 9 Example Use Naïve Gauss elimiatio to solve 0x + 5 + 0x 45 x.49 + 7x.75 5x + + x 9 Use six sigificat digits with choppig i your calculatios. Solutio Workig i the matrix form 0 5 0 x.49 7 45 5.75 x 9 Forward Elimiatio of Ukows First step Divide Row by 0 ad the multiply it by, that is, multiply Row by / 0 0. 5. ([ 0 5 0] [ 45] ) 0. 5 gives Row as.5.5 6.75
Gaussia Elimiatio 04.06.7 Subtract the result from Row.49 7 [ ] [.75] [.5.5] [ 6.75] 0 0.00 8.5 8.50 to get the resultig equatios as 0 5 0 x 0 0.00 8.5 45 8.50 5 x 9 Divide Row by 0 ad the multiply it by 5, that is, multiply Row by 5 / 0 0. 5 ([ 0 5 0] [ 45] ) 0. 5 gives Row as [ 5.75.5] [.5] Subtract the result from Row 5 9 [ 5.75.5] [.5] 0.75 0.5.5 to get the resultig equatios as 0 5 0 x 45 0 0.00 8.5 8.50 0.75 0. 5 x. 5 Secod step Now for the secod step of forward elimiatio, we will use Row as the pivot equatio ad elimiate Row : Colum. Divide Row by 0.00 ad the multiply it by.75, that is, multiply Row by.75/ 0.00 750. ([ 0 0.00 8.5] [ 8.50] ) 750 gives Row as [ 0.75 75] [ 77.75] Rewritig withi 6 sigificat digits with choppig [ 0.75 75] [ 77.7] Subtract the result from Row 0.75 0.5.5 [ 0.75 75] [ 77.7] 0 0 75.5 75.45 Rewritig withi 6 sigificat digits with choppig [ 0 0 75.5] [ 75.4] to get the resultig equatios as 0 5 0 x 45 0 0.00 8.5 8.50 0 0 75.5 x 75.4 This is the ed of the forward elimiatio steps.
04.06.8 Chapter 04.06 Back substitutio We ca ow solve the above equatios by back substitutio. From the third equatio, 75.5 x 75.4 x 75.4 75.5 0.999995 Substitutig the value of x i the secod equatio 0.00x + 8.5x 8.50 8.50 8.5x 0.00 8. 50 8. 5 0.999995 0.00 8.50 8.49995 0.00 0.0005 0.00.05 Substitutig the value of x ad x i the first equatio, 0x + 5 + 0x 45 45 5 0x x 0 45 5.05 0 0.999995 0 45 5.75 9.99995 0 9.5 9.99995 0 9.500 0 0.965 Hece the solutio is x [ X ] x 0.965.05 0.999995 Compare this with the exact solutio of
Gaussia Elimiatio 04.06.9 [ X ] x x Are there ay pitfalls of the Naïve Gauss elimiatio method? Yes, there are two pitfalls of the Naïve Gauss elimiatio method. Divisio by zero: It is possible for divisio by zero to occur durig the begiig of the steps of forward elimiatio. For example 5x + 6x 4x + 5 + 7x 6 9x + + x 5 will result i divisio by zero i the first step of forward elimiatio as the coefficiet of x i the first equatio is zero as is evidet whe we write the equatios i matrix form. 0 5 6 x 4 5 7 6 9 x 5 But what about the equatios below: Is divisio by zero a problem? 5x + 6 + 7x 8 0x + + x 5 0x + 7 + 9x 56 Writte i matrix form, 5 6 7 x 8 0 5 0 7 9 x 56 there is o issue of divisio by zero i the first step of forward elimiatio. The pivot elemet is the coefficiet of x i the first equatio, 5, ad that is a o-zero umber. However, at the ed of the first step of forward elimiatio, we get the followig equatios i matrix form 5 6 7 x 8 0 0 0 7 9 x 6 Now at the begiig of the d step of forward elimiatio, the coefficiet of x i Equatio would be used as the pivot elemet. That elemet is zero ad hece would create the divisio by zero problem.
04.06.0 Chapter 04.06 So it is importat to cosider that the possibility of divisio by zero ca occur at the begiig of ay step of forward elimiatio. Roud-off error: The Naïve Gauss elimiatio method is proe to roud-off errors. This is true whe there are large umbers of equatios as errors propagate. Also, if there is subtractio of umbers from each other, it may create large errors. See the example below. Example Remember Example where we used Naïve Gauss elimiatio to solve 0x + 5 + 0x 45 x.49 + 7x.75 5x + + x 9 usig six sigificat digits with choppig i your calculatios? Repeat the problem, but ow use five sigificat digits with choppig i your calculatios. Solutio Writig i the matrix form 0 5 0 x 45.49 7.75 5 x 9 Forward Elimiatio of Ukows First step Divide Row by 0 ad the multiply it by, that is, multiply Row by / 0 0. 5. ([ 0 5 0] [ 45] ) 0. 5 gives Row as [.5.5] [ 6.75] Subtract the result from Row.49 7.75 [.5.5] [ 6.75] 0 0.00 8.5 8.50 to get the resultig equatios as 0 5 0 x 45 0 0.00 8.5 8.50 5 x 9 Divide Row by 0 ad the multiply it by 5, that is, multiply Row by 5 / 0 0. 5. ([ 0 5 0] [ 45] ) 0. 5 gives Row as [ 5.75.5] [.5] Subtract the result from Row 5 9 [ 5.75.5] [.5] 0.75 0.5.5
Gaussia Elimiatio 04.06. to get the resultig equatios as 0 5 0 x 45 0 0.00 8.5 8.50 0.75 0. 5 x. 5 Secod step Now for the secod step of forward elimiatio, we will use Row as the pivot equatio ad elimiate Row : Colum. Divide Row by 0.00 ad the multiply it by.75, that is, multiply Row by.75/ 0.00 750. ([ 0 0.00 8.5] [ 8.50] ) 750 gives Row as [ 0.75 75] [ 77.75] Rewritig withi 5 sigificat digits with choppig [ 0.75 75] [ 77] Subtract the result from Row 0.75 0.5.5 [ 0.75 75] [ 77] 0 0 75 74 Rewritig withi 6 sigificat digits with choppig [ 0 0 75] [ 74] to get the resultig equatios as 0 5 0 x 0 0.00 8.5 45 8.50 0 0 75 x 74 This is the ed of the forward elimiatio steps. Back substitutio We ca ow solve the above equatios by back substitutio. From the third equatio, 75x 74 x 74 75 0.99995 Substitutig the value of x i the secod equatio 0.00x + 8.5x 8.50 8.508.5x 0.00 8. 508. 5 0.99995 0.00 8.508.499575 0.00 8.508.4995 0.00
04.06. Chapter 04.06 0.005 0.00.5 Substitutig the value of x ad 0x + 5 + 0x 45 45 5 0x x 0 45 5.5 0 0.99995 0 45.5 9.9995 0.5 9.9995 0.5005 0.500 0 0.65 Hece the solutio is x [ X ] x 0.65.5 0.99995 Compare this with the exact solutio of x [ X ] x x i the first equatio, What are some techiques for improvig the Naïve Gauss elimiatio method? As see i Example, roud off errors were large whe five sigificat digits were used as opposed to six sigificat digits. Oe method of decreasig the roud-off error would be to use more sigificat digits, that is, use double or quad precisio for represetig the umbers. However, this would ot avoid possible divisio by zero errors i the Naïve Gauss elimiatio method. To avoid divisio by zero as well as reduce (ot elimiate) roud-off error, Gaussia elimiatio with partial pivotig is the method of choice.
Gaussia Elimiatio 04.06. How does Gaussia elimiatio with partial pivotig differ from Naïve Gauss elimiatio? The two methods are the same, except i the begiig of each step of forward elimiatio, a row switchig is doe based o the followig criterio. If there are equatios, the there th are forward elimiatio steps. At the begiig of the k step of forward elimiatio, oe fids the maximum of a kk, a k +, k,, a k The if the maximum of these values is a pk i the th p row, k p, the switch rows p ad k. The other steps of forward elimiatio are the same as the Naïve Gauss elimiatio method. The back substitutio steps stay exactly the same as the Naïve Gauss elimiatio method. Example 4 I the previous two examples, we used Naïve Gauss elimiatio to solve 0x + 5 + 0x 45 x.49 + 7x.75 5x + + x 9 usig five ad six sigificat digits with choppig i the calculatios. Usig five sigificat digits with choppig, the solutio foud was x [ X ] x 0.65.5 0.99995 This is differet from the exact solutio of [ ] x X x Fid the solutio usig Gaussia elimiatio with partial pivotig usig five sigificat digits with choppig i your calculatios. Solutio 0 5 0 x 45.49 7.75 5 x 9
04.06.4 Chapter 04.06 Forward Elimiatio of Ukows Now for the first step of forward elimiatio, the absolute value of the first colum elemets below Row is 0,, 5 or 0,, 5 So the largest absolute value is i the Row. So as per Gaussia elimiatio with partial pivotig, the switch is betwee Row ad Row to give 0 5 0 x 45.49 7.75 5 x 9 Divide Row by 0 ad the multiply it by, that is, multiply Row by / 0 0. 5. ([ 0 5 0] [ 45] ) 0. 5 gives Row as [.5.5] [ 6.75] Subtract the result from Row.49 7.75 [.5.5] [ 6.75] 0 0.00 8.5 8.50 to get the resultig equatios as 0 5 0 x 45 0 0.00 8.5 8.50 5 x 9 Divide Row by 0 ad the multiply it by 5, that is, multiply Row by 5 / 0 0. 5. ([ 0 5 0] [ 45] ) 0. 5 gives Row as [ 5.75.5] [.5] Subtract the result from Row 5 9 [ 5.75.5] [.5] 0.75 0.5.5 to get the resultig equatios as 0 5 0 x 0 0.00 8.5 45 x 8.50 0.75 0. 5 x. 5 This is the ed of the first step of forward elimiatio. Now for the secod step of forward elimiatio, the absolute value of the secod colum elemets below Row is 0.00,. 75 or 0.00,.75
Gaussia Elimiatio 04.06.5 So the largest absolute value is i Row. So Row is switched with Row to give 0 5 0 x 45 0.75 0.5.5 0 0.00 8.5 x 8.50 Divide Row by.75 ad the multiply it by 0.00, that is, multiply Row by 0.00/.75 0.00066. ([ 0.75 0.5] [.5] ) 0. 00066 gives Row as [ 0 0.00099998 0.00088] [ 0.000886] Subtract the result from Row 0 0.00 8.5 8.50 [ 0 0.00099998 0.00088] [ 0.000886] 0 0 8.50088 8.500884 Rewritig withi 5 sigificat digits with choppig [ 0 0 8.500] [ 8.500] to get the resultig equatios as 0 5 0 x 45 0.75 0.5.5 0 0 8.500 x 8.500 Back substitutio 8.500 x 8.500 x 8.500 8.500 Substitutig the value of x i Row.75x + 0.5x.5.5 0.5 x.75.5 0.5.75.5 0.5.75.75.75 Substitutig the value of x ad x i Row 0x + 5 + 0x 45 45 5 0x x 0
04.06.6 Chapter 04.06 45 5 0 0 45 5 0 0 0 0 0 0 0 So the solutio is x [ X ] x This, i fact, is the exact solutio. By coicidece oly, i this case, the roud-off error is fully removed. Ca we use Naïve Gauss elimiatio methods to fid the determiat of a square matrix? Oe of the more efficiet ways to fid the determiat of a square matrix is by takig advatage of the followig two theorems o a determiat of matrices coupled with Naïve Gauss elimiatio. Theorem : Let [A] be a matrix. The, if [B] is a matrix that results from addig or subtractig a multiple of oe row to aother row, the det( A ) det( B) (The same is true for colum operatios also). Theorem : Let [A] be a matrix that is upper triagular, lower triagular or diagoal, the det( A ) a a... a ii... a a ii i This implies that if we apply the forward elimiatio steps of the Naïve Gauss elimiatio method, the determiat of the matrix stays the same accordig to Theorem. The sice at the ed of the forward elimiatio steps, the resultig matrix is upper triagular, the determiat will be give by Theorem.
Gaussia Elimiatio 04.06.7 Example 5 Fid the determiat of 5 5 [A] 64 8 44 Solutio Remember i Example, we coducted the steps of forward elimiatio of ukows usig the Naïve Gauss elimiatio method o [A] to give 5 5 [ B ] 0 4.8.56 0 0 0.7 Accordig to Theorem det( A ) det( B) 5 ( 4.8) 0.7 84.00 What if I caot fid the determiat of the matrix usig the Naïve Gauss elimiatio method, for example, if I get divisio by zero problems durig the Naïve Gauss elimiatio method? Well, you ca apply Gaussia elimiatio with partial pivotig. However, the determiat of the resultig upper triagular matrix may differ by a sig. The followig theorem applies i additio to the previous two to fid the determiat of a square matrix. Theorem : Let [A] be a matrix. The, if [B] is a matrix that results from switchig oe row with aother row, the det( B) det( A). Example 6 Fid the determiat of 0 7 0 [A].099 6 5 5 Solutio The ed of the forward elimiatio steps of Gaussia elimiatio with partial pivotig, we would obtai 0 7 0 [B] 0.5 5 0 0 6.00 det B 0.5 6. ( ) 00
04.06.8 Chapter 04.06 50.05 Sice rows were switched oce durig the forward elimiatio steps of Gaussia elimiatio with partial pivotig, det( A) det( B) 50.05 Example 7 Prove Solutio det( A ) [ A][ A] det det det det ( A ) ( A A ) det ( I ) ( A) det( A ) ( A) [ I] det ( A ) matrix ad det( A ) 0 If [A] is a, what other statemets are equivalet to it?. [A] is ivertible.. [ A ] exists.. [ A ][ X ] [ C] has a uique solutio. 4. [ A ][ X ] [0] solutio is [ X ] [0]. 5. [ A][ A] [ I] [ A] [ A]. Key Terms: Naïve Gauss Elimiatio Partial Pivotig Determiat