2 Elimination methods for solving linear equations

Transcription

1 Elimination methos for solving linear equations Elimination methos for solving linear equations.... Overview.... Gaussian elimination Writing the program - Gaussian elimination Writing the program - Back substitution Partial pivoting....4 Why we shouln t use this algorithm?....5 Problems with Gaussian elimination....6 LU ecomposition What happens if we have to o a row swap uring the Gaussian elimination? recipe for oing LU ecomposition Summary....8 ppenix: Coe to o Gaussian elimination with partial pivoting... Overview In this lecture we will take another look at linear equations. ore importantly, we are going to start using computers to o the har work for us. - -

2 In this lecture, we are going to write a program, which can solve a set of linear equations, using the metho of Gaussian elimination. The program you will meet in this lecture will contain the following elements. We want to write a program (a program or subprogram is referre to as a function in atlab), which will take a matrix, an a right-han sie vector b as inputs, an give back a vector which contains the solution to xb. function [x] GaussianEliminate(, b) %work out the number of equations N length(b) comment return We nee to write our program in a text file an make sure that it has the extension GaussianEliminate.m. To call the program in atlab, we use woul write in the comman line: GaussianEliminate(atrix, RightHanSie) Note : in these lecture notes, computer coe will in a ifferent font, an enclose in a box. - -

3 . Gaussian elimination You will have probably seen Gaussian elimination before, so hopefully this will just be a review. Here we will attempt to write an algorithm which can be performe by a computer. lso, for the time being we will assume that we have a square system of equations with a unique solution to our equations. robust algorithm woul check to make sure there was a solution before proceeing. Lets consier the system of equations x b (-) 3 3 x x x 3 b b b We coul write this in augmente form as 3 3 b b b 3 3 (-) (-3) Whilst we go through the metho of solving these equations in the hanout, we will go through (in parallel, on the boar) the following example: x x x 3-3 -

4 Now we can o our row operations (equivalent to aing multiples of equations together to eliminate variables, or if you prefer, to multiplying both sies of Eq. - by an elementary matrix which performs the same row operation). Now we begin trying to reuce our system of equations to a simpler form. Using row, we can eliminate element by subtracting / times row from row. Row is the pivot row an is the pivot element 3 3 b b b 3 (-3) The following operations -> (,)/(,) -> -> 3 b ->b b give 3 3 b b b 3 (,) (,) (,)* (,) (,) (,)* (,3) (,3) (,3)* b() b() b()* (-4) Similarly, we can perform a similar operation to eliminate 3 by subtracting 3 / 3 times row from row

5 3 -> 3 3 -> 3 -> 3 3 b 3 ->b 3 b 3 b b b 3 (3,)/(,) (3,) (3,) (,)* (3,) (3,) (,)* (3,3) (3,3) (,3)* b(3) b(3) b()* (-5) Having mae all the elements below the pivot in this column zero, we move onto the next column. The pivot is now the element Now we can eliminate using row / (3,)/(,) -> -> (3,) (3,) (,)* (3,3) (3,3) (,3)* b 3 ->b 3 b b(3) b(3) b()* giving 3 b b b 3 (-6) Now that the system of equations is in a triangular form, the solution can reaily obtaine by back-substitution. Notice, that to get to this stage, we worke on each column in turn, eliminating all the elements below the iagonal (pivot) element

6 Now we have an upper triangular system, we can use back substitution x 3 b 3 / x(3) b(3)/(3,3); x (b - x 3 )/ x() (b()-(,3)*x(3))/(,); x (b - x - 3 x 3 )/ x() (b()-(,)*x()- (,3)*x(3))/(,);.. Writing the program - Gaussian elimination Rather than type out each comman into atlab, we can use "for loops", to loop through these operations. We can also take avantage of atlab s ability to work with whole rows an columns of a matrix. If you want to access an entire row or column of a matrix, you can use o the following: (,:) [,, 3 ] (:,) [,, ] T or parts of a matrix (,:en) [, 3 ] So a row operation, e.g. subtracting *row from row i is: (i,:) (i,:) * (,:) We nee two loops, one to loop through the columns (we fin the iagonal element on each column an eliminate each element below it) an an inner loop to go through each row uner the pivot element. for column:(n-) %work on all the rows below the iagonal element for row (column+):n % the row operation goes here en%loop through rows en %loop through columns - 6 -

7 If we a the coe to o a row operation within these loops our program looks like: function [x] GaussianEliminate(, b) %work out the number of equations N length(b) %Gaussian elimination for column:(n-) %work on all the rows below the iagonal element for row (column+):n %work out the value of (row,column)/(column,column); %o the row operation (row,:) (row,:)-*(column,:) b(row) b(row)-*b(column) en%loop through rows en %loop through columns return We still havent finishe, but we are half way there

8 .. Writing the program - Back substitution For the back substitution, we start at the bottom row an work upwars. For each row we nee to o: x i, i bi jx j j i+ to N i,i We can write a for loop to accomplish the back-substitution %back substitution for rown:-: x(row) b(row); for i(row+):n x(row) x(row)-(row,i)*x(i); en x(row) x(row)/(row,row); en ing the back substitution the main program gives: - 8 -

9 function [x] GaussianEliminate(, b) %work out the number of equations N length(b) %Gaussian elimination for column:(n-) %work on all the rows below the iagonal element for row (column+):n %work out the value of (row,column)/(column,column); %o the row operation (row,:) (row,:)-*(column,:) b(row) b(row)-*b(column) en%loop through rows en %loop through columns %back substitution for rown:-: x(row) b(row); for i(row+):n x(row) x(row)-(row,i)*x(i); en x(row) x(row)/(row,row); en %return the answer x x ; return - 9 -

10 .3 Partial pivoting This basically means row swapping. Before performing elimination on a column, it is avantageous to swap rows an promote the row with the largest value in that column to the iagonal. e.g. if our matrix is 3 Our algorithm woul fail (ivie by zero). But we can swap row, with row, which has the largest absolute value in this column. (note that we must also swap the rows in the RHS vector b) 3 Now our algorithm can procee as normal. If we always swap rows to move the largest element to iagonal, the values of ij will always be less than one. This reuces the amount of accumulate numerical error in the calculation. (Note: we woul also have to swap the rows in the right han sie vector too, since all we are oing is re-orering the equations) - -

11 .4 Why we shouln t use this algorithm? There any several reasons why we shoul not use this algorithm.. atlab will compute the solution to x b using its own (more efficient solvers) with the comman \b. Our algorithm contains lots of for loops. peculiarity of atlab makes for loops very slow, but vector or matrix operations very fast. The vector or matrix operations are compile. However, each step in our algorithm (incluing each loop of the for loop) must first be interprete before execution. 3. There are funamental problems with Gaussian elimination. - -

12 .5 Problems with Gaussian elimination We can moify the algorithm to work out how many subtraction + multiplication operations it performs for a given size of matrix. NumOperations ; %Gaussian elimination for column:(n-) %work on all the rows below the iagonal element for row (column+):n %work out the value of (row,column)/(column,column); %o the row operation (row,column:en) (row,column:en)-... *(column,column:en); b(row) b(row)-*b(column); NumOperations NumOperations+N-column+; en%loop through rows en %loop through columns - -

13 35 3 Gaussian Elimination Back substitution Number of operations 5 5 y.*x 3 -.*x 5 y.5*x -.5*x Number of equations We fin that the number of operations require to perform Gaussian elimination is O(N 3 ), where N is the number of equations. For large systems of equations Gaussian elimination is very inefficient even for the fastest super computer! On the other han, the number of operations require for back-substitution scales with O(N ). Back substitution is much more efficient for large systems than Gaussian elimination

14 .6 LU ecomposition Suppose we want to solve the same set of equations but with several ifferent right han sies. We on t want to use the expense of Gaussian elimination every time we solve the equations, ieally we woul like to only o the Gaussian elimination once. e.g. x b, x b, x 3 b 3 which may be written as 3 3 b b b x x x If by Gaussian elimination, we coul factor our matrix into two matrices, L an U, such that LU * * * * * * * * * 3 3, (-7) we coul then solve for each right han sie using only forwar, followe by backwar substitution

15 x b LUx b let y Ux Ly b Then solve Ux y (Forwar substitution) (Backwar substitution) Lets start with the matrix from our previous example When we eliminate the element (subtract / times row from row ), we can keep multiplying by a matrix that unoes this row operation, so that the prouct remains equal to When we eliminate the element 3 (subtract 3 3 / times row from row 3), we can multiply by a matrix that unoes this row operation, so that the prouct remains equal to

16 When we eliminate the element (subtract / times row from row 3), we can keep multiply by a matrix that unoes this row operation, so that the prouct remains equal to I.e. we have performe LU ecomposition. In practice we ont bother recoring the matrices, we can just write own the matrix L

17 .6. What happens if we have to o a row swap uring the Gaussian elimination? Suppose we ha reache the following stage in the elimination process This is a permutation matrix an we neee to exchange rows an ultiplying by a permutation matrix will swap the rows of a matrix. The permutation matrix is just an ientity matrix, whose rows have been interchange. We can then continue as normal

18 .6. recipe for oing LU ecomposition. Write own a permutation matrix, P (which is initially the ientity matrix) P. Write own the atrix you want to ecompose 3. Starting with column, row swap to promote the largest value in the column to the iagonal. Do exactly the same row swap to the atrix P. P 4. Eliminate all the elements below the iagonal in column. Recor the multiplier use for elimination where you create the zero. Here we i row 3 row 3 -.5*row

19 5. ove onto the next column. Swap rows to move the largest element to the iagonal. (Do the same row swap to P) P 6. Eliminate the elements below the iagonal row 3 row 3 - /3*row /3 4/3 7. Repeat 5 an 6 for all columns 8. Write own L, U an P.5.5 /3.5 4/3 implies U.5.5 L 4/3.5 /3 an P - 9 -

20 .7 Summary Gaussian elimination can be slow (the number of operations require to o Gaussian elimination is O(N 3 ) where N is the number of equations. Back substitution only takes O(N ) If we want to solve the set of equations repeately with ifferent right han sies, LU ecomposition means that we ont have to o Gaussian elimination every time. This is a consierable saving. atlab has inbuilt atlab routines for solving linear equations (\) an LU ecomposition (LU). They will generally be better than any you write yourself - -

21 .8 ppenix: Coe to o Gaussian elimination with partial pivoting function [x] GaussianEliminate(,b) % Solves x b by Gaussian elimination %work out the number of equations N length(b); %Gaussian elimination for column:(n-) %swap rows so that the row we are using to eliminate %the entries in the rows below is larger than the %values to be eliminate. [ummy,inex] max(abs((column:en,column))); inexinex+column-; temp (column,:); (column,:) (inex,:); (inex,:) temp; temp b(column) b(column) b(inex); b(inex) temp; %work on all the rows below the iagonal element for row (column+):n %work out the value of (row,column)/(column,column); %o the row operation (result isplaye on screen) (row,column:en) (row,column:en)-*(column,column:en) ; b(row) b(row)-*b(column); en%loop through rows en %loop through columns %back substitution for rown:-: x(row) b(row); for i(row+):n x(row) x(row)-(row,i)*x(i); en x(row) x(row)/(row,row); en x x return - -