ITERATION METHODS. These are methods which compute a sequence of progressively accurate iterates to approximate the solution

Transcription

1 ITERATION METHODS These are methods whch compute a sequence of progressvely accurate terates to approxmate the soluton of Ax = b. We need such methods for solvng many large lnear systems. Sometmes the matrx s too large to be stored n the computer memory, makng a drect method too dffcult to use. More mportantly, the operatons cost of n for Gaussan elmnaton s too large for most large systems. Wth teraton methods, the cost can often be reduced to somethng of cost O ³ n or less. Even when a specal form for A canbeusedtoreducethe cost of elmnaton, teraton wll often be faster. There are other, more subtle, reasons, whch we do not dscuss here.

2 JACOBI S ITERATION METHOD We begn wth an example. Consder the lnear system 9x + x + x = b x + 0x + x = b x + 4x + x = b In equaton #k, solveforx k : x = 9 [b x x ] x = 0 [b x x ] x = [b x 4x ] Let x (0) = x (0) T,x(0),x(0) be an ntal guess to the soluton x. Thendefne x (k+) = 9 x (k+) = 0 x (k+) = b x (k) x (k) b x (k) x (k) b x (k) 4x (k) for k =0,,,...ThsscalledtheJacob teraton method or the method of smultaneous replacements.

3 NUMERICAL EXAMPLE. Let b =[0, 9, 0] T. The soluton s x =[,, ] T. To measure the error, we use Error = kx x (k) k =max x x (k) k Error Rato E E E E E E E E E E E E E x (k) x (k) x (k)

4 GAUSS-SEIDEL ITERATION METHOD Agan consder the lnear system 9x + x + x = b x + 0x + x = b x + 4x + x = b and solve for x k n equaton #k: x = 9 [b x x ] x = 0 [b x x ] x = [b x 4x ] Now mmedately use every new terate: x (k+) = 9 x (k+) = 0 x (k+) = b x (k) x (k) b x (k+) x (k) b x (k+) 4x (k+) for k = 0,,,... Ths s called the Gauss-Sedel teraton method or the method of successve replacements.

5 NUMERICAL EXAMPLE. Let b =[0, 9, 0] T. The soluton s x =[,, ] T. To measure the error, we use Error = kx x (k) k =max x x (k) k Error Rato E E E E E E E x (k) x (k) x (k) The values of Rato do not approach a lmtng value wth larger values of the teraton ndex k.

6 A GENERAL SCHEMA Rewrte Ax = b as Nx = b + Px () wth A = N P a splttng of A. Choose N to be nonsngular. Usually we want Nz = f to be easly solvable for arbtray f. The teraton method s Nx (k+) = b + Px (k), k =0,,,..., () EXAMPLE. LetN be the dagonal of A, andletp = N A. The teraton method s the Jacob method: a, x (k+) for k =0,,... = b nx j= j6= a,j x (k) j, n

7 EXAMPLE. LetN be the lower trangular part of A, ncludng ts dagonal, and let P = N A. The teraton method s the Gauss-Sedel method: X j= a,j x (k+) j for k =0,,... = b nx j=+ a,j x (k) j, n EXAMPLE. Another method could be defned by lettng N be the trdagonal matrx formed from the dagonal, super-dagonal, and sub-dagonal of A, wth P = N A: N = a, a, 0 0 a, a, a, a n,n a n,n a n,n 0 0 a n,n a n,n Solvng Nx (k+) = b + Px (k) uses the algorthm for trdagonal systems from 6.4.

8 CONVERGENCE When does the teraton method () converge? Subtract () from (), obtanng N ³ x x (k+) = P ³ x x (k) x x (k+) = N P ³ x x (k) e (k+) = Me (k), M = N P () wth e (k) x x (k) Return now to the matrx and vector norms of 6.5. Then e (k+) e kmk (k), k 0 Thus the error e (k) converges to zero f kmk <, wth e (k) kmk k e (0), k 0

9 EXAMPLE. For the earler example wth the Jacob method, x (k+) = 9 x (k+) = 0 x (k+) = b x (k) x (k) b x (k) x (k) b x (k) 4x (k) M = kmk = =0.66 Ths s consstent wth the earler table of values, although the actual convergence rate was better than predcted by ().

10 EXAMPLE. For the earler example wth the Gauss- Sedel method, x (k+) = 9 x (k+) = 0 x (k+) = b x (k) x (k) b x (k+) x (k) b x (k+) 4x (k+) M = = kmk = Ths too s consstent wth the earler numercal results.

11 DIAGONALLY DOMINANT MATRICES Matrces A for whch X n a, > a,j, j= j6= =,...,n are called dagonally domnant. FortheJacobtera- ton method, M = 0 a, a,n a, a, a, 0 a,n a, a,..... a n, a n,n 0 a n,n a n,n Wth dagonally domnant matrces A, kmk = max n nx j= j6= a,j a, < (4) Thus the Jacob teraton method for solvng Ax = b s convergent.

12 GAUSS-SEIDEL ITERATION Assumng A s dagonally domnant, we can show that the Gauss-Sedel teraton wll also converge. However, constructng M = N P s not reasonable for ths method and an alternatve approach s needed. Return to the error equaton Ne (k+) = Pe (k) and wrte t n component form for the Gauss-Sedel method: X j= a,j e (k+) j = nx j=+ a,j e (k) j, n e (k+) = X j= a,j e (k+) a j, nx j=+ a,j a, e (k) j (5)

13 Introduce α = X a,j j= a,, β = nx a,j j=+ a,, wth α = β n =0. Takngboundsn(5), e(k+) α Let be an ndex for whch e(k+) n e (k+) +β e (k), =,..., n (6) n e(k+) = = max e (k+) Then usng = n (6), e (k+) e α (k+) e + β (k) Defne Then e (k+) β α η =max β α e (k) e (k+) η e (k)

14 For A dagonally domnant, t can be shown that η kmk (7) where kmk s for the defnton of M for the Jacob method, gven earler n (4) as kmk = max n nx j= j6= a,j a, = max (α + β ) < n Consequently, for A dagonally domnant, the Gauss- Sedel method also converges and t does so more rapdly than the Jacob method n most cases. Showng (7) follows by showng β (α + β α ) 0, n For our earler example wth A of order, we have µ =0.75 ThssnotasgoodascomputngkMk drectly for the Gauss-Sedel method, but t does show that the rate of convergence s better than for the Jacob method.

15 Snce CONVERGENCE: AN ADDENDUM kmk = kn P k kn kkp k, kmk < ssatsfed f N satsfes kn kkpk < Usng P = N A, ths can be rewrtten as ka Nk < kn k We also want to choose N so that systems Nz = f s easly solvable. GENERAL CONVERGENCE THEOREM: Nx (k+) = b + Px (k), k =0,,,..., wll converge, for all rght sdes b and all ntal guesses x (0), f and only f all egenvalues λ of M = N P satsfy λ < Ths s the bass of dervng other splttngs A = N P that lead to convergent teraton methods.

16 RESIDUAL CORRECTION METHODS N be an nvertble approxmaton of the matrx A; let x (0) x for the soluton of Ax = b. Defne r (0) = b Ax (0) Snce Ax = b for the true soluton x, r (0) = Ax Ax (0) = A(x x (0) )=Ae (0) wth e (0) = x x (0).Letê (0) be the soluton of andthendefne Nê (0) = r (0) x () = x (0) +ê (0) Repeat ths process nductvely.

17 RESIDUAL CORRECTION For k =0,,..., defne r (k) = b Ax (k) Nê (k) = r (k) x (k+) = x (k) +ê (k) Ths s the general resdual correcton method. To see how ths fts nto our earler framework, proceed as follows: Thus, x (k+) = x (k) +ê (k) = x (k) + N r (k) = x (k) + N (b Ax (k) ) Nx (k+) = Nx (k) + b Ax (k) = b +(N A)x (k) = b + Px (k) Sometmes the resdual correcton scheme s a preferable way of approachng the development of an teratve method.