M ULTIGRID C OMPUTING Wy Multigrid Metods Are So Efficient Originally introduced as a way to numerically solve elliptic boundary-value problems, multigrid metods, and teir various multiscale descendants, ave since been developed and applied to various problems in many disciplines. Tis introductory article provides te basic concepts and metods of analysis and outlines some of te difficulties of developing efficient multigrid algoritms. 5-965/06/$0.00 006 IEEE Copublised by te IEEE CS and te AIP IRAD YAVNEH Multigrid computational metods are well known for being te fastest numerical metods for solving elliptic boundary-value problems. Over te past 30 years, multigrid metods ave earned a reputation as an efficient and versatile approac for oter types of computational problems as well, including oter types of partial differential equations and systems and some integral equations. In addition, researcers ave successfully developed generalizations of te ideas underlying multigrid metods for problems in various disciplines. (See te Multigrid Metods Resources sidebar for more details.) Tis introductory article presents te fundamentals of multigrid metods, including explicit algoritms, and points out some of te main pitfalls using elementary model problems. Tis material is mostly intended for readers wo ave a practical interest in computational metods but little or no acquaintance wit multigrid tecniques. Te article also provides some background for tis special issue and oter, more advanced publications. Tecnion Israel Institute of Tecnology Basic Concepts Let s begin wit a simple example based on a problem studied in a different context. Global versus Local Processes Te ometown team as won te regional cup. Te players are to receive medals, so it s up to te coac to line tem up, equally spaced, along te goal line. Alas, te field is muddy. Te coac, wo abors mud, tries to accomplis te task from afar. He begins by numbering te players 0 to N and orders players 0 and N to stand by te left and rigt goal posts, respectively, wic are a distance L apart. Now, te coac could, for example, order player i, i =,, N, to move to te point on te goal line tat is at a distance il/n from te left goal post. Tis would be a global process tat would solve te problem directly. However, it would require eac player to recognize te left-and goal post, perform some fancy aritmetic, and gauge long distances accurately. Suspecting tat is players aren t up to te task, te coac reasons tat if eac player were to stand at te center point between is two neigbors (wit players 0 and N fixed at te goal posts), is task would be accomplised. From te local rule, a global order will emerge, e pilosopizes. Wit tis in mind, te coac devises te following simple iterative local process. At eac iteration, e blows is wistle, and player i, i =,, N, COMPUTING IN SCIENCE & ENGINEERING
MULTIGRID METHODS RESOURCES Radii Petrovic Fedorenko, and Nikolai Sergeevitc Bakvalov 3 first conceived multigrid metods in te 960s, but Aci Brandt first acieved teir practical utility and efficiency for general problems in is pioneering work in te 970s. 4 6 Classical texts on multigrid metods include an excellent collection of papers edited by Wolfgang Hackbusc and Ulric Trottenberg, 7 Brandt s guide to multigrid metods, 8 and te classical book by Hackbusc. 9 Notable recent textbooks on multigrid include te introductory tutorial by William Briggs and is colleagues 0 and a compreensive textbook by Trottenberg and is colleagues. Many oter excellent books and numerous papers are also available, mostly devoted to more specific aspects of multigrid tecniques, bot teoretical and practical. References. R.P. Fedorenko, A Relaxation Metod for Solving Elliptic Difference Equations, USSR Computational Matematics and Matematical Pysics, vol., no. 4, 96, pp. 09 096.. R.P. Fedorenko, Te Speed of Convergence of One Iterative Process, USSR Computational Matematics and Matematical Pysics, vol. 4, no. 3, 964, pp. 7 35. 3. N.S. Bakvalov, On te Convergence of a Relaxation Metod wit Natural Constraints on te Elliptic Operator, USSR Computational Matematics and Matematical Pysics, vol. 6, no. 5, 966, pp. 0 35. 4. A. Brandt, A Multi-Level Adaptive Tecnique (MLAT) for Fast Numerical Solution to Boundary Value Problems, Proc. 3rd Int l Conf. Numerical Metods in Fluid Mecanics, H. Cabannes and R. Temam, eds., Lecture Notes in Pysics 8, Springer, 973, pp. 8 89. 5. A. Brandt, Multi-Level Adaptive Tecniques (MLAT): Te Multigrid Metod, researc report RC 606, IBM T.J. Watson Researc Ctr., 976. 6. A. Brandt, Multi-Level Adaptive Solutions to Boundary-Value Problems, Matematics of Computation, vol. 3, no. 38, 977, pp. 333 390. 7. W. Hackbusc and U. Trottenberg, eds., Multigrid Metods, Springer- Verlag, 98. 8. A. Brandt, 984 Multigrid Guide wit Applications to Fluid Dynamics, GMD- Studie 85, GMD-FIT, Postfac 40, D-505, St. Augustin, West Germany, 985. 9. W. Hackbusc, Multi-Grid Metods and Applications, Springer, 985. 0. W.L. Briggs, V.E. Henson, and S.F. McCormick, A Multigrid Tutorial, nd ed., SIAM Press, 000.. U. Trottenberg, C.W. Oosterlee, and A. Scüller, Multigrid, Academic Press, 00. (a) (b) (c) (d) (e) (f) Figure. Jacobi s player-alignment iteration. Red disks sow te current position, and blue disks sow te previous position, before te last wistle blow: (a) initial position, (b) one wistle blow, (c) slow convergence, (d) fast convergence, (e) oversoot, and (f) damped iteration. moves to te point tat s at te center between te positions of player i and player i + just before te wistle was blown. Tese iterations are repeated until te errors in te players positions are so small tat te mayor, wo will be pinning te medals, won t notice. Tis is te convergence criterion. Undeniably, te coac deserves a name; we ll call im Mr. Jacobi. Figure a sows te players initial apazard position, and Figure b te position after one iteration (wistle blow). (For clarity, we assume tat te players aren t even on te goal line initially. Tis pessimistic view does not alter our analysis significantly. We can envision purely lateral movements by projecting te players positions onto te goal line.) As I ll sow, Jacobi s metod is con- NOVEMBER/DECEMBER 006 3
(a) (b) (c) Figure. Multiscale player alignment. Red disks sow te current position, and blue disks sow te previous position, before te last wistle blow: (a) large, (b) medium, and (c) small scales. vergent tat is, te errors in te players positions will eventually be as small as we wis to make tem. However, tis migt take a long time. Suppose te players positions are as sown in Figure c. In tis case, te players move sort distances at eac iteration. Sadly, tis slow crawl toward convergence will persist for te duration of te process. In contrast, convergence is obtained in a single iteration in te position sown in Figure d. Wat is te essential property tat makes te convergence slow in te position of Figure c, yet fast in Figure d? In Figure c, we ave a largescale error. But Jacobi s process is local, employing only small-scale information tat is, eac player s new position is determined by is near neigborood. Tus, to eac player, it seems (from is myopic viewpoint) tat te error in position is small, wen in fact it isn t. Te point is tat we cannot correct wat we cannot detect, and a large-scale error can t be detected locally. In contrast, Figure d sows a small-scale error position, wic can be effectively detected and corrected by using te local process. Finally, consider te position in Figure e. It clearly as a small-scale error, but Jacobi s process doesn t reduce it effectively; rater, it introduces an oversoot. Not every local process is suitable. However, tis problem is easily overcome witout compromising te process s local nature by introducing a constant damping factor. Tat is, eac player moves only a fraction of te way toward is designated target at eac iteration; in Figure f, for example, te players movements are damped by a factor /3. Multiscale Concept Te main idea beind multigrid metods is to use te simple local process but to do so at all scales of te problem. For convenience, assume N to be a power of two. In our simple example, te coac begins wit a large-scale view in wic te problem consists only of te players numbered 0, N/, and N. A wistle blow puts player N/ alfway between is large-scale neigbors (see Figure a). Next, addressing te intermediate scale, players N/4 and 3N/4 are activated, and tey move at te sound of te wistle to te midpoint between teir mid-scale neigbors (see Figure b). Finally, te small scale is solved by Jacobi s iteration at te remaining locations (Figure c). Tus, in just tree wistle blows generally, log (N) and only N individual moves, te problem is solved, even toug te simple local process as been used. Because te players can move simultaneously at eac iteration in parallel, so to speak te number of wistle blows is important in determining te overall time, and log (N) is encouraging. For million players, for example, Jacobi s process would require a mere 0 wistle blows. Researcers ave successfully exploited te idea of applying a local process at different scales in many fields and applications. Our simple scenario displays te concept but ides te difficulties. In general, developing a multiscale solver for a given problem involves four main tasks: coosing an appropriate local process, coosing appropriate coarse (large-scale) variables, coosing appropriate metods for transferring information across scales, and developing appropriate equations (or processes) for te coarse variables. Depending on te application, eac of tese tasks can be simple, moderately difficult, or extremely callenging. Tis is partly te reason wy multigrid continues to be an active and triving researc field. We can examine tese four tasks for developing a multiscale solver by using a sligtly less trivial problem. Before doing tis, let s analyze Jacobi s player-alignment algoritm and give some matematical substance to our intuitive notions on te dependence of its efficiency on te scale of te error. 4 COMPUTING IN SCIENCE & ENGINEERING
(a) (b) (c) Figure 3. Tree eigenvectors. We associate scale wit te wavelengt, wic is inversely proportional to k: (a) v (), (b) v (3), and (c) v (6). Analyzing Jacobi s Player-Alignment Algoritm First, observe tat Jacobi s iteration acts similarly and independently in te directions parallel and perpendicular to te goal line. Terefore, it suffices to consider te convergence wit respect to just one of tese directions. Accordingly, let te column ( 0) ( 0) ( 0) T vector e = [ e,..., e N ] denote te players initial errors, eiter in te direction perpendicular or parallel to te goal line. We can define te error as te player s present coordinate, subtracted from te correct coordinate (to wic e sould eventually converge.) For convenience, we don t include players 0 and N in tis vector, but we keep in mind ( 0) ( 0) tat e 0 = e N = 0. For n =,,, te new error in te location of player i after iteration n is given by te average of te errors of is two neigbors just before te iteration namely, ( n) ( n ) ( e e e n ) i = i + + i. () We can write tis in vector-matrix form as e (n) = Re (n ), () were R is an N by N tridiagonal matrix wit one-alf on te upper and lower diagonals and 0 on te main diagonal. For example, for N = 6, we ave 0 0 0 0 0 0 0 R = 0 0 0. 0 0 0 0 0 0 0 (3) Te beavior of te iteration and its relation to scale is most clearly observed for an error, e, wic is an eigenvector of R, satisfying Re = e, for some (scalar) eigenvalue,. Te matrix R as N eigenvalues; for k =,, N, tey are given by ( k) kπ λ = cos. (4) Te N corresponding eigenvectors are given by kπ sin kπ ( k) sin v =. N k sin ( ) π (To verify tis, use te trigonometric identity, (5) ki ( + ) π ki sin sin + π = kπ kiπ cos sin to sow tat Rv (k) = (k) v (k).) Figure 3 sows te first, tird, and sixt eigenvectors. We associate scale wit te wavelengt, wic is inversely proportional to k. Tus, small k corresponds to largescale eigenvectors, and vice versa. Te eigenvectors v (k) are linearly independent, spanning te space R N, so we can write any initial error as a linear combination of te eigenvectors, N ( 0) ( k) e = c k v, (6) k= for some real coefficients, c k. Repeated multiplication by te iteration matrix, R, yields te error after n iterations: N ( n) n ( 0) n ( k) e = R e = R ckv k= N N n ( k) ( k) n ( k) = ckr v = ck( λ ) v k= k= N n kπ k = ck cos N v. k= (7) Because (k) = cos(k /N) < for k N, NOVEMBER/DECEMBER 006 5
(a) (b) (c) Figure 4. Te smooting property of te damped Jacobi relaxation. (a) Initial position, (b) after four relaxations, and (c) after eigt relaxations. we conclude tat Jacobi s metod converges because (k) n tends to zero as n tends to infinity. It s illuminating to examine te rate of convergence for different values of k. For k/n <<, we ave ( k) n kπ ( λ ) = cos n n kπ N e, (8) were we can verify te approximation by using a Taylor series expansion. We find tat te coefficients of eigenvectors corresponding to small k values cange only sligtly in eac iteration (see Figure c). Indeed, we see ere tat n = O(N ) iterations are required to reduce suc an error by an order of magnitude. For million players, for example, undreds of billions of wistle blows are required, compared to just 0 using te multiscale approac. For k = N/, (k) = cos(k /N) = 0, and convergence is immediate (see Figure d). As k approaces N, te scale of te error becomes even smaller, and cos(k /N) approaces, wic is consistent wit te oversoot we saw in Figure e tat is, at eac iteration, te error mainly canges sign, wit ardly any amplitude cange. Damped Jacobi Iteration s Smooting Factor To overcome te oversoot problem, we introduce a damping factor. If we damp te players movements by a factor /3, we obtain ( n) ( n ) ( n ) ( ei ei ei e n ) = + + + i 3 3 and in vector-matrix form,, (9) ( n) ( n ) ( n ) e = Rde I + e, (0) R 3 3 wit R d denoting te damped Jacobi iteration matrix and I te identity matrix. Te eigenvectors clearly remain te same, but te eigenvalues of R d are given by ( k) ( k) kπ λd = + λ = + cos. () 3 3 3 3 We can now quantify te claim tat Jacobi s iteration, damped by a factor /3, is efficient in reducing small-scale errors. To do tis, let s (somewat arbitrarily at tis point) split te spectrum down te middle, rougly, and declare all eigenvectors wit k < N/ smoot (large scale), and eigenvectors wit N/ k < N roug (small scale). Observe tat, as we vary k from N/ to N, spanning te roug eigenvectors, cos(k /N) ( k) varies from 0 to (nearly), and terefore λ d varies from /3 to (nearly) /3. Te fact tat ( k) λ d /3 for all te roug eigenvectors implies tat te size of te coefficient of eac roug eigenvector is reduced in eac iteration to at most /3 its size prior to te iteration independently of N. We say tat te iteration s smooting factor is /3. If we begin wit an initial error tat s some apazard combination of te eigenvectors, ten after just a few iterations, te coefficients of all te roug eigenvectors will be greatly reduced. Tus, te damped Jacobi iteration (usually called relaxation) is efficient at smooting te error, altoug, as we ve seen, it is inefficient at reducing te error once it as been smooted. Figure 4 exibits tese properties. Relaxation, one observes, irons out te wrinkles but leaves te fat. D Model Problem Te idea of using a local rule to obtain a global order is, of course, ancient. Peraps te most common example in matematics is differential equations, wic ave a global solution determined by a local rule on te derivatives (plus boundary conditions). To advance our discussion, we require an example tat s sligtly less trivial tan player alignment. Consider te D boundaryvalue problem: 6 COMPUTING IN SCIENCE & ENGINEERING
d u Lu = f( x), x in ( 0, ), dx u( 0) = ul, u( ) = u r, () were f is a given function, u and u r are te boundary conditions, and u is te solution we re seeking. We adopt te common numerical approac of finite difference discretization. A uniform mes is defined on te interval [0, ], wit mes size = /N and N + mes points given by x i + i for i = 0,,..., N. Our vector of unknowns te discrete approximation to u is denoted as u = [ u,... un ] T, were we exclude te boundary values for convenience but keep in mind tat u0 = ul, un = u. Te rigt-and side vector is denoted by f r = [ f,..., fn ] T, wit fi = f( xi ). Alternatively, we could use some local averaging of f. Te standard second-order accurate finite difference discretization of te second derivative now yields a system of equations for u, wic we can write as ui ui + ui + ( Lu ) i = fi, i =,..., N, u0 = ul, un = ur. (3) We can also write tis system more concisely as L u = f. Te system is tridiagonal and easy to solve by standard metods, but it s useful for examining te multigrid solver s aforementioned tasks. Observe te similarity to te player-alignment problem: if we set f 0, te solution u is obviously a straigt line connecting te point (0, u ) wit (, u r ). Te discrete solution is ten a set of values along tis line, eac equal to te average of its two neigbors. We can easily adapt Jacobi s algoritm to Equation 3. Let u (0) denote an initial approximation to u tat satisfies te boundary conditions. n Ten, at te nt iteration, we set u i, i =,, N, to satisfy te it equation in te system in Equation 3, wit te neigboring values taken from te n st iteration: n u n i u ( ) n i u ( ) = + + i fi, i =,..., N. (4) Te convergence beavior of Jacobi s algoritm is identical to its beavior in te player-alignment problem. Denote te error in our approximation after te nt iteration by e n u u n =, (5) n n wit e0 = en = 0 because all our approximations satisfy te boundary conditions. Now, subtract Equation 4 from te equation ui ui ui = + + fi, wic we get by isolating u i in Equation 3. Tis yields n n ( ) ei ei e n ( ) = + + i, (6) wic is exactly like Equation. Similarly, damped Jacobi relaxation, wit a damping factor of /3, is defined by n ( ) n ( ) n n ui + u + i ui = ui +, (7) 3 3 fi and its error propagation properties are identical to tose of te player-alignment problem in Equations 9 troug. Naïve Multiscale Algoritm We next naïvely extend te multiscale algoritm of te player-alignment problem to te D model problem. Te main point of tis exercise is to sow tat in order to know wat to do at te large scales, we must first propagate appropriate information from te small scales to te large scales. For convenience, we still assume N is a power of two. Similarly to te player-alignment process, we begin by performing Jacobi s iteration for te problem defined at te largest scale, wic consists of te variables [ u0, u. Te mes size at tis scale is 0.5, and u N /, un ] 0 and u N are prescribed by te boundary conditions. Te Jacobi step tus fixes u N / : un / = u0 + un 05. fn / Next, based on te values of te largest scale, we perform a Jacobi iteration at te second-largest scale, obtaining u N /4 and u3 N / 4. We continue until we ave determined all variables, just as in te player-alignment problem. Tis algoritm is obviously computationally efficient. Unfortunately, it fails to give te correct answer (unless f is trivial) because we aren t solving te rigt equations on te coarse grids. More precisely, te rigt-and sides are wrong.. NOVEMBER/DECEMBER 006 7
(a) (b) Figure 5. Aliasing. Red disks correspond to te coarse grid: (a) v (3) and (b) v (3). Aliasing Figure 5a sows a smoot eigenvector (k = 3) sampled on a grid wit N = 6 and also on a twicecoarser grid, N = 8, consisting of te fine grid s even-indexed mes points. Tis repeats in Figure 5b, but for a roug eigenvector (k = 3 on te finer grid), wit coefficient. Evidently, te two are indistinguisable on te coarse grid, wic demonstrates te well-known penomenon of aliasing. Te coarse grid affords rougly alf as many eigenvectors as te fine grid, so some pairs of fine-grid eigenvectors must alias. Specifically, by Equation 5, te even-indexed terms in eigenvectors v (k) and v (N k) are equal to te negative of eac oter. Tus, if f contains some rougness tat is, if we write f as a linear combination of eigenvectors, it will ave roug eigenvectors wit nonzero coefficients it won t be represented correctly on te coarse grid. As a somewat extreme example, if f is a linear combination of v (k) and v (N k) for some k, wit weigts of equal magnitude but opposite sign, it will vanis wen sampled on te coarser grid. Te definition of te smooting factor, wic was based on a partitioning of te spectrum into smoot and roug parts, foresadowed our use of a twice-coarser grid. Smoot eigenvectors are tose tat can be represented accurately on te twice-coarser grid, wereas roug eigenvectors alias wit smoot ones. We can conclude tat constructing correct equations for coarse grids requires information from te fine grid. For te D model problem, we can get around te aliasing problem by successive local averaging. However, in more general problems, we can t do tis witout spending a substantial computational effort, wic would defeat te purpose of te multiscale approac. We must come to grips wit te fact tat coarse grids are useful only for representing smoot functions. Tis means tat we must take care in coosing our coarse-grid variables. Taxonomy of Errors We ve seen tat damped Jacobi relaxation smootes te error. In tis context, it is important to distinguis between two distinct types of error. Te discretization error, given at mes point i by ux ( i) ui, is te difference between te solution of te differential equation (sampled on te grid) and tat of te discrete system. Tis error is determined by te problem, te discretization, and te mes size. Iteration as no effect on tis error, of course, because it only affects our approximation to u. Only te algebraic error, wic is te difference between te discrete solution and our current approximation to it (Equation 5), is smooted. Our plan, terefore, is to first apply damped Jacobi relaxation, wic smootes te algebraic error, and ten construct a linear system for tis error vector and approximate it on a coarser grid. Tis can continue recursively, suc tat, on eac coarser grid, we solve approximate equations for te next-finer grid s algebraic error. Te motivation for using te algebraic error as our coarse-grid variable is twofold. Because we re limited in our ability to compute an accurate approximation to u on te coarse grid due to aliasing, we need to resort to iterative improvement. We sould terefore use te coarse grid to approximately correct te fine-grid algebraic error, rater tan to dictate te fine-grid solution. Moreover, relaxation as smooted te algebraic error, and tus we can approximate it accurately on te coarse grid. Let ũ denote our current approximation to u after performing several damped Jacobi relaxations, and let e = u u denote te corres- ponding algebraic error, wic is known to be smoot (see Figure 4). Subtract from Equation 3 te trivial relation = = Lu Lu, i,..., N, i i u 0 = ul, u N = ur, to obtain a system for te algebraic error, 8 COMPUTING IN SCIENCE & ENGINEERING
= = Le fi Lu ri, i,... N, i i e0 = 0, en = 0, were r i is te it residual. Tis is te problem we must approximate on te coarser grid. We can write it concisely as L e = r. Once we compute a good approximation to e, we ten add it to ũ, and tus improve our approximation to u. Finest grid Relaxation Multigrid Algoritm To transfer information between grids, we need suitable operators. Consider just two grids: te fine grid, wit mes size and wit N mes points excluding boundary points, and te coarse grid, wit mes size and N/ mes points. We use a restriction operator, denoted I, for fine-to-coarse data transfer. Common coices are injection, defined for a fine-grid vector, g, by = =, N I g gi, i,..., i tat is, te coarse-grid value is simply given by te fine-grid value at te corresponding location, or full weigting, wic is defined by = ( + + ) I g gi gi i +, i 4 N i =,...,, were we apply a local averaging. For coarse-to-fine data transfer, we use a prolongation operator, denoted by I. A common coice is te usual linear interpolation, defined for a coarse-grid vector g by gi /, if i is even, ( I g ) = i g i ( ( + )/ + g( i )/ ), if i is odd, i =,..., N. Additionally, we must define an operator on te coarse grid, denoted L, tat approximates te fine grid s L. A common coice is to simply discretize te differential equation on te coarse grid using a sceme similar to te one applied on te fine grid. Armed wit tese operators, and using recursion to solve te coarse-grid problem approximately, we can now define te multigrid V-Cycle (see Figure 6). We require two parameters, commonly denoted by and, tat designate, respectively, te number of damped Jacobi relaxations performed before and after we apply te coarse-grid correction. Te multigrid algoritm is defined as Coarsest grid Algoritm V-Cycle: u = VCycle(u,, N, f,, ). If N ==, solve L u = f and return u ;. u = Relax(u,, N, f, ); relax times. 3. r = f L u ; compute residual vector. 4. f = I r ; restrict residual. 5. u = 0; set u to zero, including boundary conditions. 6. u = VCycle(u,, N/, f,, ); treat coarse-grid problem recursively. 7. u = u + I u ; interpolate and add correction. 8. u = Relax(u,, N, f, ); relax times; 9. Return u. Let s examine te algoritm line by line. In line, we ceck weter we ve reaced te coarsest grid, N =. If so, we solve te problem. Because tere s only a single variable tere, one Jacobi iteration witout damping does te job. In line, damped Jacobi iterations are applied. We compute te residual in line 3, and restrict it to te coarse grid in line 4. We now need to tackle te coarsegrid problem and obtain te coarse-grid approximation to te fine-grid error. Noting tat te coarse-grid problem is similar to te fine-grid one, we apply a recursive call in line 6. Note tat te boundary conditions on te coarse grid are zero (line 5) because tere s no error in te fine-grid boundary values, wic are prescribed. Also, te Restriction Prolongation Figure 6. A scematic description of te V-Cycle. Te algoritm proceeds from left to rigt and top (finest grid) to bottom (coarsest grid) and back up again. On eac grid but te coarsest, we relax times before transferring (restricting) to te next-coarser grid and times after interpolating and adding te correction. NOVEMBER/DECEMBER 006 9
initial approximation to u must be zero. After returning from te recursion, we interpolate and add te correction to te fine-grid approximation (line 7). In line 8, we relax te fine-grid equations times, and te resulting approximation to te solution is returned in line 9. Generally, te V-Cycle doesn t solve te problem exactly, and it needs to be applied iteratively. However, as I ll demonstrate later, eac suc iteration typically reduces te algebraic error dramatically. D Model Problem Te multiscale algoritm isn t restricted to one dimension, of course. Consider te D boundaryvalue problem: u u Lu + = x y u = b( x, y), ( x, y) on Ω, f( x, y), ( x, y) inω, (8) were f and b are given functions and u is te solution we re seeking. For simplicity, our domain is te unit square, = (0, ), wit denoting te boundary of. Tis is te well-known Poisson problem wit Diriclet boundary conditions. To discretize te problem, we define a uniform grid wit mes size and mes points given by ( xi, yj ) = ( i, j), i, j = 0,,..., N, were = /N. Our vector of unknowns te discrete approximation to u is again denoted u, b denotes te discrete boundary conditions and f is te discrete rigt-and side function. Te standard second-order-accurate finite difference discretization now yields a system of equations for u, wic we can write as ui, j + ui+, j + ui, j + u i, j+ 4ui, j ( Lu ) i, j = fi, j, i, j =,... N, ui, j= bi, j, i = 0 or i = N or j = 0 or j = N, (9) or more concisely as L u = f. We can easily adapt Jacobi s algoritm to te D problem: at eac iteration, we cange our approximation to u i, j, i, j =,, N, to satisfy te (i, j)t equation in te system in Equation 9, wit te neigboring values taken from te previous iteration. We can generalize our smooting analysis of damped Jacobi relaxation to tis case. By minimizing te smooting factor wit respect to te damping, we ten find tat te best acievable smooting factor is 0.6, obtained wit a damping factor of 0.8. To apply te multigrid V-Cycle to te D model problem, we still need to define our coarse grids, along wit appropriate intergrid transfer operators. We coarsen naturally by eliminating all te mes points wit eiter i or j odd, suc tat rougly onefourt of te mes points remain. For restriction, we can once again use simple injection, defined for a fine-grid vector, g, by = = N I g gi, j, i, j,...,, i, j tat is, te coarse-grid value is simply given by te fine-grid value at te corresponding location, or full weigting, defined by = ( + I g gi, j gi, j+ i, j 6 + gi +, j + gi+, j+ gi, j + g i+, j + + gi, j + gi, j+, + 4g i, j N i, j =,...,, were a local averaging is applied. For coarse-tofine data transfer, a common coice is bilinear interpolation, defined for a coarse-grid vector g by I g = i, j g i /, j /, if i is even and j is even, g ( (i +)/, j / + g (i )/, j / ), if i is odd and j is even, g ( i /,( j +)/ + g i /,( j )/ ), if i is even and j is odd, 4 g ( (i +)/,( j +)/ + g (i )/, (j+)/ + g (i +)/,( j )/ + g (i )/,( j )/ ) if i is odd and j is odd, for i =,, N. 0 COMPUTING IN SCIENCE & ENGINEERING
Numerical Tests To demonstrate te multigrid algoritm s effectiveness, we test it on te D and D model problems wit N = 8 and compare its convergence beavior to tat of simple Jacobi relaxation. We apply V-Cycles wit =, = denoted V(, ) to a problem wit a known solution, beginning wit a random initial guess for te solution. (It s easy to sow tat te asymptotic convergence beavior is independent of te particular solution.) We use (bi)linear interpolation and fullweigting restriction and plot te fine-grid algebraic error s root mean square (RMS) versus te number of iterations. Figure 7 sows te results for te D and D model problems. Te V- Cycle s computational cost is similar to tat of a few Jacobi iterations, as I will sow in te next section. Hence, for fairness, we consider every 0 Jacobi relaxations as a single iteration in te comparison, so tat 00 iterations marked in te plot actually correspond to,000 Jacobi relaxations. Tus, a V-Cycle in tese plots is no more costly tan a single Jacobi iteration. We can see tat eac V-Cycle reduces te error by rougly an order of magnitude, wereas Jacobi relaxation converges slowly. Te smooting analysis can approximately predict te fast multigrid convergence: on te fine grid, te reduction factor of roug errors per eac relaxation sweep is bounded from above by te smooting factor, wereas te smoot errors are nearly eliminated altogeter by te coarse-grid correction. Computational Complexity Te number of operations performed in te V- Cycle on eac grid is clearly proportional to te number of mes points because tis olds for eac subprocess relaxation, residual computation, restriction, and prolongation. Because te number of variables at eac grid is approximately a constant fraction of tat of te next finer grid (onealf in D and one-fourt in D), te total number of operations per V-Cycle is greater tan te number of operations performed on just te finest grid only by a constant factor. In D, tis factor is bounded by + / + /4 + =, and in D by + /4 + /6 + = 4/3. Te amount of computation required for a single Jacobi relaxation on te finest grid (or a calculation of te residual, wic is approximately te same) is called a work unit (WU). Our V-Cycle requires tree WUs for relaxation on te finest grid and one more for computing te residual. Te prolongation plus te restriction operations togeter require no more tan one WU. Tus, te fine-grid work is rougly Error norm (a) Error norm (b) 0 0 0 5 0 0 0 5 0 0 0 5 0 0 0 5 0 0 40 60 80 00 Iterations 0 0 40 60 80 00 Iterations five WUs per V-Cycle. Terefore, te entire work is approximately 0 WUs per V-Cycle in D and seven in D. V(,) cycles Ten Jacobi relaxations V(,) cycles Ten Jacobi relaxations Figure 7. Convergence of te algebraic error. Multigrid V(, ) cycles versus Jacobi iterations comprised of 0 relaxations eac, wit N = 8: (a) D and (b) D. Finest grid Coarsest grid Relaxation No relaxation Restriction Prolongation Figure 8. A scematic description of te full multigrid algoritm for =. Te algoritm proceeds from left to rigt and top (finest grid) to bottom (coarsest grid.) Initially, we transfer te problem down to te coarsest grid. We solve te problem on te coarsest grid and ten interpolate tis solution to te second-coarsest grid and perform a V-Cycle. Tese two steps are repeated recursively to finer and finer grids, ending wit te finest grid. NOVEMBER/DECEMBER 006
Finally, observe tat on eac grid in te ierarcy, te operations (relaxation, prolongation and restriction) can be performed in parallel. Tus, te number of sequential steps required by te V-Cycle is proportional to te number of grids, wic is merely O(log N). Te Full Multigrid Algoritm In te numerical experiments, we ve used a random initial guess for te solution for testing purposes. In practice, we want to enlist te best possible initial guess. A natural and effective approac is to first solve te problem approximately on a coarser grid and interpolate tis solution to te fine grid to be used as a first approximation. Tis is applied recursively, yielding te full multigrid (FMG) algoritm. (Figure 8 gives a scematic description.) We need a positive parameter,, wic denotes te number of V-Cycles applied at eac level of te algoritm. Te FMG algoritm is ten defined as Algoritm FMG: u = FMG(, N, f, b,,, ). If N ==, solve L u = f and return u ;. f = I f ; restrict f. 3. b = I ˆ b ; restrict boundary conditions. 4. u = FMG(, N/, f, b,,, ); recursive call; 5. u = I ˆ u ; interpolate initial approximation, excluding boundary conditions. 6.Repeat times: u = VCycle(u,, N, f,, ); apply V-Cycles. 7. Return u. Here, Î is a restriction operator for te boundary values, and Î is a prolongation operator tat migt differ from te one we use in te V-Cycle. Generally, it must be more accurate, so a common coice is (bi)cubic interpolation. Recall te distinction we made earlier between te discretization and algebraic errors. Te total error is te sum of tese two errors tat is, te difference between te solution to te differential equation (sampled on te fine grid) and te current discrete approximation. Tis is te relevant measure of te current approximation s accuracy. Once te algebraic error becomes smaller tan te discretization error, te latter begins to dominate te total error. Clearly, no advantage exists in working ard to reduce te algebraic error muc furter. Indeed, if we want to spend an additional computational effort at tis point, we sould reduce te discretization error by appealing to a yet finer grid. For many problems, researcers ave sown tat te FMG algoritm can yield small algebraic errors compared to te discretization error even wit just =. Te FMG algoritm s computational complexity is linear (an FMG algoritm wit = costs about twice as muc as a V-Cycle in D and about one and a tird as muc in D). Te FMG algoritm, terefore, yields an accurate solution to te problem at te cost of just a few WUs typically only several dozen operations per fine-grid variable. Te multigrid algoritms I describe in tis article provide a glimpse into te complex and often subtle macinery of multiscale computational metods. Tese algoritms are immediately useful for simple linear elliptic problems on simple domains. Wit relatively minor modifications, tey can be generalized to andle nonlinear, anisotropic, or moderately inomogeneous problems as well as elliptic systems. Still more elaborate modifications are required to obtain efficient solvers for singularperturbation problems and for problems wit discontinuous coefficients, complex domains, or unstructured grids. Tese topics ave been researced widely over te last 30 years, and several of tem are addressed in tis special issue. Researcers ave resolved many of te difficulties, wile oters remain open and subject to researc. Exploiting tese ideas outside te realm of differential equations is particularly callenging. Wile te details of te metods tat turn out to be optimal for different problems vary widely, te concepts and fundamental tasks I describe in tis article are at te basis of most efficient multiscale approaces. Reference. I.A. Wagner and A.M. Bruckstein, Row Straigtening via Local Interactions, Circuits, Systems, and Signal Processing, vol. 6, no. 3, 997, pp. 87 305. Irad Yavne is a professor in te Faculty of Computer Science, Tecnion Israel Institute of Tecnology. His researc interests include multiscale computational tecniques, scientific computing and computational pysics, image processing and analysis, and geopysical fluid dynamics. Yavne as a PD in applied matematics from te Weizmann Institute of Science, Israel. Contact im at irad@cs.tecnion.ac.il. COMPUTING IN SCIENCE & ENGINEERING